Pipeline for Nix - #2209
Open
chinyeungli wants to merge 73 commits into
Open
Conversation
- Get the input and extract content into the from/ codebase. - Build the source and place the built files into the to/ codebase. - Run scans. - Identify sources in from/ that are used in the build. Signed-off-by: Chin Yeung Li <tli@nexb.com>
…ses detected in the codebase #1767 * Introduce new "LICENSE_ISSUE" tag * License deduplication/simplification is not working well; work in progress Signed-off-by: Chin Yeung Li <tli@nexb.com>
This commit is for testing purpose and is definitely not ready. Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
* Use Docker for building instead of Cargo * Accept only one PURL as input * Use the .rlib found in .d files for D2D mapping * Remove unnecessary code Signed-off-by: Chin Yeung Li <tli@nexb.com>
* Add comparison logic * Add tests * Update extra_data fields * Better code organization * etc.. Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
…2101) Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: Rishabh Rohil <rishabhrohil024@gmail.com>
Signed-off-by: Rishabh Rohil <rishabhrohil024@gmail.com>
…2102) Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
…#2106) Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
…2124) Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: OmAnand857 <allansmith2561@gmail.com>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: Aayush Kumar <code@aayushk.dev>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
#2187) Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
…2192) Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: Mrityunjay Raj <mr.raj.earth@gmail.com>
Signed-off-by: Prajakta Kamble <prajuu2812@gmail.com>
Signed-off-by: dikshaa2909 <dikshadeware@gmail.com>
… docstring (#2184) Signed-off-by: Prajakta Kamble <prajuu2812@gmail.com>
…2193) Signed-off-by: tdruez <tdruez@aboutcode.org>
…2194) Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: tdruez <tdruez@aboutcode.org>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
- Merge branch '1767_dedicalted_pipeline_for_rust' into 1938_pipeline_for_nix without merging rsut.py and scan_rust_package.py - Update d2d matching to include <name>.h/.c to <name>.h.in/.c.in Signed-off-by: Chin Yeung Li <tli@nexb.com>
…#1938 - Remove the "rust" code/refernece that's delivered from 1767 Signed-off-by: Chin Yeung Li <tli@nexb.com>
Signed-off-by: Chin Yeung Li <tli@nexb.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issues
Changes
This PR creates a dedicated pipeline for Nix packages.
Requirements
Summary
Notes for deployment to development mapping
As Nix is not tied to a single programming language ecosystem, this pipeline executes all currently supported D2D steps across multiple languages.
Handling Missing Debug Symbols:
For

pkg:nix/nixpkgs/SDL_mixer@1.2.12?system=x86_64-linux, as seen in https://search.devbox.sh/v2/pkg?name=SDL_mixer , there is nodebugbuild found under the "outputs" section. Therefore, even the ELF binary is not stripped:there is no dwarf symbols collected and no development to deployment mapping can be performed
Template File Mapping (.in files):
For
pkg:nix/nixpkgs/openssl@3.6.0?out=debug&system=x86_64-linux, following are some of the dwarf symbols collected:However, these exact header files do not exist in the source codebase; they are generated from GNU Autoconf template files. To complete the mapping, the code has been updated to map to the corresponding .in template files:

Source-Only Scans:

For
pkg:nix/nixpkgs/haskellPackages.aasam@0.2.0.0?system=x86_64-linux&commit=a3ae4cdd64f675cf2580affecdee01c401b43638, this package does not exist on https://search.devbox.sh/, so there is no binary can be fetched, but the code can still fetch and build the sources based on the provided commit hash and run the basic scan:Notes
Because the pipeline relies on a Linux-based Docker container (nixos/nix) to build sources, providing a PURL for a non-Linux system (e.g., darwin) triggers a system barrier warning.
When this happens, the pipeline safely falls back to evaluating the source using the container's native Linux environment. The extracted source tree will contain Linux-specific patches rather than macOS patches. The impact on the d2d mapping is expected to be minimal, though a small number of files may remain unmapped due to missing OS-specific structural patches.
Checklist