Changelog

v34.4.0 (unreleased)

  • Add support for CycloneDX 1.6 outputs and inputs. Also, the CycloneDX outputs can be downloaded as 1.6, 1.5, and 1.4 spec versions.

v34.3.0 (2024-04-10)

v34.2.0 (2024-03-28)

v34.1.0 (2024-03-27)

  • Add support for importing CycloneDX SBOM 1.2, 1.3, 1.4 and 1.5 spec formats. https://github.com/nexB/scancode.io/issues/1045

  • The pipeline help modal is now available from all project views: form, list, details. The docstring are converted from markdown to html for proper rendering. https://github.com/nexB/scancode.io/pull/1105

  • Add a new CollectSymbols pipeline (addon) for collecting codebase symbols using Universal Ctags. https://github.com/nexB/scancode.io/pull/1116

  • Capture errors during the inspect_elf_binaries pipeline execution. Errors on resource inspection are stored as project error message instead of global pipeline failure. The problematic resource path is stored in the message details and displayed in the message list UI as a link to the resource details view. https://github.com/nexB/scancode.io/issues/1121 https://github.com/nexB/scancode.io/issues/1122

  • Use the package_only option in scancode get_package_data API in inspect_packages pipeline, to skip license and copyright detection in extracted license and copyright statements found in package metadata. https://github.com/nexB/scancode-toolkit/pull/3689

  • Rename the match_to_purldb pipeline to match_to_matchcode, and add MatchCode.io API settings to ScanCode.io settings.

  • In the DiscoveredPackage model, rename the “datasource_id” attribute to “datasource_ids” and add a new attribute “datafile_paths”. This is aligned with the scancode-toolkit Package model, and package detection information is now stored correctly. Also update the UI for discovered packages to show the corresponding package datafiles and their datasource IDs. A data migration is included to facilitate the migration of existing data. https://github.com/nexB/scancode.io/issues/1099

  • Add PurlDB tab, displayed when the PURLDB_URL settings is configured. When loading the package details view, a request is made on the PurlDB to fetch and and display any available data. https://github.com/nexB/scancode.io/issues/1125

  • Create a new management command purldb-scan-queue-worker, that runs scancode.io as a Package scan queue worker for PurlDB. purldb-scan-queue-worker gets the next available Package to be scanned and the list of pipeline names to be run on the Package from PurlDB, creates a Project, fetches the Package, runs the specified pipelines, and returns the results to PurlDB. https://github.com/nexB/scancode.io/pull/1078 https://github.com/nexB/purldb/issues/236

  • Update matchcode-toolkit to v4.0.0

v34.0.0 (2024-03-04)

v33.1.0 (2024-02-02)

  • Rename multiple pipelines for consistency and precision:

    • docker: analyze_docker_image

    • root_filesystems: analyze_root_filesystem_or_vm_image

    • docker_windows: analyze_windows_docker_image

    • inspect_manifest: inspect_packages

    • deploy_to_develop: map_deploy_to_develop

    • scan_package: scan_single_package

    A data migration is included to facilitate the migration of existing data. Only the new names are available in the web UI but the REST API and CLI are backward compatible with the old names. https://github.com/nexB/scancode.io/issues/1044

  • Generate CycloneDX SBOM in 1.5 spec format, migrated from 1.4 previously. The Package vulnerabilities are now included in the CycloneDX SBOM when available. https://github.com/nexB/scancode.io/issues/807

  • Improve the inspect_manifest pipeline to accept archives as inputs. https://github.com/nexB/scancode.io/issues/1034

  • Add support for “tagging” download URL inputs using the “#<fragment>” section of URLs. This feature is particularly useful in the map_develop_to_deploy pipeline when download URLs are utilized as inputs. Tags such as “from” and “to” can be specified by adding “#from” or “#to” fragments at the end of the download URLs. Using the CLI, the uploaded files can be tagged using the “filename:tag” syntax while using the –input-file arguments. In the UI, tags can be edited from the Project details view “Inputs” panel. On the REST API, a new upload_file_tag field is available to use along the upload_file. https://github.com/nexB/scancode.io/issues/708

v33.0.0 (2024-01-16)

v32.7.0 (2023-10-25)

v32.6.0 (2023-08-29)

v32.5.2 (2023-08-14)

Security release: This release addresses the security issue detailed below. We encourage all users of ScanCode.io to upgrade as soon as possible.

  • GHSA-6xcx-gx7r-rccj: Reflected Cross-Site Scripting (XSS) in license endpoint The license_details_view function was subject to cross-site scripting (XSS) attack due to inadequate validation and sanitization of the key parameter. The license views were migrated class-based views are the inputs are now properly sanitized. Credit to @0xmpij for reporting the vulnerability. https://github.com/nexB/scancode.io/security/advisories/GHSA-6xcx-gx7r-rccj https://github.com/nexB/scancode.io/issues/847

  • Add bandit analyzer and Django “check –deploy” to the check/validation stack. This helps to ensure that we do not introduce know code vulnerabilities and deployment issues to the codebase. https://github.com/nexB/scancode.io/issues/850

  • Migrate the run_command function into a safer usage of the subprocess module. Also fix various warnings returned by the bandit analyzer. https://github.com/nexB/scancode.io/issues/850

  • Replace the scancode.run_scancode function by a new run_scan that interact with scancode-toolkit scanners without using subprocess. This new function is used in the scan_package pipeline. The SCANCODE_TOOLKIT_CLI_OPTIONS settings was renamed SCANCODE_TOOLKIT_RUN_SCAN_ARGS. Refer to the documentation for the next “dict” syntax. https://github.com/nexB/scancode.io/issues/798

v32.5.1 (2023-08-07)

Security release: This release addresses the security issue detailed below. We encourage all users of ScanCode.io to upgrade as soon as possible.

—

v32.5.0 (2023-08-02)

WARNING: After upgrading the ScanCode.io codebase to this version, and following the docker compose build, the permissions of the /var/scancodeio/ directory of the Docker volumes require to be updated for the new app user, using: docker compose run -u 0:0 web chown -R app:app /var/scancodeio/

v32.4.0 (2023-07-13)

v32.3.0 (2023-06-12)

v32.2.0 (2023-04-25)

v32.1.0 (2023-03-23)

v32.0.1 (2023-02-20)

v32.0.0 (2022-11-29)

  • Add a new “find vulnerabilities” pipeline to lookup vulnerabilities in the VulnerableCode database for all project discovered packages. Vulnerability data is stored in the extra_data field of each package. More details about VulnerableCode at https://github.com/nexB/vulnerablecode/ https://github.com/nexB/scancode.io/issues/101

  • Add a new “inspect manifest” pipeline to resolve packages from manifest, lockfile, and SBOM. The resolved packages are created as discovered packages. Support PyPI “requirements.txt” files, SPDX document as JSON “.spdx.json”, and AboutCode “.ABOUT” files. https://github.com/nexB/scancode.io/issues/284

  • Generate SBOM (Software Bill of Materials) compliant with the SPDX 2.3 specification as a new downloadable output. https://github.com/nexB/scancode.io/issues/389

  • Generate CycloneDX SBOM (Software Bill of Materials) as a new downloadable output. https://github.com/nexB/scancode.io/issues/389

  • Display Webhook status in the Run modal. The WebhookSubscription model was refined to capture delivery data. https://github.com/nexB/scancode.io/issues/389

  • Display the current active step of a running pipeline in the “Pipeline” section of the project details view, inside the run status tag. https://github.com/nexB/scancode.io/issues/300

  • Add proper pagination for API actions: resources, packages, dependencies, and errors.

  • Refine the fields ordering in API Serializers based on the toolkit order. https://github.com/nexB/scancode.io/issues/546

  • Keep the current filters state when submitting a search in list views. https://github.com/nexB/scancode.io/issues/541

  • Improve the performances of the project details view to load faster by deferring the the charts rendering. This is especially noticeable on projects with a large amount of codebase resources and discovered packages. https://github.com/nexB/scancode.io/issues/193

  • Add support for filtering by “Other” values when filtering from the charts in the Project details view. https://github.com/nexB/scancode.io/issues/526

  • CodebaseResource.for_packages now returns a list of DiscoveredPackage.package_uid or DiscoveredPackage.package_url if DiscoveredPackage.package_uid is not present. This is done to reflect the how scancode-toolkit’s JSON output returns package_uid``s in the ``for_packages field for Resources.

  • Add the model DiscoveredDependency. This represents Package dependencies discovered in a Project. The scan_codebase and scan_packages pipelines have been updated to create DiscoveredDepdendency objects. The Project API has been updated with new fields:

    • dependency_count - The number of DiscoveredDependencies associated with the project.

    • discovered_dependencies_summary - A mapping that contains following fields:

      • total - The number of DiscoveredDependencies associated with the project.

      • is_runtime - The number of runtime dependencies.

      • is_optional - The number of optional dependencies.

      • is_resolved - The number of resolved dependencies.

    These values are also available on the Project view. https://github.com/nexB/scancode.io/issues/447

  • The dependencies field has been removed from the DiscoveredPackage model.

  • Create directory CodebaseResources in the rootfs pipeline. https://github.com/nexB/scancode.io/issues/515

  • Add ProjectErrors when the DiscoveredPackage could not be fetched using the provided package_uid during the assemble_package step instead of failing the whole pipeline. https://github.com/nexB/scancode.io/issues/525

  • Escape paths before using them in regular expressions in CodebaseResource.walk(). https://github.com/nexB/scancode.io/issues/525

  • Disable multiprocessing and threading by default on macOS (“spawn” start method). https://github.com/nexB/scancode.io/issues/522

v31.0.0 (2022-08-25)

v30.2.0 (2021-12-17)

v30.1.1 (2021-11-23)

v30.1.0 (2021-11-22)

  • Synchronize QUEUED and RUNNING pipeline runs with their related worker jobs during worker maintenance tasks scheduled every 10 minutes. If a container was taken down while a pipeline was running, or if pipeline process was killed unexpectedly, that pipeline run status will be updated to a FAILED state during the next maintenance tasks. QUEUED pipeline will be restored in the queue as the worker redis cache backend data is now persistent and reloaded on starting the image. Note that internaly, a running job emits a “heartbeat” every 60 seconds to let all the workers know that it is properly running. After 90 seconds without any heartbeats, a worker will determine that the job is not active anymore and that job will be moved to the failed registry during the worker maintenance tasks. The pipeline run will be updated as well to reflect this failure in the Web UI, the REST API, and the command line interface. https://github.com/nexB/scancode.io/issues/130

  • Enable redis data persistence using the “Append Only File” with the default policy of fsync every second in the docker-compose. https://github.com/nexB/scancode.io/issues/130

  • Add a new tutorial chapter about license policies and compliance alerts. https://github.com/nexB/scancode.io/issues/337

  • Include layers in docker image data. https://github.com/nexB/scancode.io/issues/175

  • Fix a server error on resource details view when the compliance alert is “missing”. https://github.com/nexB/scancode.io/issues/344

  • Migrate the ScanCodebase pipeline from scancode.run_scancode subprocess to scancode.scan_for_application_packages and scancode.scan_for_files. https://github.com/nexB/scancode.io/issues/340

v30.0.1 (2021-10-11)

v30.0.0 (2021-10-8)

v21.9.6

v21.8.2

v21.6.10

v21.5.12

  • Adds a new way to fetch docker images using skopeo provided as a plugin using docker:// reference URL-like pointers to a docker image. The syntax is docker://<docker image> where <docker image> is the string that would be used in a “docker pull <docker image>” command. Also rename scanpipe.pipes.fetch.download() to fetch_http() https://github.com/nexB/scancode.io/issues/174

  • Pipeline status modals are now loaded asynchronously and available from the project list view.

  • Fix an issue accessing codebase resource content using the scan_codebase and load_inventory pipelines. https://github.com/nexB/scancode.io/issues/147

v21.4.28

v21.4.14

v21.4.5

v1.1.0 (2021-02-16)

v1.0.7 (2021-02-01)

v1.0.6 (2020-12-23)

v1.0.5 (2020-12-07)

v1.0.4 (2020-11-17)

v1.0.3 (2020-09-24)

v1.0.2 (2020-09-18)

v1.0.1 (2020-09-12)

v1.0.0 (2020-09-09)

  • Initial release