Command Line Interface

The scanpipe command can be executed using the Docker Compose command line interface.

If the Docker Compose stack is already running, you can execute the command as follows:

docker compose exec -it web scanpipe COMMAND

If the ScanCode.io services are not currently running, you can use the following command:

docker compose run --rm web scanpipe COMMAND

Additionally, you can start a new Docker container and execute multiple scanpipe commands within a bash session:

docker compose run web bash
scanpipe COMMAND
scanpipe COMMAND
...

Warning

In order to add local input files to a project using the Command Line Interface, extra arguments need to be passed to the docker compose command.

For instance --volume /path/on/host:/target/path/in/container:ro will mount and make available the host path inside the container (:ro stands for read only).

docker compose run --volume /home/sources:/sources:ro \
    web scanpipe create-project my_project --input-file="/sources/image.tar"

Note

In a local development installation, the scanpipe command is directly available as an entry point in your virtualenv and is located at <scancode.io_root_dir>/bin/scanpipe.

$ scanpipe –help

Lists all sub-commands available, including Django built-in commands. ScanPipe’s own commands are listed under the [scanpipe] section:

$ scanpipe --help
...
[scanpipe]
    add-input
    add-pipeline
    archive-project
    create-project
    delete-project
    execute
    list-project
    output
    show-pipeline
    status

$ scanpipe <subcommand> –help

Displays help for the provided sub-command.

For example:

$ scanpipe create-project --help
usage: scanpipe create-project [--input-file INPUTS_FILES]
    [--input-url INPUT_URLS] [--copy-codebase SOURCE_DIRECTORY]
    [--pipeline PIPELINES] [--execute] [--async]
    name

Create a ScanPipe project.

positional arguments:
  name                  Project name.

$ scanpipe create-project <name>

Creates a ScanPipe project using <name> as a Project name. The project name must be unique.

Optional arguments:

--pipeline PIPELINES Pipelines names to add on the project.

Tip

Use the “pipeline_name:group1,group2” syntax to select steps groups:

--pipeline map_deploy_to_develop:Java,JavaScript

--input-file INPUTS_FILES Input file locations to copy in the input/ work directory.

Tip

Use the “filename:tag” syntax to tag input files: --input-file path/filename:tag
--input-url INPUT_URLS Input URLs to download in the input/ work directory.

Tip

Use the “url#tag” syntax to tag downloaded files: --input-url https://url.com/filename#tag
--copy-codebase SOURCE_DIRECTORY Copy the content of the provided source directory into the codebase/ work directory.
--execute Execute the pipelines right after project creation.
--async Add the pipeline run to the tasks queue for execution by a worker instead of running in the current thread. Applies only when --execute is provided.

Warning

Pipelines are added and are executed in order.

$ scanpipe list-project [–search SEARCH] [–include-archived]

Lists ScanPipe projects.

Optional arguments:

--search SEARCH Limit the projects list to this search results.
--include-archived Include archived projects.

Tip

Only the project names are listed by default. You can display more details about each project by providing the --verbosity 2 or --verbosity 3 options.

$ scanpipe add-input –project PROJECT [–input-file FILES] [–input-url URLS]

Adds input files in the project’s work directory.

--input-file INPUTS_FILES Input file locations to copy in the input/ work directory.

Tip

Use the “filename:tag” syntax to tag input files: --input-file path/filename:tag
--input-url INPUT_URLS Input URLs to download in the input/ work directory.

Tip

Use the “url#tag” syntax to tag downloaded files: --input-url https://url.com/filename#tag
--copy-codebase SOURCE_DIRECTORY Copy the content of the provided source directory into the codebase/ work directory.

For example, assuming you have created beforehand a project named “foo”, this will copy ~/docker/alpine-base.tar to the foo project input/ directory:

$ scanpipe add-input --project foo --input-file ~/docker/alpine-base.tar

Warning

Make sure to mount your local sources volume in the Docker setup:

--volume /host/sources:/sources:ro --input-file /sources/image.tar

You can also provide URLs of files to be downloaded to the foo project input/ directory:

$ scanpipe add-input --project foo --input-url https://github.com/nexB/scancode.io-tutorial/releases/download/sample-images/30-alpine-nickolashkraus-staticbox-latest.tar

Note

Docker images can be provided as input using their Docker reference with the docker://docker-reference syntax. For example:

$ [...] --input-url docker://redis
$ [...] --input-url docker://postgres:13
$ [...] --input-url docker://docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2

See https://docs.docker.com/engine/reference/builder/ for more details about references.

$ scanpipe add-pipeline –project PROJECT PIPELINE_NAME [PIPELINE_NAME …]

Adds the PIPELINE_NAME to a given PROJECT. You can use more than one PIPELINE_NAME to add multiple pipelines at once.

Warning

Pipelines are added and are executed in order.

For example, assuming you have created beforehand a project named “foo”, this will add the docker pipeline to your project:

$ scanpipe add-pipeline --project foo analyze_docker_image

Tip

Use the “pipeline_name:group1,group2” syntax to select steps groups:

--pipeline map_deploy_to_develop:Java,JavaScript

$ scanpipe execute –project PROJECT

Executes the next pipeline of the PROJECT project queue.

Optional arguments:

--async Add the pipeline run to the tasks queue for execution by a worker instead of running in the current thread.

$ scanpipe show-pipeline –project PROJECT

Lists all the pipelines added to the PROJECT project.

$ scanpipe status –project PROJECT

Displays status information about the PROJECT project.

Note

The full logs of each pipeline execution are displayed by default. This can be disabled providing the --verbosity 0 option.

$ scanpipe output –project PROJECT –format {json,csv,xlsx,spdx,cyclonedx,attribution}

Outputs the PROJECT results as JSON, XLSX, CSV, SPDX, CycloneDX, and Attribution. The output files are created in the PROJECT output/ directory.

Multiple formats can be provided at once:

$ scanpipe output --project foo --format json xlsx spdx cyclonedx attribution

Optional arguments:

--print Print the output to stdout instead of creating a file. This is not compatible with the XLSX and CSV formats. It cannot be used when multiple formats are provided.

Refer to Mount projects workspace to access your outputs on the host machine when running with Docker.

$ scanpipe archive-project –project PROJECT

Archives a project and remove selected work directories.

Optional arguments:

--remove-input Remove the input/ directory.
--remove-codebase Remove the codebase/ directory.
--remove-output Remove the output/ directory.
--no-input Does not prompt the user for input of any kind.

$ scanpipe reset-project –project PROJECT

Resets a project removing all database entrie and all data on disks except for the input/ directory.

Optional arguments:

--no-input Does not prompt the user for input of any kind.

$ scanpipe delete-project –project PROJECT

Deletes a project and its related work directories.

Optional arguments:

--no-input Does not prompt the user for input of any kind.

$ scanpipe create-user <username>

Note

This command is to be used when ScanCode.io’s authentication system SCANCODEIO_REQUIRE_AUTHENTICATION is enabled.

Creates a user and generates an API key for authentication.

You will be prompted for a password. After you enter one, the user will be created immediately.

The API key for the new user account will be displayed on the terminal output.

User <username> created with API key: abcdef123456

The API key can also be retrieved from the Profile settings menu in the UI.

Warning

Your API key is like a password and should be treated with the same care.

By default, this command will prompt for a password for the new user account. When run non-interactively with the --no-input option, no password will be set, and the user account will only be able to authenticate with the REST API using its API key.

Optional arguments:

--no-input Does not prompt the user for input of any kind.