Compliance Scanner

Quick Start

Run a compliance assessment on the project in current directory, using Xygeni Scanner:

xygeni compliance -n MyProject

which produces an output similar to this:

Stop software supply-chain attacks!               xygeni.io

Running xygeni...
2022-11-09 11:52:09.815 [main] INFO  Xygeni - Start xygeni
2022-11-09 11:52:35.002 [main] INFO  ComplianceEngine - Checkpoints terminated. Duration = 0.000112221s
2022-11-09 11:52:35.008 [main] INFO  ComplianceEngine - Analysis complete in 24.180512222s

Compliance for cis_sscs standard: non-compliant (2.67)
Checkpoints run: 13
┌───────────────────────────────────────┬───────┬─────┬────────┬────────────────────────────────┬───┐
│Checkpoint                             │Status │Level│Severity│Category                        │Op?│
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/branch_deletions_denied       │ fail  │  0  │critical│source_code/code_changes        │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/code_in_vcs                   │ fail  │  0  │critical│source_code/code_changes        │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/force_push_denied             │ fail  │  0  │critical│source_code/code_changes        │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/identity_verified             │ fail  │  0  │critical│source_code/contribution_access │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/minimum_approvals             │ fail  │  0  │critical│source_code/code_changes        │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/anonymous_access              │  ok   │ 10  │critical│artifacts/access_to_artifacts   │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/contain_security_md           │  ok   │ 10  │critical│source_code/repository          │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/minimum_admins_org            │  ok   │ 10  │critical│source_code/contribution_access │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/package_hooks                 │  ok   │ 10  │critical│artifacts/package_hooks         │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/inactive_users                │  ok   │ 10  │  low   │source_code/inactive_users      │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/steps_as_code                 │  ok   │ 10  │  low   │build_pipelines/pipeline_instruc│   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/package_org_mfa               │  na   │  0  │critical│artifacts/package_org_mfa       │   │
├───────────────────────────────────────┼───────┼─────┼────────┼────────────────────────────────┼───┤
│cis_sscs/inactive_branches             │unknown│  0  │  low   │source_code/code_changes        │   │
└───────────────────────────────────────┴───────┴─────┴────────┴────────────────────────────────┴───┘

You may now log in the Xygeni Dashboard and view the Compliance Assessment results.

Purpose

Compliance Assessment checks compliance with Software Supply-Chain Security standards and guidelines.

A standard is a list of checkpoints, arranged in categories. A software project is compliant with a standard ("passes") only when all the standard’s required checkpoints passed.

Checkpoints could be optional. Optional checkpoints are reported, but they do not alter the compliance status. Non-optional checkpoints, on the other hand, are required for compliance with the standard.

The assessment report gives a status (PASS, PARTIAL, FAIL) and a compliance level between 0 (not compliant at all) and 10 (fully compliant).

Standards

See supported standards and guidelines for more details.

Handling results

Remember that for a standard to pass, all its required checkpoints must pass.

A non-compliant checkpoint could be verified following its documented Verification section, and made compliant by following the recommended actions in the Remediation section.

Usage

A Compliance Assessment scan is launched using the compliance command of the xygeni executable.

With --help the usage is displayed:

xygeni compliance --help

Usage:

xygeni compliance [-huV] [-n=<name>]
  [-s=<standard>]
  [-d=<directory>] [-e=<excludePatterns>] [-i=<includePatterns>]
  [--[no-]all-files]
  [-repo=<repo>] [--repo-branch=<repoBranch>]
  [-o=<output>] [-f=<format>]... [--report-columns=<cols>]
  [-c=<config>]
  [--checkpoints=<checkpoints>] [--skip-checkpoints=<checkpoints>]
  [--custom-standards-dir=<customStandardsDir>] [--[no-] conf-download]]
  [--never-fail | --fail-on=<failOnInput>]
  [@<filename>...]


Check compliance with supply-chain standards.

Parameters:
      [@<filename>...]       One or more argument files containing options.
  -n, --name=<name>          The software name.
  -u, --upload               Upload report to Xygeni server.
  -h, --help                 Show this help message and exit.
  -V, --version              Print version information and exit.

Input files options:
  -d, --dir=<directory>      The directory to analyze (default: current
                               directory).
  -i, --include=<includePatterns>
                             Include patterns, comma-separated (optional).
  -e, --exclude=<excludePatterns>
                             Exclude patterns, comma-separated (optional).
                               Example: '**/test/**'
      --[no-]all-files       Scan all files (the default). With --no-all-files,
                             scan tracked files under git only.

Repository options:
      -repo, --repository=<repo>
                             The repository. Either a URL or scm:owner/repo,
                             like 'github:tensorflow/tensorflow'
      --repo-branch=<ref>    The repository branch or commit SHA to checkout for analysis.
                             HEAD if unspecified.

Repository options:
  -repo, --repository=<repo> The repository. Either a URL or scm:owner/repo, like
                             'github:tensorflow/tensorflow'
  --repo-branch=<repoBranch> The repository branch or commit SHA to checkout for
                             analysis. HEAD if unspecified.

Output options:
  -o, --output=<output>      Output file. Use 'stdout' or '-' for standard
                               output, 'stderr' for standard error.
  -f, --format=<format>      Output formats: none, text, json, csv, sarif.
      --report-columns=<reportColumns>
                             Report columns, separated by commas (default:
                               config property report/columns)

Configuration options:
  -s, --standard=<standard>  ID of the standard to check (default from config file).
  -c, --conf=<config>        Configuration file (default: xygeni.compliance. yml).
      --checkpoints=<checkpoints>
                             Comma-separated list of IDs for checkpoints to run.
                             Alternatives: severity:SEV, category:CAT, tag:TAG or 'all'
      --skip-checkpoints=<checkpoints>
                             Comma-separated list of IDs for checkpoints to ignore.
                             Alternatives: severity:SEV, category:CAT or tag:TAG
      --custom-standards-dir=<customStandardsDir>
                             Directory with custom standards and checkpoints.
      --[no-]conf-download   Download scanner config? (default: true}

Exit options:
      --never-fail           Do not fail: always exit with code 0, even with
                               non-compliant checkpoints.
      --fail-on=<failOnInput>
                             Comma-separated list of detector IDs, SEVERITY or GUARDRAIL (expression, file or server reference) that will force a non-zero exit code.
                             Examples:
                               --fail-on 'critical'
                               --fail-on 'guardrail secret-policy on secrets when severity >= high then @exitcode(127)'
                               --fail-on 'file:path-to/my-guardrail.xyflow'
                               --fail-on '#my-guardrail'

Configuration

$XYGENI_HOME denotes here the path where the Xygeni scanner is installed.

The Compliance Assessment Scanner is configured in the YAML file $XYGENI_HOME/conf/xygeni.compliance.yml:

# Configuration for the xygeni compliance scanner

# This is the default standard to use. Overridden by -s/--standard command line argument.
standard: cis_sscs
#standard: openssf_scorecard

# List of checkpoint specs.
# A checkpoint spec is either a checkpoint ID, or a prop:value pair,
# where prop is one of severity, category or tag.
# Leave empty for no restriction (all checkpoints in requested standard and not disabled will be chosen).
# Examples:
#   runCheckpoints: ['openssf_scorecard/binary_artifacts']
#     will run that single checkpoint (when in the chosen standard).
#   runCheckpoints: ['check-01', 'check-02', 'category:branch-protection']
#     will run two explicit checkpoints, plus all in the 'branch-protection' category.
#   runCheckpoints: ['severity:critical', 'severity:high']
#     Will run all checkpoints in the chosen standard with critical or high severity.
# Command-line property --checkpoints overrides this.

# List of checkpoints to run: A list of individual checkpoint IDs, or groups of checkpoints
# given as 'severity:SEV', 'category:CAT' or 'tag:TAG' could be used.
# runCheckpoints: [ 'severity:high', 'category:' ] will run all checkpoints with severity 'high' or greater.
# runCheckpoints: ['openssf_scorecard/binary_artifacts'] will run just that one.
# Command-line property --checkpoints overrides this.
runCheckpoints: []

# Same format as runCheckpoints, but for skipping the selected checkpoints.
# Command-line property --skip-checkpoints overrides this.
skipCheckpoints: []

# The directory containing YAML files for custom standards and checkpoints.
# Leave it null for no default. The --custom-standards-dir command-line argument overwrites this.
customStandardsDir: null

# scanMode: sequential|parallel
mode: sequential

# Includes: list of glob patterns to include in analysis.
includes: []

# Excludes: list of glob patterns to exclude from analysis.
excludes:
  - ".git/**/*"
  # ...
  - "**/.xygeni.*.baseline.json"

report:
# ...

Standards are configured with different YAML files located under the $XYGENI_HOME/conf/compliance/standards directory of the xygeni scanner. Each standard provides some information and the list of checkpoints that are to be checked for compliance assessment.

Checkpoints are configured with different YAML files located under the $XYGENI_HOME/conf/compliance/checkpoints directory of the xygeni scanner.

A custom standard may include checkpoints taken from different standards, if desired.

You might use the Policy Administration tool for editing standards to be enforced at the organization.

To avoid scanner updates overwriting your configurations, you may define a directory where custom standards and checkpoints could be loaded with the --custom-standards-dir command-line argument.

How to…​

This section presents common use-cases for evaluating compliance against a given supply-chain security standard across the software lifecycle.

Exclude certain checkpoints from a standard assessment

There could be certain checkpoints that should not be run for a certain project, or because the practice enforced by the checkpoint does not fit with the organization policy. The list of checkpoints that should be active for a standard to enforce, according to the organization policy, could be set in the Policy page in Xygeni Administration.

Disabling checkpoints could be done, in order of prevalence: - by disabling the intended checkpoint(s), listing their IDs in the --skip-checkpoints option. - with the runCheckpoints / skipCheckpoints properties in the scanner YAML. - at the policy level, - by deactivating the checkpoint at the central configuration level (e.g. by setting enabled: false in the checkpoint YAML configuration).

A common use case is to disable issues with info / low severity (--skip-checkpoints=low) or equivalently enabling only high or critical issues (--checkpoints=high).

Add compliance assessment as a git hook

a) Direct registering script as a git hook

To run Xygeni scanner at client-side, before a commit is applied, you may add a simple pre-commit executable script like this (example for Linux and macOS):

#!/usr/bin/env bash

RED='\033[1;31m' # Bold red
GRAY='\033[1;90m' # Bold gray
NC='\033[0m' # No Color

# Add Xygeni scanner location to the PATH (if not in PATH already)
PATH=${PATH}:${XYGENI_HOME}

if ! [ -x "$(command -v xygeni)" ]
then
    echo -e "${RED}Xygeni scanner could not be found.${NC} Please make sure Xygeni is installed properly.\nDocumentation can be found here: https://docs.xygeni.io/scanner/xygeni_scanner.html#_installation"
    exit 1
fi

# Run secrets scan on the files changed in the commit ("staged") only
# "no new secrets of high+ severity":
xygeni -q secrets -d . --staged-files --fail-on=high --no-stdin --format text --output ./.secrets.txt
EXIT_CODE=$?

if [[ $EXIT_CODE -gt 127 ]]; then
  # Print secrets found if anu
  echo -e "${RED}Secrets found !${NC} Please remove them before committing changes."
  cat ./.secrets.txt
elif [[ $EXIT_CODE -ne 0 ]]; then
  echo -e "${RED}Xygeni scan failed.${NC} Please contact your Xygeni administrator."
  echo -e "Log files can be found in ${GRAY}logs/noproject/secrets_noproject.log${NC}"
fi

# Cleanup and tell the git commit to stop (not 0) or continue (0)
rm -f ./.secrets.txt >/dev/null 2>&1
exit $EXIT_CODE

The --fail-on option is configured to make the build fail when at least one issue with critical or high severity is found (so the scanner acts as a security gate).

For running the scan as an optional pipeline check, use --fail-on=never instead.

The --no-stdin option is recommended to avoid the scanner to get the files to process from the standard input. Under some environments, this could make the scanner to analyze no files at all.

This example is a similar security gatekeeper, but commits are allowed only when no scan of the given types has a critical issue (only the scanner execution is shown, for brevity):

#!/usr/bin/env bash

${XYGENI_HOME}/xygeni scan --fail-on=critical --no-upload --run=secrets,misconf,iac \
  --format=text --output=.report.txt

When a critical issue is detected in any of the scans for secrets, misconfigurations or IaC flaws, the commit will be aborted.

Use compliance command to limit the scope of the scan to compliance assessment.

b) Using pre-commit framework

There are some popular frameworks for improving the hook mechanism provided by Git, mainly aimed at sharing the scripts and simplifying the configuration when multiple actions need to be added for a git event.

One of the most popular frameworks is pre-commit. The list of actions required are listed in a YAML file, and `pre-commit manages the installation and execution of scripts written in any language.

Follow the instructions given in pre-commit quick start for installing the framework. Essentially you need python and pip, and run pip install pre-commit in the node where the git commit will be intercepted.

Then add the following entry to you .pre-commit-config.yaml:

# ... Other configuration properties ...
repos:
  # ... more repos and hooks ...

  - repo: https://github.com/xygeni/xygeni-action
    rev: v2.1
    hooks:
    - id: xygeni
      stages: [commit]
      args: [
        '-q', 'scan', '--format=text',
        '--run=secrets,suspectdeps,misconf,iac,codetamper,compliance',
        '--fail-on=critical'] 

Scan configuration with the args property.

Or, for running the scanner Docker image:

- repo: git://github.com/xygeni/xygeni-action
  rev: v1.2
  hooks:
  - id: xygeni-docker
    stages: [commit]
    args: [
      '-q', 'scan', '--format=text',
      '--run=secrets,suspectdeps,misconf,iac,codetamper,compliance',
      '--fail-on=critical']

Then install the git hook scripts using

# You may test the hook configuration with
#   pre-commit try-repo or pre-commit run
pre-commit install

Xygeni scanner will run before each commit. The configuration given above works for setting Xygeni as a security gate to avoid commits having critical security issues. To avoid break the builds, --fail-on=never could be used instead.

git commit -m "this commit contains issues!"

  Running xygeni (script) scan ........Failed!
  ...

To disable the pre-commit hook, for example when after analysis the findings found were deemed as false positives or on inactive / no-risk issues, use SKIP=xygeni in front of the git command: ---

SKIP=xygeni git commit -m "this commit contains issues!"
    xygeni SKIPPED...

Use compliance command to limit the scope of the scan to compliance assessment.

Add compliance assessment to a CI step

You may use compliance results to block a commit.

Under GitHub, you may use Xygeni GitHub Action, running the compliance command only, and with fail_on argument configured to fail the build only when important checkpoints do not pass:

jobs:
  xygeni-scan:
    runs-on: ubuntu-latest
    name: xygeni-github-action
    steps:
      # Checkout the repository sources (GITHUB_WORKSPACE)
      - name: Checkout
        uses: actions/[email protected]
        fetch-depth: 0

      # Other steps (not shown)

      # Security gate checking compliance with CIS SSCS standard
      # Build fails when any critical checkpoint in standard FAILs
      - name: Xygeni-Scanner
        uses: xygeni/[email protected]
        id: Xygeni-Scanner
        with:
          token: ${{ secrets.XYGENI_TOKEN }}
          run: compliance
          fail_on: 'severity:critical'
          standard: cis_sscs

The fail_on is configured to make the build fail when at least one critical checkpoint do not pass. For having the compliance check as optional, use fail_on: never instead.

Define a custom standard

A new standard may be created from existing standards by simply listing the new standard’s checkpoints in `$XYGENI_HOME/conf/compliance/standards/<my_new_standard>.yml:

# My Software Supply-Chain Security standard
id: my_new_standard

description: >-
  The XYZ standard provides prescriptive guidance for establishing
  a secure configuration posture for Software Development Platforms and Pipelines.

url: https://...

checkpoints:
  - ID_CHECKPOINT_1
  - ID_CHECKPOINT_2
  # ...

Where ID_CHECKPOINT_1, ID_CHECKPOINT_2…​ are the checkpoint identifier. The configuration for each checkpoint is loaded from <custom-standards-dir>/<ID_CHECKPOINT>.yml file, if available, or from a resource loaded from compliance/checkpoints/<ID_CHECKPOINT>.yml path.

The easiest way to define your own standard is using the Policy Administration. This allows to mix-and-match checkpoints from different standards into a new one, possibly changing checkpoint severities and its _optional flag.

An alternative option is to create your standard’s my_standard.yml file, located in a specific $XYGENI_HOME/conf/compliance/custom_standards directory, testing it, and then upload the configuration using the central configuration.

Command-line example:

xygeni -q compliance -n NAME -d DIR --format=text --output=stdout --upload \
  --standard=my_standard --custom-standards-dir=$XYGENI_HOME/conf/compliance/my_standards