Scanning in Google Cloud Build by using Synopsys Detect

Table of Contents


Introduction

This page describes how to integrate the Black Duck scan client into a Google Cloud Build workflow.  By following the process, you can ensure that newly-built containers are automatically scanned as part of a post-build custom step.  For more information, refer to Google Cloud Build's documentation about build steps and custom build steps.

The Black Duck integration with Google Cloud Build leverages a special Docker image that contains Synopsys Detect, which is publicly available in the Google Container Registry.  To invoke Synopsys Detect in Google Cloud Build, modify your Google Cloud Build JSON/YAML build configuration scripts to invoke Synopsys Detect as part of a custom post-build step. To maximize security, Synopsys recommends protecting your Black Duck credentials by using Google's Key Management Service.

Before you start

You must have completed the steps to set up your environment to use Google Cloud Build, which includes the following steps:

  • Creating a project
  • Enabling billing
  • Enabling the Cloud Build API
  • Installing the Cloud SDK

For more information about these steps, refer to the Before you begin topic in the Cloud Build Quickstart Guides.

Google Cloud Build service account

When you run a custom Google Cloud Build step, the step uses a service account.  This account is created when the Cloud Build API is enabled. Ensure that the service account is displayed in the IAM (Identity and Access Management) screen, and that it is assigned the Cloud Build role. The service account has the format:
<Project number>@cloudbuild.gserviceaccount.com

Encrypting Black Duck credentials by using KMS

Synopsys Detect sends scan data to Black Duck, which processes and displays scan results.  For Synopsys Detect to connect to Black Duck, it must authenticate by using API tokens. To maximize security, Synopsys recommends that you use Google's Key Management Service to securely store your API Token.  Click here for instructions about adding credentials to Google Cloud's Key Management Service.

Creating the YAML/JSON build request and submitting it to Google Cloud Build

Google Cloud Build requires a build configuration file in either YAML or JSON format to instruct it on what to build.  To invoke a Black Duck scan, this configuration file must include a custom step that provides information about the scan client.  
The configuration stanzas specify the following information:

  • Target location of the image to be scanned. 
  • User name for Black Duck.
  • Reference to the encrypted credentials in KMS.
  • Name of the project in Black Duck that stores the scan information. 

The following file is an example of a build specification file in YAML format.

Sample build specification YAML step for Black Duck
- name: 'gcr.io/cloud-marketplace/blackduck-public/synopsys-cloudbuild-scanner'
  secretEnv: ['BD_TOKEN']
  args:
  - '--blackduck.url'
  - '<<Black Duck URL>>'
  - '--blackduck.api.token'
  - '$$BD_TOKEN'
  - '--blackduck.trust.cert'
  - 'true'
  - '--detect.project.name'
  - 'ducky-crm-gcloud'
  - '--detect.project.version.name'
  - 'source'
  - '--detect.tools'
  - 'SIGNATURE_SCAN'
  - '--detect.source.path'
  - '/workspace'

secrets:
- kmsKeyName: projects/[PROJECT_ID]/locations/[Location]/keyRings/[Keyring name]/cryptoKeys/[Key name]
  secretEnv:
    BD_TOKEN: <base64-encoded encrypted Blackduck Token from previous step>


Google's infrastructure is unable to read an environment variable in the value for the kmsKeyName field.  Therefore, you must provide the hard-coded project ID value in the kmsKeyName field. 
In the build specification YAML file example, the $PROJECT_ID is not used in the kmsKeyName field because you must provide a hard-coded value.

Note: If your instance of Black Duck uses a self-signed certificate, ensure that you include the --blackduck.trust.cert argument in the args section of the YAML/JSON file that invokes the Cloud Build scanner. This allows the scanner to connect to a Black Duck instance whose certificate is not signed by a trusted third party. Refer to the build spec file example.

Caution: Build errors might occur when the --detect.tools argument and value are not included in the list of arguments.

Invoking a build request by using build triggers

As an alternative to invoking Google Cloud Build with a build-management system such as Jenkins, you can invoke the build process with Google Container Registry's build triggers.  You can easily create a build trigger that instructs Google Cloud Build to automatically build your image whenever changes are pushed to the build source from any repository, including a cloud storage bucket. Click here for documentation about Google's Build Trigger functionality. 

Note: To use Build Triggers, your repository must contain build configuration information in a cloudbuild.yaml file.

Submitting a build request using gcloud and verifying the results

Initiate a Google Cloud Build build by using the following gcloud command.

gcloud builds submit --config cloudbuild.yaml .

In this example, cloudbuild.yaml is the build configuration file.  You must run this command from the source code home directory where the build specification file lives. After a successful run, a message displays, which is similar to the following example:

You should now be able to see the scan results in Black Duck.

Examples

All examples use the ducky-crm repo. Fork and build these example files making minor substitutions for practice.

Scanning application source code and artifacts after a build

To scan the application source code that you're compiling in Google Cloud Build, insert the Synopsys Cloud Build Scanner step after your build step. 

To break the build in case of policy violations, use the --detect.policy.check.fail.on.severities argument similar to the following example. 

Sample build specification YAML file to build a Maven project and scan code and artifacts
- name: 'gcr.io/cloud-builders/mvn'
  args: [ 'clean', 'compile', '-DskipTests' ]
- name: 'gcr.io/cloud-marketplace/blackduck-public/synopsys-cloudbuild-scanner'
  secretEnv: [ 'BD_TOKEN' ]
  args:
  - '--blackduck.url'
  - '<<Black Duck URL>>'
  - '--blackduck.api.token'
  - '$$BD_TOKEN'
  - '--blackduck.trust.cert'
  - 'true'
  - '--detect.project.name'
  - 'ducky-crm-cloudbuild'
  - '--detect.project.version.name'
  - 'source'
  - '--detect.tools' 
  - 'SIGNATURE_SCAN,BINARY_SCAN' # List of Scanners to Run
  - '--detect.source.path' 
  - '/workspace' # Target for Signature Scan, Cloud Build saves all to /workspace by default.
  - '--detect.binary.scan.file.path'
  - './target/ducky-crm-0.3.0.war' # Target for Binary Scan. Must be pointed at exact binary location
  - '--detect.policy.check.fail.on.severities'
  - 'BLOCKER'

secrets:
- kmsKeyName: projects/[PROJECT-ID]/locations/global/keyRings/[KEYRING-NAME]/cryptoKeys/[KEY-NAME]
  secretEnv:
    BD_TOKEN: <base64-encoded encrypted Black Duck Token>
artifacts:
  objects:
    location: 'gs://<location to save artifacts>/'
    paths: ['./target/ducky-crm-0.3.0.war']

Scanning an image as Quality Gate before pushing to GCR

To scan a freshly built image that's not yet present in GCR, save to a tar archive file after you build the image and scan. This can be configured to stop the image being pushed to GCR if Synopsys Detect fails.

In this example, a substitution is used to pass the name of the Docker Image to multiple steps.

Sample build specification YAML to scan an image before pushing to GCR
- name: 'gcr.io/cloud-builders/mvn'
  args: [ 'clean', 'compile', '-DskipTests' ]
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/${_IMAGE_NAME}', '.']
- name: 'gcr.io/cloud-builders/docker'
  args: ['save', '-o', './${_IMAGE_NAME}.tar', 'gcr.io/$PROJECT_ID/${_IMAGE_NAME}']
- name: 'gcr.io/cloud-marketplace/blackduck-public/synopsys-cloudbuild-scanner'
  secretEnv: [ 'BD_TOKEN' ]
  args:
  - '--blackduck.url'
  - '<<Black Duck URL>>'
  - '--blackduck.api.token'
  - '$$BD_TOKEN'
  - '--blackduck.trust.cert'
  - 'true'
  - '--detect.project.name'
  - '${_IMAGE_NAME}'
  - '--detect.project.version.name'
  - 'container-tar'
  - '--detect.tools' # List of Scanners to Run
  - 'SIGNATURE_SCAN,BINARY_SCAN'
  - '--detect.source.path' # Target for Signature Scan
  - './${_IMAGE_NAME}.tar'
  - '--detect.binary.scan.file.path'
  - './${_IMAGE_NAME}.tar' # Target for Binary Scan
  - '--detect.policy.check.fail.on.severities'
  - 'BLOCKER'
substitutions: 
  _IMAGE_NAME: ducky-crm-cb

secrets:
- kmsKeyName: projects/[PROJECT-ID]/locations/global/keyRings/[KEYRING-NAME]/cryptoKeys/[KEY-NAME]
  secretEnv:
    BD_TOKEN: <base64-encoded encrypted Black Duck Token>
images:
  - gcr.io/$PROJECT_ID/${_IMAGE_NAME}

Scanning an Image from GCR

To scan an existing image from GCR, first pull and save the image to a tar archive file; then, you can invoke Synopsys Detect against the tar archive file. 

Sample build specification YAML file to scan an image in GCR
- name: 'gcr.io/cloud-builders/docker'
  args: ['pull', '${_IMAGE_NAME}']
- name: 'gcr.io/cloud-builders/docker'
  args: ['save', '-o', './${_IMAGE_NAME}.tar', 'gcr.io/$PROJECT_ID/${_IMAGE_NAME}']
- name: 'gcr.io/cloud-marketplace/blackduck-public/synopsys-cloudbuild-scanner'
  secretEnv: [ 'BD_TOKEN' ]
  args:
  - '--blackduck.url'
  - '<<Black Duck URL>>'
  - '--blackduck.api.token'
  - '$$BD_TOKEN'
  - '--blackduck.trust.cert'
  - 'true'
  - '--detect.project.name'
  - '${_IMAGE_NAME}'
  - '--detect.project.version.name'
  - 'container-tar'
  - '--detect.tools' # List of Scanners to Run
  - 'SIGNATURE_SCAN,BINARY_SCAN'
  - '--detect.source.path' # Target for Signature Scan
  - './${_IMAGE_NAME}.tar'
  - '--detect.binary.scan.file.path'
  - './${_IMAGE_NAME}.tar' # Target for Binary Scan
  - '--detect.policy.check.fail.on.severities'
  - 'BLOCKER'
substitutions: 
  _IMAGE_NAME: ducky-crm-cb

secrets:
- kmsKeyName: projects/[PROJECT-ID]/locations/global/keyRings/[KEYRING-NAME]/cryptoKeys/[KEY-NAME]
  secretEnv:
    BD_TOKEN: <base64-encoded encrypted Black Duck Token>

Creating an attestation for Binary Authorization

If you're using Google Cloud Binary Authorization and you want to create an attestation from the result of the Black Duck scan, click here for instructions.