Version 1.0.2

Overview

The Black Duck GitHub Pull Request Scanner makes it easy to scan GitHub repositories without configuring a continuous integration (CI) tool. 
You might want to scan without using a CI tool in the following circumstances:

• Your organization does not have a CI tool.
• Your organization has a CI tool, but you don't have permission to configure it.
• Your organization has multiple CI tools, and you’d like to run a direct scan without configuring multiple CI systems.

The Black Duck GitHub Pull Request Scanner is also useful for Black Duck demonstrations and testing before you modify your production CI system. 
The Black Duck GitHub Pull Request Scanner calls the Black Duck Detect utility, and leverages the open source Concourse CI tool to initiate a build, ensuring scan results from Black Duck Detect. 

System requirements

• 1 quad-core Intel Xeon processor, 16 GB RAM, minimum 500 GB disk space.
• Linux kernel versions 3.19 and later: 
  • RedHat Enterprise Linux versions 7.4 and later.
  • Ubuntu versions 15.04 and later.
  • SUSE Linux Enterprise Server versions 12 SP2 and later.
• Docker with engine versions 1.13 and later.
• Docker Compose versions 1.16 and later.
• Java 8 JDK.

Building and installation

1. Download and extract the archive from https://github.com/blackducksoftware/hub-gprs/releases.
2. Using a Bash shell, navigate to the setup directory.
3. Run ./setup.sh  and respond to the prompts.

Securing the Black Duck GitHub scanner

Plain text is used for communication between the browser and Black Duck GitHub Pull Request Scanner (Black Duck GitHub Scanner), which is not secure.
However, GitHub tokens and private keys are sent back and forth by using this channel.
Do the following steps to secure your Black Duck GitHub Scanner installation.

1. In a Bash shell, navigate to the setup directory in the directory where you cloned the repository.

2. Copy a PKCS12 trust store file as keys/ui/keystore.p12.
   To generate a self-signed key, run the following command:

   keytool -genkey -alias hub-scm -storetype PKCS12 -keyalg RSA -keysize 2048 -keystore keys/ui/keystore.p12 -validity 36500

   
   

3. Respond to the prompts, and make a note of the password because you use it in the next step.

4. Add the following command to your shell environment, and substitute mypassword with the password from step 3.
   

   export UI_STARTUP_OPTS="-Dserver.ssl.key-store=/opt/keys/keystore.p12 -Dserver.ssl.key-store-password=mypassword -Dserver.ssl.keyStoreType=PKCS12 -Dserver.ssl.keyAlias=hub-scm"

   
   

5. Restart the application by running the following command:
   ./shutdown.sh followed by ./startup.sh

Starting the application

When setup is done, run the following command to start Black Duck GitHub Scanner.

./startup.sh 

When the system starts, you access it at http://(hostname):8666

Stopping the application

To stop the application, run the following command.

./shutdown.sh

Uninstalling the application

To uninstall the application, run the following command.

./reset.sh

When you run this command, delete the cloned repository.

Using the Black Duck GitHub scanner

This following list describes the actions that you can take by using the Black Duck GitHub scanner:

• Logging in.
• Configuring a source code management repository.
• Configuring a monitoring repository.

• Automatic scanning of pull requests.

• Discontinuing monitoring.
• Deleting an SCM.

• Injecting files into builds.

• Applying settings to multiple repositories.

The following section describes the items in the list:

Logging in

Log in to the application when the application is installed, configured, and running.
Log in with Black Duck instance accounts to which Black Duck GitHub Scanner is connected at setup. 

Configuring a source code management repository

Use the following process to configure a source code management (SCM) repository:

1. On the navigation bar, click SCMs.
   An empty list of SCMs displays.
2. Click New to open the Add repository screen, and then complete the following fields:
   • Name: friendly name for this SCM repository.
   • API Token: this token enables the Black Duck GPRS to check GitHub for pull requests.
     Get a personal use token by logging into a GitHub account with permissions to access the relevant repositories.
     Then, complete the process for Creating a personal access token for the command line.
     • If only public repositories are monitored, the repo:status scope is sufficient.
     • If private repositories are monitored, you must be granted the entire repo scope.
   • API Endpoint: If you use github.com, use the default value (https://api.github.com).
     If you use GitHub enterprise on a private host, the API endpoint is https://my-github-enterprise/api/v3.

   • Private Key: If you are only monitoring pull requests on public repositories, ignore this field.
     If at least one private repository on this SCM is monitored, this field must contain a private key with a corresponding public key registered to an account that is authorized to clone the scanned repositories.
     For more information about configuring SSH keys in GitHub, refer to Connecting to GitHub with SSH. 

     The private key can't have a passphrase.

     
     

   • GitHub host name: If you use github.com, use the default value (www.github.com). Otherwise, specify the host name or IP address of the installation. Omit the https://  part of the address.
   • Skip SSL Verification: When you use GitHub Enterprise configured with a self-signed certificate, attempts to connect to its API and to pull code from pull requests fail because of the untrusted certificate signer.
     Select this Check box to prevent that failure.
     Leave the box unchecked when using github.com.

Configuring a monitoring repository

When an SCM is configured, do the following tasks:

1. Click Repos on the navigation bar to configure a repository to be monitored for pull requests.

   To add or edit repositories, Black Duck users must be assigned the codescanner role.

   
   

2. Click New to create a new repository, and then complete the following fields.
   • Source: an SCM, as configured in the previous section.
   • Repository: A repository on that SCM. For example, to monitor the Scan-SCM repository on github.com, the repository value is blackducksoftware/hub-scm. 
     To examine all dependencies, Black Duck GitHub Scanner builds the source code. The subsequent fields determine how the project is built. 

   • Build Type: A preset of common values for the fields that follow.
     For example, for most Maven projects built on Java 8, setting this value to Maven might be enough to configure the repository without modifying other fields.
     

     Only the build types that are shown in this box are supported. Only Gradle versions 3.1 and later are supported. For some build-type selections, you must configure specific Hub Detect properties. For example, for a project named python3, you must add --detect.python.python3=true to the detect arguments field.

     
     

The following advanced options might be required in special cases:

• Build Image: Docker Black Duck name of an image on which the build is run.
• Build Image tag: Docker Black Duck tag of the image on which the build is run.
• Build Command: command required to build the repository and pull in all dependencies.
• Project Name: name of a Black Duck project to which scan results are uploaded.
  The project can already exist on Black Duck, or it can be a new project.
  • If the project exists, the Black Duck user whose credentials are provided at configuration must have access to that project.
  • If name is left blank, Black Duck Detect provides a project name derived from the build metadata.
• Version Name: The name of the version to be created in the Black Duck project, which might be specified even when the project name is not specified.
  • If the version name is left blank, it is named Pull Request X, where X is the numerical GitHub identifier of the pull request.
  • If the version name is specified, all pull request scans are written to one project version.

Automatic scanning of pull requests

All pull requests are submitted to a repository configured for monitoring as previously described. After configuration, an automatic build is triggered, and a Black Duck Detect scan is initiated by Black Duck GitHub Scanner.

Clicking Repos in the navigation bar displays a status indicator next to each repository.
The status values are as follows:

• Never Built: no pull requests have been made on this repository since it was configured for monitoring.
• In Progress: a new pull request is detected and is being built and analyzed.
• Failed: the attempt to build or scan the most recent pull request failed.
• Violation: the attempt to build and scan the most recent pull request succeeded, but one or more components violate the current policy as configured on Black Duck.
• Success: the attempt to build and scan the most recent pull request succeeded and no policy violations are found.

Discontinuing monitoring

To stop monitoring a repository for pull requests, click Repos in the navigation bar and click the delete icon ( ) next to the name of the repository. When the repository disappears from the list, its pull requests are no longer monitored.

Deleting an SCM

To delete an SCM, click SCMs on the navigation bar and then click the delete icon () next to the SCM to be deleted.
You only delete those SCMs that don't have monitored repositories.
For SCMs that have monitored repositories, you must first delete those repositories on the Repositories screen.

Injecting files into builds

Building a project may require configuration or metadata files that reside outside the source repository.
For example, the configuration for the build system to use a specific binary artifact repository might be stored in a file that is not checked into GitHub.
Or, if the build must be executed from a specific shell script, that script can be uploaded.

Inject files into the build environment by using the following process.

1. Click Files.
2. In the Upload new files dialog, select a file to upload. The file size must be 1 MB or smaller.
3. Specify a name for this file; this is a name that you can use to identify it, not the name of the file in the build environment.
4. Click Submit.
5. Next, click Repositories to navigate to the repository and the build in which you have injected this file.
6. In File Injection, click the plus () icon to add a file injection. 
7. Select the file you uploaded from the drop-down menu.
8. In the Target field, type the absolute path to which the file is copied.
   The directory into which the file is to be placed must exist in the build image.
9. When you have added the files to inject and specified their targets, click Submit.

Applying settings to multiple repositories

GitHub accounts and GitHub enterprise installations can contain multiple projects of the same kind. In other words, these projects can be built and scanned using identical settings.
Cloning enables you to apply the build and scan settings for one repository to many other repositories on the same GitHub server and account. 
To apply settings to multiple repositories, use the following procedure:

1. Configure one SCM and repository. Ensure that it successfully builds and scans.
2. From the Repositories tab, click the clone icon () to open The Clone Repository panel.
3. In the Clone Repository panel, type the names of all repositories to be scanned using the same settings.
4. Click + (plus icon) to add more repositories.

5. Click Submit.

   Repositories that belong to other accounts not configured for the source repository can only be added when they are public repositories, or if the user whose private key and API token are configured in the SCM settings has access to those repositories.

   
   

Upgrading

To upgrade to a newer version of the application, use the following process:

1. Pull or extract the upgrade version.
2. Navigate into the setup directory using a Bash shell.

3. Run the following command:

   ./update.sh

   
   

This step stops the application, and builds and installs the upgrade. 
When the upgrade is finished, the upgraded application automatically restarts. 
All configured SCMs and repositories are retained and unaffected by the completion of the upgrade.

Troubleshooting

Enabling build log capture

Because repositories monitored by Black Duck GitHub Scanner must be built before scanning, there is a possibility that one or more of the builds can fail.
To troubleshoot build failures, run the following command in your shell before you run the startup script.

export HUB_SCM_BUILD_LOG_DIR=/var/logs/mybuildlogs 

Replace /var/log/mybuildlogs with the directory that you want to store the logs.

Avoiding Black Duck timeouts

When a build completes and Black Duck Detect performs a scan, it uploads the results of that scan to Black Duck and waits for Black Duck to respond with whether or not the contents of the scanned build violate the configured policy.
By default, Black Duck is given five minutes to process the scan and build a Bill of Materials (BOM). For very large projects or under heavy load, five minutes might be insufficient.
To increase this timeout, add the following line to Black Duck Detect Arguments.

--detect.api.timeout=N

Release Notes

Version 1.0.2

• Updates Spring dependencies and internal Docker version.
• Adds support for Black Duck's global_code_scanner role.
• Adds support for Gradle 4.6.

Version 1.0.1

• If you are using Black Duck Hub versions 4.5.0 or higher, you must use version 1.0.1 or later of the GitHub Pull Request scanner. Version 1.0.0 of the GitHub Pull Request scanner is not compatible with Black Duck Hub versions 4.5.0 and higher.

Version 1.0.0

• First release of the product.