Skip to content

Commit

Permalink
Release 2022.2.0
Browse files Browse the repository at this point in the history
  • Loading branch information
@sig-rlloyd
sig-rlloyd committed Feb 28, 2022
1 parent a2c1fb4 commit 5c11df6
Show file tree
Hide file tree
Showing 66 changed files with 1,067 additions and 1,237 deletions.
32 changes: 4 additions & 28 deletions Important_Upgrade_Announcement.md
View file
Original file line number Diff line number Diff line change
@@ -1,38 +1,14 @@
# Overview
Customers upgrading from a version prior to 4.2, will need to perform a data migration as part of their upgrade process. The detailed instructions are in the README.md doc file in the folder for the orchestration method you used in your Black Duck installation. If you are performing a new install, these steps are not required.
Customers upgrading from a version prior to 4.2 should contact Synopsys Technical Support before proceeding.

# Disk Space
The data migration will temporarily require an additional free disk space at approximately 2.5 times your original database volume size to hold the database dump and the new 4.2 database volume. As a rule-of-thumb, if the volume upon which your database resides is at least 60% free, there should be enough disk space.

# Upgrade Steps
At a high level, the steps are as follows. Actual commands to run the steps are located in the README.md doc for your particular orchestration method.

1. Verify Disk space. (see above)

2. Bring down the Black Duck containers - Use the appropriate commands per your orchestration type to bring down your Black Duck instance. This is done to ensure no one is writing to the database during the migration procedure.

3. Start the currently installed Black Duck (prior to version 4.2) in data migration mode - For each orchestration method, a configuration file is provided to only start the containers needed for the migration. Use that file to bring up those containers. This will start the only the database containers needed for the dump.

4. Create a dump of the database – use the provided script and commands listed in the README.md file to create the database dump.

5. Bring Down the data migration mode containers – once the dump is complete, use the appropriate commands as documented for your orchestration method to bring down the database migration containers.

6. Using the new orchestration files, start the version of Black Duck to which you migrating in data migration mode. This will pull the new images and upgrade the database software.

7. Using the provided scripts, restore the database dump from step 4

8. Bring down the data migration mode containers

9. Using the new orchestration files, start the new version of Black Duck to which your are upgrading (i.e 4.2 or later)

10. Verify that the data is migrated and Black Duck is on the upgraded version

11. Once everything looks good, you can remove the old data volume

If you perform the upgrade without migrating the data, the upgrade will successfully complete, however, the system will be left with an empty database as the data was not migrated. If this does occur, the data is safe in the old volume, however, administrators need will perform the migration to before the old data can be accessed in the Black Duck application.
# Database Migration
When upgrading to Black Duck version 2022.2.0 or later, database migration is done automatically. Details are provided in the README.md doc for your particular orchestration method.

# External Databases
If your Black Duck instance was configured to use an external database (like Amazon RDS), administrators will essentially need to do the same thing. The recommended approach is to migrate your data to a 9.6 instance of PostgreSQL and configure your system to point to that instance. If an administrator attempts to perform an upgrade to Black Duck 4.2 on a system that is connected to a non-9.6 PostgreSQL database, the application will fail to start, however the data remains safe.
If your Black Duck instance was configured to use an external database (like Amazon RDS), administrators will need to follow the upgrade process defined by the PostgreSQL provider. The recommended approach is to migrate your data to a 13.x instance of PostgreSQL and configure your system to point to that instance.

# Contact Support
If you have any questions or concerns, please contact the Customer Support Organization to help you through this process.
Expand Down
121 changes: 20 additions & 101 deletions README.containers.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Containers
----

There are fifteen containers that make up the application. Here are quick descriptions for them.
There are a number of containers that make up the application. Here are quick descriptions for them.


# Web App Container (blackduck-webapp)
Expand Down Expand Up @@ -36,18 +36,11 @@ There are times when running in other types of orchestrations that it is useful
* logstash - $HUB_LOGSTASH_HOST
* cfssl - $HUB_CFSSL_HOST

## Resources/Constraints

* Default Max Java Heap Size: 2GB
* Container Memory: 2.5GB
* Container CPU: 1cpu

## Users/Groups

This container runs as UID 8080. If the container is started as UID 0 (root) then the user will be switched to UID 8080:root before executing it's main process.
This container is also able to be started as a random UID as long as it is also started within the root group (GID/fsGroup 0).


# Authentication Container (blackduck-authentication)
----

Expand Down Expand Up @@ -79,12 +72,6 @@ The container will need to expose 8443 to other containers that will links to it
* registration - $HUB_REGISTRATION_HOST
* webapp - $HUB_WEBAPP_HOST

## Resources/Constraints

* Default Max Java Heap Size: 512MB
* Container Memory: 1GB
* Container CPU: 1cpu

## Users/Groups

This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing it's main process.
Expand Down Expand Up @@ -119,12 +106,6 @@ The container will need to expose 8443 to other containers that will links to it
* logstash - $HUB_LOGSTASH_HOST
* registration - $HUB_REGISTRATION_HOST

## Resources/Constraints

* Default Max Java Heap Size: N/A
* Container Memory: 4.5GB
* Container CPU: 1cpu

## Users/Groups

This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing it's main process.
Expand Down Expand Up @@ -159,12 +140,6 @@ The container will need to expose 8443 to other containers that will links to it
* logstash - $HUB_LOGSTASH_HOST
* registration - $HUB_REGISTRATION_HOST

## Resources/Constraints

* Default Max Java Heap Size: 4 GB
* Container Memory: 4.5GB
* Container CPU: 1cpu

## Users/Groups

This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing it's main process.
Expand Down Expand Up @@ -202,19 +177,11 @@ There are times when running in other types of orchestrations that it is useful
* logstash - $HUB_LOGSTASH_HOST
* cfssl - $HUB_CFSSL_HOST

## Resources/Constraints

* Default Max Java Heap Size: 2GB
* Container Memory: 2.5GB
* Container CPU: 1cpu

## Users/Groups

This container runs as UID 8080. If the container is started as UID 0 (root) then the user will be switched to UID 8080:root before executing it's main process.
This container is also able to be started as a random UID as long as it is also started within the root group (GID/fsGroup 0).



# Job Runner Container (blackduck-jobrunner)
----

Expand Down Expand Up @@ -248,12 +215,6 @@ To support any such use case, these environment variables can be set to override
* logstash - $HUB_LOGSTASH_HOST
* cfssl - $HUB_CFSSL_HOST

## Resources/Constraints

* Default Max Java Heap Size: 4GB
* Container Memory: 4.5GB
* Container CPU: 1cpu

## Users/Groups

This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing it's main process.
Expand Down Expand Up @@ -287,12 +248,6 @@ There are times when running in other types of orchestrations that it is useful
* logstash - $HUB_LOGSTASH_HOST
* cfssl - $HUB_CFSSL_HOST

## Resources/Constraints

* Default Max Java Heap Size: 512MB
* Container Memory: 640MB
* Container CPU: unspecified

## Users/Groups

This container runs as UID 8080. If the container is started as UID 0 (root) then the user will be switched to UID 8080:root before executing it's main process.
Expand Down Expand Up @@ -330,16 +285,29 @@ In this case, these environment variables can be used to replace service names.
* logstash - $HUB_LOGSTASH_HOST
* cfssl - $HUB_CFSSL_HOST

### Resources/Constraints
## Users/Groups

This container runs as UID 1001 by default. If the container is started as UID 0 (root) then the user will be switched to UID 1001:root before executing its main process.


## DB Upgrade Container (blackduck-postgres-upgrader)
----

### Container Description

The DB Upgrade container is a transient container that performs database version upgrades (e.g., from PostgreSQL 9.6.x to PostgreSQL 11.x) when necessary, then exits.

### Scalability

There should only be a single instance of this container. It currently cannot be scaled.

### Links/Ports

* Default Max Java Heap Size: N/A
* Container Memory: 3GB
* Container CPU: 1cpu
This container does not connect to any other containers/services.

## Users/Groups

This container runs as UID 70. If the container is started as UID 0 (root) then the user will be switched to UID 70:root before executing it's main process.
This container is not able to start with any other user id.
This container runs as UID 0 by default. If upgrading Black Duck from a version prior to 2022.2.0, the container must be run with a UID having permission to restructure the PostgreSQL data volume and change its ownership from UID 70 to UID 1001.


# Documentation Container (blackduck-documentation)
Expand Down Expand Up @@ -368,12 +336,6 @@ There are times when running in other types of orchestrations that it is useful
* logstash - $HUB_LOGSTASH_HOST
* cfssl - $HUB_CFSSL_HOST

## Resources/Constraints

* Default Max Java Heap Size: 512MB
* Container Memory: 512MB
* Container CPU: unspecified

## Users/Groups

This container runs as UID 8080. If the container is started as UID 0 (root) then the user will be switched to UID 8080:root before executing it's main process.
Expand Down Expand Up @@ -418,12 +380,6 @@ There are times when running in other types of orchestrations that any individua
* documentation - $HUB_DOC_HOST
* upload cache - $HUB_UPLOAD_CACHE_HOST

## Resources/Constraints

* Default Max Java Heap Size: N/A
* Container Memory: 512MB
* Container CPU: unspecified

## Users/Groups

This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing it's main process.
Expand All @@ -441,12 +397,6 @@ To support any such use case, these environment variables can be set to override

* logstash - $HUB_LOGSTASH_HOST

## Resources/Constraints

* Default Max Java Heap Size: 256MB
* Container Memory: 384MB
* Container CPU: unspecified

## Users/Groups

This container runs as UID 1000. If the container is started as UID 0 (root) then the user will be switched to UID 1000:root before executing it's main process.
Expand All @@ -468,12 +418,6 @@ There should only be a single instance of this container. It currently cannot be

The container will need to expose port 5044 to other containers/services that will link to it.

## Resources/Constraints

* Default Max Java Heap Size: 1GB
* Container Memory: 1GB
* Container CPU: unspecified

## Users/Groups

This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing it's main process.
Expand All @@ -496,18 +440,11 @@ There should only be a single instance of this container. It currently cannot be

The container will need to expose port 8888 to other containers/services that will link to it.

## Resources/Constraints

* Default Max Java Heap Size: N/A
* Container Memory: 512MB
* Container CPU: unspecified

## Users/Groups

This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing it's main process.
This container is also able to be started as a random UID as long as it is also started within the root group (GID/fsGroup 0).


# RabbitMQ Container (rabbitmq)
----

Expand Down Expand Up @@ -540,12 +477,6 @@ To support any such use case, these environment variables can be set to override

* cfssl - $HUB_CFSSL_HOST

## Resources/Constraints

* Default Max Java Heap Size: N/A
* Container Memory: 1GB
* Container CPU: unspecified

## Users/Groups

This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing it's main process.
Expand Down Expand Up @@ -586,12 +517,6 @@ To support any such use case, these environment variables can be set to override
* logstash - $HUB_LOGSTASH_HOST
* rabbitmq - $RABBIT_MQ_HOST

## Resources/Constraints

* Default Max Java Heap Size: N/A
* Container Memory: 512MB
* Container CPU: unspecified

## Other configurable environment variables
* Default disk size for source files: 4GB ($MAX_TOTAL_SOURCE_SIZE_MB)
* Default Data Retention Days: 180 ($DATA_RETENTION_IN_DAYS)
Expand Down Expand Up @@ -637,12 +562,6 @@ To support any such use case, these environment variables can be set to override
* rabbitmq - $RABBIT_MQ_HOST
* webserver - $HUB_WEBSERVER_HOST

## Resources/Constraints

* Default Max Java Heap Size: N/A
* Container Memory: 2GB
* Container CPU: 1cpu

## Users/Groups

This container runs as UID 0.
44 changes: 9 additions & 35 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,17 @@

This repository contains orchestration files and documentation for deploying Black Duck Docker containers.

## Location of Black Duck 2021.10.5 archive:
## Location of Black Duck 2022.2.0 archive:

https://github.com/blackducksoftware/hub/archive/v2021.10.5.tar.gz
https://github.com/blackducksoftware/hub/archive/v2022.2.0.tar.gz

NOTE:

Customers upgrading from a version prior to 2018.12.0 will experience a longer than usual upgrade time due to a data migration needed to support new features in subsequent releases. Upgrade times will depend on the size of the Black Duck database. If you would like to monitor the process of the upgrade, please contact Synopsys Customer Support for instructions.

Customers upgrading from a version prior to 4.2, will need to perform a data migration as part of their upgrade process. A high level description of the upgrade is located in the Important_Upgrade_Announcement.md file in the root directory of this package. Detailed instructions to perform the migration located in the individual README.md doc file in the directory for the each orchestration method folder.
Customers upgrading from a version prior to 4.2 should contact Synopsys Technical Support for assistance.

Customers upgrading from a version prior to 2022.2.0 will have their PostgreSQL data volume automatically migrated from PostgreSQL 9.6.x to PostgreSQL 11.x.

## Previous Versions

Expand All @@ -28,6 +30,7 @@ https://github.com/blackducksoftware/hub/releases
* https://hub.docker.com/r/blackducksoftware/blackduck-logstash/
* https://hub.docker.com/r/blackducksoftware/blackduck-nginx/
* https://hub.docker.com/r/blackducksoftware/blackduck-postgres/
* https://hub.docker.com/r/blackducksoftware/blackduck-postgres-upgrader/
* https://hub.docker.com/r/blackducksoftware/blackduck-registration/
* https://hub.docker.com/r/blackducksoftware/blackduck-scan/
* https://hub.docker.com/r/blackducksoftware/blackduck-webapp/
Expand All @@ -46,37 +49,8 @@ Swarm (mode), Kubernetes, and OpenShift are supported as of Black Duck (Hub) 4.2

## Requirements

### Orchestration Version Requirements

Black Duck supports the following orchestration environments:

* Docker 18.03.x
* Docker 18.06.x
* Docker 18.09.x
* Docker 19.03.x (CE or EE)
* Kubernetes 1.9.x-1.17
* Red Hat OpenShift Container Platform 3.8-3.11
* Red Hat OpenShift Container Platform 4.1
* Red Hat OpenShift Container Platform 4.3
* Red Hat OpenShift Container Platform 4.4

### Minimum Hardware Requirements

This is the minimum hardware that is needed to run a single instance of each container. The sections below document the individual requirements for each container if they will be running on different machines or if more than one instance of a container will be run (right now only Job Runners support this).

* 6 CPUs
* 26 GB RAM for the minimum Redis configuration; 29 GB for an optimal configuration providing higher availability for Redis-driven caching.
* 250 GB DISK SPACE

Please note there that these are the minimum hardware requirements. These will likely need to be increased with larger or multiple concurrent scans.

Also, for Swarm, Kubernetes and OpenShift, note that these requirements are only for Black Duck itself and do not include other resources that are required to run the cluster overall.

### Additional Resources when Binary Scanning is Enabled
* Refer to the Black Duck 'Installing Black Duck Using Docker Swarm' document for complete, up-to-date requirements information for orchestrating Black Duck using Docker Swarm.
* Refer to the Black Duck 'Installing Black Duck Using Kubernetes' document for complete, up-to-date requirements information for orchestrating Black Duck using Kubernetes.
* Refer to the Black Duck 'Installing Black Duck Using OpenShift' document for complete, up-to-date requirements information for orchestrating Black Duck using OpenShift.

There are variations of the orchestration files that will add additional containers for use in Binary Scanning. If these additional containers
are added, then the following additional resources would be required:

* 1 ADDITIONAL CPU
* 4 GB ADDITIONAL RAM
* 100 GB ADDITIONAL DISK SPACE
Loading

0 comments on commit 5c11df6

Please sign in to comment.