Skip to content

mozilla-releng/mozilla-linux-pkg-manager

Folders and files

NameName
Last commit message
Last commit date

Latest commit

d3ce9f1 · Jan 30, 2024

History

66 Commits
Nov 15, 2023
Dec 7, 2023
Nov 21, 2023
Nov 16, 2023
Nov 15, 2023
Nov 15, 2023
Nov 21, 2023
Nov 21, 2023
Nov 15, 2023
Nov 15, 2023
Dec 5, 2023
Nov 15, 2023
Nov 15, 2023
Dec 7, 2023
Jan 30, 2024
Dec 7, 2023
Nov 15, 2023

Repository files navigation

Task Status pre-commit.ci status Code Coverage

mozilla-linux-pkg-manager

mozilla-releng/mozilla-linux-pkg-manager is a Python tool for managing Mozilla product packages hosted in Linux software repositories hosted on Google Cloud Platform. It can be used to clean-up obsolete Firefox Nightly versions.

Requirements

  • Python 3.11 or higher
  • Poetry (for dependency management)

Development

Installing mozilla-linux-pkg-manager

  1. Install Poetry: If not already installed, install Poetry by following the instructions from the official Poetry website.
  2. Clone the Repository: Clone the mozilla-linux-pkg-manager repository using the command git clone https://github.com/mozilla-releng/mozilla-linux-pkg-manager.git.
  3. Install Dependencies: Navigate to the repository's root directory and run poetry install to install the required dependencies.

Setup Authentication

The easiest way to authenticate is using the Google Cloud SDK:

gcloud auth application-default login

Note that this command generates credentials for the Google Cloud Platform client libraries.

Setup the Development Environment

To set up the environment for running mozilla-linux-pkg-manager set the following variables:

# defaults to /path/to/home/.config/gcloud/application_default_credentials.json
export GOOGLE_APPLICATION_CREDENTIALS=[/path/to/google/application/credentials/file.json]
export GOOGLE_CLOUD_PROJECT=[PROJECT_NAME]

Running mozilla-linux-pkg-manager

To run mozilla-linux-pkg-manager, use Poetry with the following command:

poetry run mozilla-linux-pkg-manager clean-up [-h] --product PRODUCT --channel CHANNEL --format FORMAT --repository REPOSITORY --region REGION [--retention-days RETENTION_DAYS] [--dry-run]

Parameters

  • --product: Specifies the Mozilla product to manage (e.g. firefox, devedition, vpn). Currently, only firefox is supported.
  • --channel: Specifies the package channel (e.g. nightly, release, beta). Currently, only nightly is supported.
  • --format: The package format (i.e. deb). Currently, only deb is supported.
  • --retention-days: Sets the retention period in days for packages in the nightly channel. This parameter is only supported on the nightly channel.
  • --dry-run: Tells the script to do a no-op run and print out a summary of the operations that will be executed.
  • --repository: The repository to perform maintenance operations on.
  • --region: The cloud region the repository is hosted in.

Example

To clean up the nightly .deb packages that are older than 7 days:

poetry run mozilla-linux-pkg-manager clean-up --product firefox --channel nightly --format deb --retention-days 7 --repository mozilla --region us

Docker

The mozilla-linux-pkg-manager tool can also be run as a Docker container using the mozillareleases/mozilla-linux-pkg-manager image.

export GOOGLE_CLOUD_PROJECT=[GOOGLE_CLOUD_PROJECT]
export GOOGLE_APPLICATION_CREDENTIALS=[/path/to/google/application/credentials/file.json]
docker run --rm \
-e GOOGLE_CLOUD_PROJECT=$GOOGLE_CLOUD_PROJECT \
-e GOOGLE_APPLICATION_CREDENTIALS=/tmp/keys/google/key.json \
-v $GOOGLE_APPLICATION_CREDENTIALS:/tmp/keys/google/key.json:ro \
mozillareleases/mozilla-linux-pkg-manager:0.7.0 \
clean-up \
--product firefox \
--channel nightly \
--format deb \
--retention-days 3 \
--repository [REPOSITORY] \
--region [REGION] \
--dry-run

Building the Docker Image

First, export the desired image name as an environment variable:

export IMAGE_NAME=mozilla-linux-pkg-manager

Then, build the Docker image:

docker buildx build --platform linux/amd64 -t $IMAGE_NAME .

This command builds a Docker image with the tag specified in $IMAGE_NAME, based on the instructions in the Dockerfile.

Running the Docker Container

To run the mozilla-linux-pkg-manager in a Docker container, you need to set the Google Application Credentials and mount them as a volume in the container. Replace [FILE_NAME].json with the name of your Google Application Credentials file and ensure the path to the credentials file is correctly set in the GOOGLE_APPLICATION_CREDENTIALS environment variable.

# defaults to /path/to/home/.config/gcloud/application_default_credentials.json
export GOOGLE_APPLICATION_CREDENTIALS=[/path/to/google/application/credentials/file.json]
export GOOGLE_CLOUD_PROJECT=[PROJECT_NAME]

docker run \
-e GOOGLE_CLOUD_PROJECT=$GOOGLE_CLOUD_PROJECT \
-e GOOGLE_APPLICATION_CREDENTIALS=/tmp/keys/google/key.json \
-v $GOOGLE_APPLICATION_CREDENTIALS:/tmp/keys/google/key.json:ro \
$IMAGE_NAME \
clean-up \
--product firefox \
--channel nightly \
--format deb \
--retention-days 3 \
--repository mozilla \
--region us

In this command:

  • The -e flag sets the GOOGLE_APPLICATION_CREDENTIALS and GOOGLE_CLOUD_PROJECT environment variables inside the container.
  • The -v flag mounts the credentials file from your host system to the container.
  • The last line specifies the command and its arguments to be executed inside the container.

Building and Installing a Python Wheel

The mozilla-linux-pkg-manager package can be packaged into a wheel file for distribution and installation.

Building the Wheel

  1. Navigate to the Project Directory: Open your terminal and navigate to the directory where your project is located.
  2. Build the Package: Execute poetry build to create the wheel file. This will generate a dist folder in your project directory containing the .whl file, whose name may vary based on the version and build.

Installing the Wheel File

  1. Navigate to the dist Directory: Move to the dist directory where the .whl file is located.
  2. Install the Wheel File: Use pip install [wheel-file-name] to install the package. Replace [wheel-file-name] with the actual name of the wheel file generated during the build process.

Using the Installed Package

After installation, the package can be used from anywhere on your system, provided you are running the Python interpreter where it was installed.

Example

To clean up nightly .deb packages that are older than 3 days:

mozilla-linux-pkg-manager clean-up --product firefox --channel nightly --format deb --retention-days 3 --repository mozilla --region us