Skip to content

Commit

Permalink
Add Docker support and update README (#13)
Browse files Browse the repository at this point in the history
* add compose.yaml

* add Dockerfile

* update README

* Dockerfile: uncomment CMD instruction

* add .dockerignore

* Dockerfile: use scratch image for second stage

* Dockerfile: use newer OpenSSL version (3.0.9)

* Dockerfile: use Alpine-based Rust image to skip building OpenSSL

* Dockerfile: expose to localhost only by default

* Dockerfile: allow building for different architectures (AMD64 and ARM64)

Dockerfile now detects the architecture being used during the build process and templates in the correct Rust target architecture

* compose.yaml: include command line

* compose.yaml: listen on 127.0.0.1 only

* compose.yaml: remove healthcheck due to using scratch image

* README.md: restore old table format

* rename compose.yaml to docker-compose.yml

* docker-compose.yml: include version line

* update .dockerignore

* README.md: update Docker instructions to expose only on localhost

* add workflow to build and push container images to quay.io

* docker-compose.yml: harden configuration

* docker workflow: add paths-ignore section

* Dockerfile: let Rustup handle architecture detection

More flexible as the build process will now automatically adapt to whatever architecture the container is being built on, without needing to explicitly list out each supported architecture

* Docker: further security hardening

- Run as a non-privileged user within the scratch container
- Add security_opt: - no-new-privileges:true to docker-compose.yml

* Dockerfile: Switch to Debian-based images and simplify build command

- rust:1.80-alpine replaced with rust:1.80 for the builder stage
- alpine:3.20 replaced with debian:12.6-slim for the user-stage
- Build command simplified to use default target architecture

* Cargo.toml: correct note on optimisations

* docker-compose.yml: use quay.io image by default

* rename docker-compose.yml to docker-compose.yaml

* compose: build image from local repo by default

* Revert "Dockerfile: Switch to Debian-based images and simplify build command"

This reverts commit ff9a378.

Reasons for reverting:
1. Compiling via musl is necessary to statically link dependencies and create a truly standalone Rust binary. [1]
2. Alpine-based Rust images are required for the build stage because such systems support dynamic linking, which is also needed for statically-linked binaries. [2]
3. Determining the target architecture and templating the correct value for the --target flag is necessary for the statically-linked binary to be built correctly. [2]

[1]: https://doc.rust-lang.org/1.13.0/book/advanced-linking.html#linux
[2]: rust-lang/rust#40174 (comment)
  • Loading branch information
perennialtech authored Aug 14, 2024
1 parent f7cbcbd commit 4d3cff6
Show file tree
Hide file tree
Showing 6 changed files with 270 additions and 22 deletions.
13 changes: 13 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Rust build artifacts
/target

# Git files
.git
.gitignore

# Docker Compose file
docker-compose.yaml

# Documentation
README.md
LICENSE
72 changes: 72 additions & 0 deletions .github/workflows/docker-build-push.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Build and Push Docker Image

# Define when this workflow will run
on:
push:
branches:
- master # Trigger on pushes to master branch
tags:
- '[0-9]+.[0-9]+.[0-9]+' # Trigger on semantic version tags
paths-ignore:
- 'Cargo.lock'
- 'LICENSE'
- 'README.md'
- 'docker-compose.yml'
workflow_dispatch: # Allow manual triggering of the workflow

# Define environment variables used throughout the workflow
env:
REGISTRY: quay.io
IMAGE_NAME: invidious/inv-sig-helper

jobs:
build-and-push:
runs-on: ubuntu-latest

steps:
# Step 1: Check out the repository code
- name: Checkout code
uses: actions/checkout@v3

# Step 2: Set up QEMU for multi-architecture builds
- name: Set up QEMU
uses: docker/setup-qemu-action@v2

# Step 3: Set up Docker Buildx for enhanced build capabilities
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

# Step 4: Authenticate with Quay.io registry
- name: Log in to Quay.io
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}

# Step 5: Extract metadata for Docker image tagging and labeling
- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# Define tagging strategy
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=sha,prefix={{branch}}-
# Define labels
labels: |
quay.expires-after=12w
# Step 6: Build and push the Docker image
- name: Build and push Docker image
uses: docker/build-push-action@v4
with:
context: .
push: true
platforms: linux/amd64,linux/arm64 # Build for multiple architectures
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
8 changes: 8 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,11 @@ lazy-regex = "3.1.0"
tub = "0.3.7"
tokio-util = { version = "0.7.10", features=["futures-io", "futures-util", "codec"]}
futures = "0.3.30"

# Compilation optimizations for release builds
# Increases compile time but typically produces a faster and smaller binary. Suitable for final releases but not for debug builds.
[profile.release]
lto = true
opt-level = 3
codegen-units = 1
panic = 'abort'
55 changes: 55 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Use the official Alpine-based Rust image as a parent image
FROM rust:1.80-alpine AS builder

# Set the working directory in the container
WORKDIR /usr/src/app

# Install build dependencies
RUN apk add --no-cache \
musl-dev \
openssl-dev \
openssl-libs-static \
pkgconfig \
patch

# Set environment variables for static linking
ENV OPENSSL_STATIC=yes
ENV OPENSSL_DIR=/usr

# Copy the current directory contents into the container
COPY . .

# Determine the target architecture and build the application
RUN RUST_TARGET=$(rustc -vV | sed -n 's/host: //p') && \
rustup target add $RUST_TARGET && \
RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target $RUST_TARGET

# Stage for creating the non-privileged user
FROM alpine:3.20 AS user-stage

RUN adduser -u 10001 -S appuser

# Stage for a smaller final image
FROM scratch

# Copy necessary files from the builder stage, using the correct architecture path
COPY --from=builder /usr/src/app/target/*/release/inv_sig_helper_rust /app/inv_sig_helper_rust
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# Copy passwd file for the non-privileged user from the user-stage
COPY --from=user-stage /etc/passwd /etc/passwd

# Set the working directory
WORKDIR /app

# Expose port 12999
EXPOSE 12999

# Switch to non-privileged user
USER appuser

# Set the entrypoint to the binary name
ENTRYPOINT ["/app/inv_sig_helper_rust"]

# Set default arguments in CMD
CMD ["--tcp", "127.0.0.1:12999"]
125 changes: 103 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,97 +1,178 @@
# Protocol Format
# inv_sig_helper

`inv_sig_helper` is a Rust service that decrypts YouTube signatures and manages player information. It offers a TCP/Unix socket interface for signature decryption and related operations.

## Features

- Decrypt YouTube `n` and `s` signatures
- Fetch and update YouTube player data
- Provide signature timestamps and player status
- Efficient signature decryption with multi-threaded JavaScript execution

## Building and Running

### Prerequisites

- Rust 1.77 or later
- Cargo
- Patch
- openssl-devel

### Building

1. Clone the repository and navigate to the project directory:

```
git clone https://github.com/iv-org/inv_sig_helper.git
cd inv_sig_helper
```

2. Build the project:

```
cargo build --release
```

### Running

The service can run in Unix socket mode (default) or TCP mode:

1. Unix socket mode:

```
./target/release/inv_sig_helper_rust
```

This creates a Unix socket at `/tmp/inv_sig_helper.sock`.

2. TCP mode:

```
./target/release/inv_sig_helper_rust --tcp [IP:PORT]
```

If no IP:PORT is given, it defaults to `127.0.0.1:12999`.

## Docker

A Dockerfile is included for containerized deployment.

1. Build the image:

```
docker build -t inv_sig_helper .
```

2. Run the container:

```
docker run -p 127.0.0.1:12999:12999 inv_sig_helper
```

Or use Docker Compose:

```
docker compose up
```

## Protocol Format

All data-types bigger than 1 byte are stored in network endian (big-endian) unless stated otherwise.

## Request Base
### Request Base
| Name | Size (bytes) | Description |
|-----------|--------------|--------------------------------------|
|opcode | 1 | The operation code to perform, A list of operations currently supported (and their data) can be found in the **Operations** chapter |
|request_id | 4 | The ID for the current request, Used to distinguish responses in the current connection |

The data afterwards depends on the supplied opcode, Please consult the **Operations** chapter for more information.

## Response Base
### Response Base
| Name | Size (bytes) | Description |
|------------|--------------|---------------------------------------|
|request_id | 4 | The ID for the request that this response is meant for |
|size | 4 | Size of the response (excluding size of request id)|

The data afterwards depends on the supplied opcode, Please consult the **Operations** chapter for more information.

## Operations
### `FORCE_UPDATE` (0x00)
### Operations
#### `FORCE_UPDATE` (0x00)
Forces the server to re-fetch the YouTube player, and extract the necessary components from it (`nsig` function code, `sig` function code, signature timestamp).

#### Request
##### Request
*No additional data required*

#### Response
##### Response
| Name | Size (bytes) | Description |
|------|--------------|-------------|
|status| 2 | The status code of the request: `0xF44F` if successful, `0xFFFF` if no updating is required (YouTube's player ID is equal to the server's current player ID), `0x0000` if an error occurred |

### `DECRYPT_N_SIGNATURE` (0x01)
#### `DECRYPT_N_SIGNATURE` (0x01)
Decrypt a provided `n` signature using the server's current `nsig` function code, and return the result (or an error).

#### Request
##### Request
| Name | Size (bytes) | Description |
|------|--------------|-------------------------------------|
|size | 2 | The size of the encrypted signature |
|string| *`size`* | The encrypted signature |

#### Response
##### Response
| Name | Size (bytes) | Description |
|------|--------------|------------------------------------------------------------------|
|size | 2 | The size of the decrypted signature, `0x0000` if an error occurred |
|string| *`size`* | The decrypted signature |

### `DECRYPT_SIGNATURE` (0x02)
#### `DECRYPT_SIGNATURE` (0x02)
Decrypt a provided `s` signature using the server's current `sig` function code, and return the result (or an error).

#### Request
##### Request
| Name | Size (bytes) | Description |
|------|--------------|-------------------------------------|
|size | 2 | The size of the encrypted signature |
|string| *`size`* | The encrypted signature |

#### Response
##### Response
| Name | Size (bytes) | Description |
|------|--------------|------------------------------------------------------------------|
|size | 2 | The size of the decrypted signature, `0x0000` if an error occurred |
|string| *`size`* | The decrypted signature |

### `GET_SIGNATURE_TIMESTAMP` (0x03)
#### `GET_SIGNATURE_TIMESTAMP` (0x03)
Get the signature timestamp from the server's current player, and return it in the form of a 64-bit integer. If there's no player, it will return 0 in the `timestamp` (Please check with `PLAYER_STATUS` if the server has a player)

#### Request
##### Request
No additional data required

#### Response
##### Response
| Name | Size (bytes) | Description |
|---------|--------------|----------------------------------------------------------|
|timestamp| 8 | The signature timestamp from the server's current player |

### `PLAYER_STATUS` (0x04)
#### `PLAYER_STATUS` (0x04)
Get the server's information about the current player.

#### Request
##### Request
No additional data required

#### Response
##### Response

| Name | Size (bytes) | Description |
|----------|--------------|-------------|
|has_player| 1 | If the server has a player, this variable will be `0xFF`. or else, it will be `0x00`|
|player_id | 4 | The server's current player ID. If the server has no player, this will always be `0x00000000`|

### `PLAYER_UPDATE_TIMESTAMP` (0x05)
#### `PLAYER_UPDATE_TIMESTAMP` (0x05)
Get the time of the last player update, The time is represented as seconds since the last update

#### Request
##### Request
No additional data required

#### Response
##### Response

| Name | Size (bytes) | Description |
|----------|--------------|-------------|
|timestamp | 8 | Seconds since the last player update |

## License

This project is open source under the AGPL-3.0 license.
19 changes: 19 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: "3"
services:
inv_sig_helper:
build:
context: .
dockerfile: Dockerfile
# image: quay.io/invidious/inv-sig-helper:latest
command: ["--tcp", "127.0.0.1:12999"]
ports:
- 127.0.0.1:12999:12999
environment:
- RUST_LOG=info
restart: unless-stopped
cap_drop:
- ALL
read_only: true
user: 10001:10001
security_opt:
- no-new-privileges:true

0 comments on commit 4d3cff6

Please sign in to comment.