Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Docker support and update README #13

Merged
merged 29 commits into from
Aug 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c7005e6
add compose.yaml
perennialtech Aug 13, 2024
da9a34e
add Dockerfile
perennialtech Aug 13, 2024
4afcadd
update README
perennialtech Aug 13, 2024
ce63a2b
Dockerfile: uncomment CMD instruction
perennialtech Aug 13, 2024
0c17e90
add .dockerignore
perennialtech Aug 13, 2024
592211a
Dockerfile: use scratch image for second stage
perennialtech Aug 13, 2024
f23f2f1
Dockerfile: use newer OpenSSL version (3.0.9)
perennialtech Aug 13, 2024
6e195d4
Dockerfile: use Alpine-based Rust image to skip building OpenSSL
perennialtech Aug 13, 2024
997dd7f
Dockerfile: expose to localhost only by default
perennialtech Aug 13, 2024
1547376
Dockerfile: allow building for different architectures (AMD64 and ARM64)
perennialtech Aug 13, 2024
2678c5c
compose.yaml: include command line
perennialtech Aug 13, 2024
c0dcc9c
compose.yaml: listen on 127.0.0.1 only
perennialtech Aug 13, 2024
aa93983
compose.yaml: remove healthcheck due to using scratch image
perennialtech Aug 13, 2024
6dc9ccd
README.md: restore old table format
perennialtech Aug 13, 2024
76ed2f4
rename compose.yaml to docker-compose.yml
perennialtech Aug 13, 2024
01ccd4c
docker-compose.yml: include version line
perennialtech Aug 13, 2024
3b09161
update .dockerignore
perennialtech Aug 13, 2024
8dbd4a6
README.md: update Docker instructions to expose only on localhost
perennialtech Aug 13, 2024
95b602e
add workflow to build and push container images to quay.io
perennialtech Aug 13, 2024
36295a9
docker-compose.yml: harden configuration
perennialtech Aug 13, 2024
0b8ddf7
docker workflow: add paths-ignore section
perennialtech Aug 13, 2024
6a07a2f
Dockerfile: let Rustup handle architecture detection
perennialtech Aug 13, 2024
1329745
Docker: further security hardening
perennialtech Aug 13, 2024
ff9a378
Dockerfile: Switch to Debian-based images and simplify build command
perennialtech Aug 14, 2024
3c0d848
Cargo.toml: correct note on optimisations
perennialtech Aug 14, 2024
d6f2463
docker-compose.yml: use quay.io image by default
perennialtech Aug 14, 2024
60737c7
rename docker-compose.yml to docker-compose.yaml
perennialtech Aug 14, 2024
ca4e3f8
compose: build image from local repo by default
perennialtech Aug 14, 2024
6b94718
Revert "Dockerfile: Switch to Debian-based images and simplify build …
perennialtech Aug 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Rust build artifacts
/target

# Git files
.git
.gitignore

# Docker Compose file
docker-compose.yaml

# Documentation
README.md
LICENSE
72 changes: 72 additions & 0 deletions .github/workflows/docker-build-push.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Build and Push Docker Image

# Define when this workflow will run
on:
push:
branches:
- master # Trigger on pushes to master branch
tags:
- '[0-9]+.[0-9]+.[0-9]+' # Trigger on semantic version tags
paths-ignore:
- 'Cargo.lock'
- 'LICENSE'
- 'README.md'
- 'docker-compose.yml'
workflow_dispatch: # Allow manual triggering of the workflow

# Define environment variables used throughout the workflow
env:
REGISTRY: quay.io
IMAGE_NAME: invidious/inv-sig-helper

jobs:
build-and-push:
runs-on: ubuntu-latest

steps:
# Step 1: Check out the repository code
- name: Checkout code
uses: actions/checkout@v3

# Step 2: Set up QEMU for multi-architecture builds
- name: Set up QEMU
uses: docker/setup-qemu-action@v2

# Step 3: Set up Docker Buildx for enhanced build capabilities
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

# Step 4: Authenticate with Quay.io registry
- name: Log in to Quay.io
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}

# Step 5: Extract metadata for Docker image tagging and labeling
- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# Define tagging strategy
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=sha,prefix={{branch}}-
# Define labels
labels: |
quay.expires-after=12w

# Step 6: Build and push the Docker image
- name: Build and push Docker image
uses: docker/build-push-action@v4
with:
context: .
push: true
platforms: linux/amd64,linux/arm64 # Build for multiple architectures
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
8 changes: 8 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,11 @@ lazy-regex = "3.1.0"
tub = "0.3.7"
tokio-util = { version = "0.7.10", features=["futures-io", "futures-util", "codec"]}
futures = "0.3.30"

# Compilation optimizations for release builds
# Increases compile time but typically produces a faster and smaller binary. Suitable for final releases but not for debug builds.
[profile.release]
lto = true
opt-level = 3
codegen-units = 1
panic = 'abort'
55 changes: 55 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Use the official Alpine-based Rust image as a parent image
FROM rust:1.80-alpine AS builder

# Set the working directory in the container
WORKDIR /usr/src/app

# Install build dependencies
RUN apk add --no-cache \
musl-dev \
openssl-dev \
openssl-libs-static \
pkgconfig \
patch

# Set environment variables for static linking
ENV OPENSSL_STATIC=yes
ENV OPENSSL_DIR=/usr

# Copy the current directory contents into the container
COPY . .

# Determine the target architecture and build the application
RUN RUST_TARGET=$(rustc -vV | sed -n 's/host: //p') && \
rustup target add $RUST_TARGET && \
RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target $RUST_TARGET

# Stage for creating the non-privileged user
FROM alpine:3.20 AS user-stage

RUN adduser -u 10001 -S appuser

# Stage for a smaller final image
FROM scratch

# Copy necessary files from the builder stage, using the correct architecture path
COPY --from=builder /usr/src/app/target/*/release/inv_sig_helper_rust /app/inv_sig_helper_rust
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

# Copy passwd file for the non-privileged user from the user-stage
COPY --from=user-stage /etc/passwd /etc/passwd

# Set the working directory
WORKDIR /app

# Expose port 12999
EXPOSE 12999

# Switch to non-privileged user
USER appuser

# Set the entrypoint to the binary name
ENTRYPOINT ["/app/inv_sig_helper_rust"]

# Set default arguments in CMD
CMD ["--tcp", "127.0.0.1:12999"]
125 changes: 103 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,97 +1,178 @@
# Protocol Format
# inv_sig_helper

`inv_sig_helper` is a Rust service that decrypts YouTube signatures and manages player information. It offers a TCP/Unix socket interface for signature decryption and related operations.

## Features

- Decrypt YouTube `n` and `s` signatures
- Fetch and update YouTube player data
- Provide signature timestamps and player status
- Efficient signature decryption with multi-threaded JavaScript execution

## Building and Running

### Prerequisites

- Rust 1.77 or later
- Cargo
- Patch
- openssl-devel

### Building

1. Clone the repository and navigate to the project directory:

```
git clone https://github.com/iv-org/inv_sig_helper.git
cd inv_sig_helper
```

2. Build the project:

```
cargo build --release
```

### Running

The service can run in Unix socket mode (default) or TCP mode:

1. Unix socket mode:

```
./target/release/inv_sig_helper_rust
```

This creates a Unix socket at `/tmp/inv_sig_helper.sock`.

2. TCP mode:

```
./target/release/inv_sig_helper_rust --tcp [IP:PORT]
```

If no IP:PORT is given, it defaults to `127.0.0.1:12999`.

## Docker

A Dockerfile is included for containerized deployment.

1. Build the image:

```
docker build -t inv_sig_helper .
```

2. Run the container:

```
docker run -p 127.0.0.1:12999:12999 inv_sig_helper
```

Or use Docker Compose:

```
docker compose up
```

## Protocol Format

All data-types bigger than 1 byte are stored in network endian (big-endian) unless stated otherwise.

## Request Base
### Request Base
perennialtech marked this conversation as resolved.
Show resolved Hide resolved
| Name | Size (bytes) | Description |
|-----------|--------------|--------------------------------------|
|opcode | 1 | The operation code to perform, A list of operations currently supported (and their data) can be found in the **Operations** chapter |
|request_id | 4 | The ID for the current request, Used to distinguish responses in the current connection |

The data afterwards depends on the supplied opcode, Please consult the **Operations** chapter for more information.

## Response Base
### Response Base
| Name | Size (bytes) | Description |
|------------|--------------|---------------------------------------|
|request_id | 4 | The ID for the request that this response is meant for |
|size | 4 | Size of the response (excluding size of request id)|

The data afterwards depends on the supplied opcode, Please consult the **Operations** chapter for more information.

## Operations
### `FORCE_UPDATE` (0x00)
### Operations
#### `FORCE_UPDATE` (0x00)
Forces the server to re-fetch the YouTube player, and extract the necessary components from it (`nsig` function code, `sig` function code, signature timestamp).

#### Request
##### Request
*No additional data required*

#### Response
##### Response
| Name | Size (bytes) | Description |
|------|--------------|-------------|
|status| 2 | The status code of the request: `0xF44F` if successful, `0xFFFF` if no updating is required (YouTube's player ID is equal to the server's current player ID), `0x0000` if an error occurred |

### `DECRYPT_N_SIGNATURE` (0x01)
#### `DECRYPT_N_SIGNATURE` (0x01)
Decrypt a provided `n` signature using the server's current `nsig` function code, and return the result (or an error).

#### Request
##### Request
| Name | Size (bytes) | Description |
|------|--------------|-------------------------------------|
|size | 2 | The size of the encrypted signature |
|string| *`size`* | The encrypted signature |

#### Response
##### Response
| Name | Size (bytes) | Description |
|------|--------------|------------------------------------------------------------------|
|size | 2 | The size of the decrypted signature, `0x0000` if an error occurred |
|string| *`size`* | The decrypted signature |

### `DECRYPT_SIGNATURE` (0x02)
#### `DECRYPT_SIGNATURE` (0x02)
Decrypt a provided `s` signature using the server's current `sig` function code, and return the result (or an error).

#### Request
##### Request
| Name | Size (bytes) | Description |
|------|--------------|-------------------------------------|
|size | 2 | The size of the encrypted signature |
|string| *`size`* | The encrypted signature |

#### Response
##### Response
| Name | Size (bytes) | Description |
|------|--------------|------------------------------------------------------------------|
|size | 2 | The size of the decrypted signature, `0x0000` if an error occurred |
|string| *`size`* | The decrypted signature |

### `GET_SIGNATURE_TIMESTAMP` (0x03)
#### `GET_SIGNATURE_TIMESTAMP` (0x03)
Get the signature timestamp from the server's current player, and return it in the form of a 64-bit integer. If there's no player, it will return 0 in the `timestamp` (Please check with `PLAYER_STATUS` if the server has a player)

#### Request
##### Request
No additional data required

#### Response
##### Response
| Name | Size (bytes) | Description |
|---------|--------------|----------------------------------------------------------|
|timestamp| 8 | The signature timestamp from the server's current player |

### `PLAYER_STATUS` (0x04)
#### `PLAYER_STATUS` (0x04)
Get the server's information about the current player.

#### Request
##### Request
No additional data required

#### Response
##### Response

| Name | Size (bytes) | Description |
|----------|--------------|-------------|
|has_player| 1 | If the server has a player, this variable will be `0xFF`. or else, it will be `0x00`|
|player_id | 4 | The server's current player ID. If the server has no player, this will always be `0x00000000`|

### `PLAYER_UPDATE_TIMESTAMP` (0x05)
#### `PLAYER_UPDATE_TIMESTAMP` (0x05)
Get the time of the last player update, The time is represented as seconds since the last update

#### Request
##### Request
No additional data required

#### Response
##### Response

| Name | Size (bytes) | Description |
|----------|--------------|-------------|
|timestamp | 8 | Seconds since the last player update |

## License

This project is open source under the AGPL-3.0 license.
19 changes: 19 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: "3"
services:
inv_sig_helper:
build:
context: .
dockerfile: Dockerfile
# image: quay.io/invidious/inv-sig-helper:latest
command: ["--tcp", "127.0.0.1:12999"]
ports:
- 127.0.0.1:12999:12999
environment:
- RUST_LOG=info
restart: unless-stopped
cap_drop:
- ALL
read_only: true
user: 10001:10001
unixfox marked this conversation as resolved.
Show resolved Hide resolved
security_opt:
- no-new-privileges:true