Skip to content

Commit

Permalink
chore: improve README, docker-compose, config example
Browse files Browse the repository at this point in the history
  • Loading branch information
vas3k committed May 13, 2024
1 parent a51c656 commit b22fcd5
Show file tree
Hide file tree
Showing 4 changed files with 112 additions and 29 deletions.
129 changes: 106 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,73 +4,156 @@
<h1>PEPIC</h1>
</div>

Pepic is a small self-hosted media proxy that helps me to upload, store, serve and convert pictures and videos on my own servers locally.
Pepic is a small self-hosted media proxy that helps me to upload, store, serve and convert pictures and videos on my own servers.

Currently, I use it as a main storage for media files in my [pet-projects](https://github.com/vas3k/vas3k.club) and on [my blog](https://vas3k.blog).

Pepic can upload and optimize media files in-flight to save you money and bandwidth. It's highly recommended to use it in combination with Cloudflare CDN for better caching.
Pepic can upload and optimize media files in-flight to save you money and bandwidth. It's highly recommended to set ip up in combination with Cloudflare CDN for better caching.

Internally, Pepic it uses [ffmpeg](https://ffmpeg.org/download.html) for videos and [vips](https://libvips.github.io/libvips/install.html) for images, which makes it quite fast and supports many media file formats.
Internally it uses [ffmpeg](https://ffmpeg.org/download.html) for videos and [vips](https://libvips.github.io/libvips/install.html) for images, which makes it quite fast and supports many media file formats.

Images: **JPG, PNG, GIF, WEBP, SVG, HEIF, TIFF, AVIF, etc**

Video: **basically everything ffmpeg supports**

Pepic is open source, however it's not meant to be used by anyone. Only if you're brave. Scroll down this README for better alternatives.
Pepic is open source, however it's not meant to be used by anyone. Only if you're brave (like me). Scroll down this README for better alternatives.


## Main Features
## Features

- **Local files upload**: Accept files in multipart/form-data or bytes and store them to a local directory.
- **Local files uploads**: Upload files as multipart/form-data or as a simple byte stream and store them to a local directory.
- **Automatic GIF to video conversion**: Convert GIFs to videos because GIFs suck, slow down web pages, and don't support hardware acceleration.
- **Media format conversion and optimization**: Convert and optimize media files on-the-fly.
- **Image and video conversion and optimization**: Transcode and optimize media files on upload or on-the-fly.
- **Dynamic resizing**: Easily resize images in real-time by modifying the URL, which helps in reducing bandwidth and storage space on devices.
- **High performance**: Pepic uses native libraries like `ffmpeg` and `vips` for video and image processing to ensure high performance and fast processing times.
- **Local and containerized environments**: Designed to run smoothly in both local environments and within Docker containers, making it versatile for development and deployment.
- **Custom configuration**: Flexible configuration options through `config.yml`, allowing adjustments to image size, quality, automatic conversion, templates, etc.
- **Custom configuration**: Flexible configuration options through `[config.yml](./etc/pepic/config.yml)`, allowing adjustments to image size, quality, automatic conversion, templates, etc.

![](static/images/screenshot1.png)

## 🤖 How to Run

1. Install `vips` and `ffmpeg` first, as they are external dependencies.
1. Install `vips` and `ffmpeg` first, as they are external dependencies

```bash
brew install vips ffmpeg
```

2. Use the following command to start a local server on [localhost:8118](http://localhost:8118).
2. Clone this repo

```bash
git clone [email protected]:vas3k/pepic.git
cd pepic
```

3. Run the following command to build and start the app

```bash
go run main.go serve --config ./etc/pepic/config.yml
```

> ⚠️ If you're getting `invalid flag in pkg-config` error, run `brew install pkg-config` and `export CGO_CFLAGS_ALLOW="-Xpreprocessor"`. Then try `go run` again.
3. Enjoy!
3. Go to [localhost:8118](http://localhost:8118) and enjoy!

## 🐳 Running in Docker
## 🐳 Using Docker Compose

1. Get [Docker](https://www.docker.com/get-started)
You can find [docker-compose.example.yml](./docker-compose.example.yml) in this repo and adapt it to your own needs.

2. Clone the repo
1. Get [Docker](https://www.docker.com/get-started) and [Docker Compose](https://www.digitalocean.com/community/tutorial-collections/how-to-install-docker-compose)

2. Download the Docker Compose example file and save it as `docker-compose.yml` on your local machine

```bash
git clone [email protected]:vas3k/pepic.git
cd pepic
curl https://raw.githubusercontent.com/vas3k/pepic/master/docker-compose.example.yml -o docker-compose.yml
```

3. Build and run the app
3. Now run it

```bash
docker build .
docker run -p 8118:8118 -v ${PWD}/uploads:/app/uploads $(docker build -q .)
docker-compose up
```

4. Go to [http://localhost:8118](http://localhost:8118) and try uploading something. You should see uploaded images or videos in the local directory (`./uploads`) after that.


## 🧶 Usage

### Configuration options

```yaml
global:
host: 0.0.0.0
port: 8118
base_url: "http://0.0.0.0:8118/" # trailing slash is important
secret_code: "" # secret word to protect you from strangers (don't use your password here, it's stored as plain text)
max_upload_size: "500M" # number + K, M, G, T or P
file_tree_split_chars: 3 # abcde.jpg -> ab/cd/e.jpg (never change this after release!)

storage:
type: fs
dir: uploads/

images:
store_originals: false # use "true" if you want byte-by-byte match of uploaded files (useful for photo blogs)
original_length: 1900 # long side length in px to auto-resize originals (only if store_originals=false)
auto_convert: false # mime type to auto-convert uploaded images ("image/jpeg", "image/png" or false)
live_resize: true # enables special URLs that return resized images (increases storage usage)
jpeg_quality: 95 # default quality for any saved jpegs
png_compression: 0 # 0 - default, -1 - no compression, -2 - best speed, -3 - best compression (yes, with minus)
gif_convert: "video/mp4" # video format for auto-converting gifs (ignored on store_originals=true)

videos:
store_originals: false # use "true" if you want to store original files (browser compatibility is on you)
original_length: 720 # resize uploaded videos (only if store_originals=false)
live_resize: false # turned off by default to save disk space and your cpu (always returns original)
auto_convert: "video/mp4" # mime type to auto-convert uploaded images (for example "video/mp4")
ffmpeg:
temp_dir: "/tmp" # temp directory for video transcoding
preset: "slow" # ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow, placebo
crf: 24 # quality factor — 0-51, where 0 is lossless, 51 — pixelated shit. 23-28 recommended.
buffer_size: 1024000 # other standard ffmpeg params, you can google them
video_codec: "libx264"
video_bitrate: "1024k"
video_profile: "main"
audio_codec: "aac"
audio_bitrate: "128k"
mov_flags: "+faststart"
pix_fmt: "yuv420p"

meta: # optional, only if you use web interface
image_templates: # add your custom templates here for easier copy-paste
- title: "URL"
template: "{{ file.Url }}"
- title: "Simple Markdown"
template: "![]({{ file.Url }})"
video_templates:
- title: "URL"
template: "{{ file.Url }}"
- title: "Simple Markdown"
template: "![]({{ file.Url }})"
multi_templates:
- title: "2 in a row"
template: "{% for file in files %}![]({{ file.Url }}) {% endfor %}"

```

4. Go to [http://localhost:8118](http://localhost:8118) and try uploading something. You should see uploaded images or videos in the data directory (`./uploads`) after that.
### Resizing images on demand

If your image URL looks like this: **`https://imgs.com/file.jpg`**

Add /500/ to its URL to get 500px (on the long side) version: **`https://imgs.com/500/file.jpg`**

Works only if `live_resize` option is set to `true`. If `live_resize=false` — it returns the original version. Same for video transcoding (where it's off by default).


### Converting file formats on demand

// Not implemented yet, sorry...


5. Try to resize an image by adding a number of pixels to its URL. For example: `https://localhost:8118/file.jpg -> https://localhost:8118/500/file.jpg`
![](static/images/screenshot2.png)

6. Check out the [etc/pepic/config.yml](etc/pepic/config.yml) file. Some stuff is turned off by default. You can tweak them for yourself and rebuild the Docker again (step 3) to apply them.


## 🚢 Production Deployment
Expand All @@ -95,7 +178,7 @@ global:
Don't forget to mount upload volume to store files on host (or you can lose those files when the container is killed).
```bash
docker run -p 8118:8118 -v /host/dir/uploads:/app/uploads --restart=always $(docker build -q .)
docker run -p 8118:8118 -v /host/dir/uploads:/app/uploads --restart=unless-stopped $(docker build -q .)
```

If you prefer docker-compose, you can use it too. Check out the included [docker-compose.example.yml](docker-compose.example.yml).
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.example.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: "3.8"
services:
app:
build: .
image: ghcr.io/vas3k/pepic:${GITHUB_SHA:-latest}
environment: # check out pepic/config/app.go for more env variables
- BASE_URL=https://i.vas3k.ru/
- STORAGE_DIR=/uploads
Expand Down
10 changes: 5 additions & 5 deletions etc/pepic/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ images:
auto_convert: false # mime type to auto-convert uploaded images ("image/jpeg", "image/png" or false)
live_resize: true # enables special URLs that return resized images (increases storage usage)
jpeg_quality: 95 # default quality of any saved jpeg
png_compression: 0 # 0 - default, -1 - no compression, -2 - best speed, -3 - best compression
png_compression: 0 # 0 - default, -1 - no compression, -2 - best speed, -3 - best compression (yes, with minus)
gif_convert: "video/mp4" # video format for auto-converting gifs (turned off then store_originals=true)

videos:
store_originals: false # use "true" if you want to store original files (browser compatibility is on you)
original_length: 480 # resize uploaded videos (only if store_originals=false)
original_length: 720 # resize uploaded videos (only if store_originals=false)
live_resize: false # turned off by default to save disk space and your cpu (always returns original)
auto_convert: "video/mp4" # mime type to auto-convert uploaded images (for example "video/mp4")
ffmpeg:
Expand All @@ -30,15 +30,15 @@ videos:
crf: 24 # quality factor — 0-51, where 0 is lossless, 51 — pixelated shit. 23-28 recommended.
buffer_size: 1024000
video_codec: "libx264"
video_bitrate: "512k"
video_bitrate: "1024k"
video_profile: "main"
audio_codec: "aac"
audio_bitrate: "128k"
mov_flags: "+faststart"
pix_fmt: "yuv420p"

meta:
image_templates: # you can add your custom blocks to make it easier to copy-paste
meta: # optional, only if you use web interface
image_templates: # add your custom templates here for easier copy-paste
- title: "URL"
template: "{{ file.Url }}"
- title: "Simple Markdown"
Expand Down
Binary file added static/images/screenshot2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit b22fcd5

Please sign in to comment.