Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Script: Hoarder LXC #567

Merged
merged 29 commits into from
Dec 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
215e66c
Add Hoarder script
vhsdream Nov 27, 2024
54cee34
Merge branch 'community-scripts:main' into hoarder
vhsdream Nov 27, 2024
772fd56
Point build.func to correct repo
vhsdream Nov 27, 2024
f3b28fc
I hate testing these scripts
vhsdream Nov 27, 2024
2bfc305
no comment
vhsdream Nov 27, 2024
aace44b
Quiesce systemctl enable output; add move DB to hoarder_data
vhsdream Nov 27, 2024
e80a917
add timeout to hoarder-workers
vhsdream Nov 27, 2024
8f65698
Restore attribution
vhsdream Nov 28, 2024
588d363
fix sourcefix source URLs; quiesce migration post-install
vhsdream Nov 28, 2024
e244959
Update hoarder-install.sh to move commands to separate lines
vhsdream Nov 28, 2024
e1c52a1
Delete install/bazarr-install.sh
MickLesk Nov 28, 2024
f2bd029
Delete ct/bazarr.sh
MickLesk Nov 28, 2024
7a805f0
Update msg_info
MickLesk Nov 28, 2024
9757d07
Update msg_ok
MickLesk Nov 28, 2024
fc259af
Update msg_info
MickLesk Nov 28, 2024
322565b
Update ct/hoarder.sh
MickLesk Nov 28, 2024
fe2949a
Remove bazarr.sh and bazarr-install.sh changes from PR
MickLesk Nov 28, 2024
70d13d2
Add Json for Website
MickLesk Nov 28, 2024
218bbae
Apply suggestions from code review
vhsdream Nov 28, 2024
08e35c9
Update ct/hoarder.sh
MickLesk Nov 28, 2024
60d5614
Update hoarder.sh
vhsdream Nov 28, 2024
ed25d43
Merge branch 'community-scripts:main' into hoarder
vhsdream Nov 28, 2024
a0f6e14
remove unnecessary dependencies; clean up worker & web install; remov…
vhsdream Nov 28, 2024
993c7e8
Merge branch 'community-scripts:main' into hoarder
vhsdream Nov 29, 2024
ccf08e7
Apply suggestions from code review
vhsdream Nov 29, 2024
91fa409
Update hoarder.sh (&>/dev/null)
MickLesk Nov 29, 2024
47def6a
Fix broken DB migration, other small changes
vhsdream Nov 29, 2024
091bf8a
Update resources in hoarder.json
vhsdream Nov 30, 2024
6d41b7e
Update install/hoarder-install.sh with commented refs for AI tagging
vhsdream Nov 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions ct/hoarder.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
#!/usr/bin/env bash
source <(curl -s https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/misc/build.func)
# Copyright (c) 2021-2024 tteck
# Author: MickLesk (Canbiz) & vhsdream
# License: MIT
# https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE

function header_info {
clear
cat <<"EOF"
__ __ __
/ / / /___ ____ __________/ /__ _____
/ /_/ / __ \/ __ `/ ___/ __ / _ \/ ___/
/ __ / /_/ / /_/ / / / /_/ / __/ /
/_/ /_/\____/\__,_/_/ \__,_/\___/_/

EOF
}
header_info
echo -e "Loading..."
APP="Hoarder"
var_disk="8"
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
var_cpu="2"
var_ram="4096"
var_os="debian"
var_version="12"
variables
color
catch_errors

function default_settings() {
CT_TYPE="1"
PW=""
CT_ID=$NEXTID
HN=$NSAPP
DISK_SIZE="$var_disk"
CORE_COUNT="$var_cpu"
RAM_SIZE="$var_ram"
BRG="vmbr0"
NET="dhcp"
GATE=""
APT_CACHER=""
APT_CACHER_IP=""
DISABLEIP6="no"
MTU=""
SD=""
NS=""
MAC=""
VLAN=""
SSH="no"
VERB="no"
echo_default
}

function update_script() {
header_info
check_container_storage
check_container_resources
if [[ ! -d /opt/hoarder ]]; then msg_error "No ${APP} Installation Found!"; exit; fi
RELEASE=$(curl -s https://api.github.com/repos/hoarder-app/hoarder/releases/latest | grep "tag_name" | awk '{print substr($2, 3, length($2)-4) }')
PREV_RELEASE=$(cat /opt/${APP}_version.txt)
if [[ ! -f /opt/${APP}_version.txt ]] || [[ "${RELEASE}" != "${PREV_RELEASE}" ]]; then
msg_info "Stopping Services"
systemctl stop hoarder-web hoarder-workers hoarder-browser
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, you don't need to stop the browser for a hoarder update

msg_ok "Stopped Services"
msg_info "Updating ${APP} to v${RELEASE}"
cd /opt
mv /opt/hoarder/.env /opt/.env
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another reason why the env files should probably be in /etc/hoarder instead.

rm -rf /opt/hoarder
wget -q "https://github.com/hoarder-app/hoarder/archive/refs/tags/v${RELEASE}.zip"
unzip -q v${RELEASE}.zip
mv hoarder-${RELEASE} /opt/hoarder
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
cd /opt/hoarder/apps/web
pnpm install --frozen-lockfile &>/dev/null
pnpm exec next build --experimental-build-mode compile &>/dev/null
cp -r /opt/hoarder/apps/web/.next/standalone/apps/web/server.js /opt/hoarder/apps/web
cd /opt/hoarder/apps/workers
pnpm install --frozen-lockfile &>/dev/null
export DATA_DIR=/opt/hoarder_data
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
cd /opt/hoarder/packages/db
pnpm migrate &>/dev/null
mv /opt/.env /opt/hoarder/.env
sed -i "s/SERVER_VERSION=${PREV_RELEASE}/SERVER_VERSION=${RELEASE}/" /opt/hoarder/.env
msg_ok "Updated ${APP} to v${RELEASE}"

msg_info "Starting Services"
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
systemctl start hoarder-browser hoarder-workers hoarder-web
msg_ok "Started Services"
msg_info "Cleaning up"
rm -R /opt/v${RELEASE}.zip
echo "${RELEASE}" >/opt/${APP}_version.txt
msg_ok "Cleaned"
msg_ok "Updated Successfully"
else
msg_ok "No update required. ${APP} is already at ${RELEASE}."
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
fi
exit
}

start
build_container
description

msg_ok "Completed Successfully!\n"
echo -e "${APP} Setup should be reachable by going to the following URL.
${BL}http://${IP}:3000${CL} \n"
180 changes: 180 additions & 0 deletions install/hoarder-install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
#!/usr/bin/env bash

# Copyright (c) 2021-2024 tteck
# Author: MickLesk (Canbiz) & vhsdream
# License: MIT
# https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE

source /dev/stdin <<< "$FUNCTIONS_FILE_PATH"
color
verb_ip6
catch_errors
setting_up_container
network_check
update_os

msg_info "Installing Dependencies"
$STD apt-get install -y \
g++ \
build-essential \
curl \
git \
sudo \
gnupg \
ca-certificates \
chromium \
mc
msg_ok "Installed Dependencies"

msg_info "Installing Additional Tools"
wget -q https://github.com/Y2Z/monolith/releases/latest/download/monolith-gnu-linux-x86_64 -O /usr/bin/monolith
chmod +x /usr/bin/monolith
wget -q https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux -O /usr/bin/yt-dlp
chmod +x /usr/bin/yt-dlp
msg_ok "Installed Additional Tools"

msg_info "Installing Meilisearch"
cd /tmp
wget -q https://github.com/meilisearch/meilisearch/releases/latest/download/meilisearch.deb
$STD dpkg -i meilisearch.deb
wget -q https://raw.githubusercontent.com/meilisearch/meilisearch/latest/config.toml -O /etc/meilisearch.toml
MASTER_KEY=$(openssl rand -base64 12)
sed -i \
-e 's|^env =.*|env = "production"|' \
-e "s|^# master_key =.*|master_key = \"$MASTER_KEY\"|" \
-e 's|^db_path =.*|db_path = "/var/lib/meilisearch/data"|' \
-e 's|^dump_dir =.*|dump_dir = "/var/lib/meilisearch/dumps"|' \
-e 's|^snapshot_dir =.*|snapshot_dir = "/var/lib/meilisearch/snapshots"|' \
-e 's|^# no_analytics = true|no_analytics = true|' \
/etc/meilisearch.toml
msg_ok "Installed Meilisearch"

msg_info "Installing Node.js"
mkdir -p /etc/apt/keyrings
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_22.x nodistro main" >/etc/apt/sources.list.d/nodesource.list
$STD apt-get update
$STD apt-get install -y nodejs
msg_ok "Installed Node.js"

msg_info "Installing Hoarder"
cd /opt
RELEASE=$(curl -s https://api.github.com/repos/hoarder-app/hoarder/releases/latest | grep "tag_name" | awk '{print substr($2, 3, length($2)-4) }')
wget -q "https://github.com/hoarder-app/hoarder/archive/refs/tags/v${RELEASE}.zip"
unzip -q v${RELEASE}.zip
mv hoarder-${RELEASE} /opt/hoarder
cd /opt/hoarder
corepack enable
export PUPPETEER_SKIP_DOWNLOAD="true"
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
export NEXT_TELEMETRY_DISABLED=1
export CI="true"
cd /opt/hoarder/apps/web
$STD pnpm install --frozen-lockfile
$STD pnpm exec next build --experimental-build-mode compile
cp -r /opt/hoarder/apps/web/.next/standalone/apps/web/server.js /opt/hoarder/apps/web
cd /opt/hoarder/apps/workers
$STD pnpm install --frozen-lockfile

export DATA_DIR=/opt/hoarder_data
HOARDER_SECRET=$(openssl rand -base64 36 | cut -c1-24)
cat <<EOF >/opt/hoarder/.env
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be in /etc/hoarder/config.env instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I originally had the env file in /etc/hoarder.env as per your suggestion, it was implied to me that it is not standard to do that for scripts in this repo. I'll leave it up to the maintainers to decide how to proceed with this sort of thing.

SERVER_VERSION=$RELEASE
NEXTAUTH_SECRET="$HOARDER_SECRET"
NEXTAUTH_URL="http://localhost:3000"
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
DATA_DIR="$DATA_DIR"
MEILI_ADDR="http://127.0.0.1:7700"
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
MEILI_MASTER_KEY="$MASTER_KEY"
BROWSER_WEB_URL="http://127.0.0.1:9222"
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
vhsdream marked this conversation as resolved.
Show resolved Hide resolved

# If you're planning to use OpenAI for tagging. Uncomment the following line:
# OPENAI_API_KEY="<API_KEY>"

# If you're planning to use ollama for tagging, uncomment the following lines:
# OLLAMA_BASE_URL="<OLLAMA_ADDR>"

# You can change the models used by uncommenting the following lines, and changing them according to your needs:
# INFERENCE_TEXT_MODEL="gpt-4o-mini"
# INFERENCE_IMAGE_MODEL="gpt-4o-mini"
EOF
echo "${RELEASE}" >"/opt/${APPLICATION}_version.txt"
msg_ok "Installed Hoarder"

msg_info "Running Database Migration"
mkdir -p ${DATA_DIR}
cd /opt/hoarder/packages/db
$STD pnpm migrate
msg_ok "Database Migration Completed"

msg_info "Creating Services"
cat <<EOF >/etc/systemd/system/meilisearch.service
[Unit]
Description=Meilisearch
After=network.target

[Service]
ExecStart=/usr/bin/meilisearch --config-file-path /etc/meilisearch.toml
Restart=always

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/systemd/system/hoarder-web.service
[Unit]
Description=Hoarder Web
Wants=network.target hoarder-workers.service
After=network.target hoarder-workers.service

[Service]
ExecStart=pnpm start
WorkingDirectory=/opt/hoarder/apps/web
EnvironmentFile=/opt/hoarder/.env
Restart=always

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/systemd/system/hoarder-browser.service
[Unit]
Description=Hoarder Headless Browser
After=network.target

[Service]
User=root
ExecStart=/usr/bin/chromium --headless --no-sandbox --disable-gpu --disable-dev-shm-usage --remote-debugging-address=127.0.0.1 --remote-debugging-port=9222 --hide-scrollbars
Restart=always

[Install]
WantedBy=multi-user.target
EOF

cat <<EOF >/etc/systemd/system/hoarder-workers.service
[Unit]
Description=Hoarder Workers
Wants=network.target hoarder-browser.service meilisearch.service
After=network.target hoarder-browser.service meilisearch.service

[Service]
ExecStart=pnpm start:prod
WorkingDirectory=/opt/hoarder/apps/workers
EnvironmentFile=/opt/hoarder/.env
Restart=always
TimeoutStopSec=5

[Install]
WantedBy=multi-user.target
EOF

systemctl -q enable --now meilisearch.service hoarder-browser.service hoarder-workers.service hoarder-web.service
msg_ok "Created Services"

motd_ssh
customize

msg_info "Cleaning up"
rm -rf /tmp/meilisearch.deb
vhsdream marked this conversation as resolved.
Show resolved Hide resolved
rm -f /opt/v${RELEASE}.zip
$STD apt-get autoremove -y
$STD apt-get autoclean -y
msg_ok "Cleaned"
34 changes: 34 additions & 0 deletions json/hoarder.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"name": "Hoarder",
"slug": "hoarder",
"categories": [
14
],
"date_created": "2024-11-28",
"type": "ct",
"updateable": true,
"privileged": false,
"interface_port": 3000,
"documentation": "https://docs.hoarder.app/",
"website": "https://hoarder.app/",
"logo": "https://raw.githubusercontent.com/hoarder-app/hoarder/refs/heads/main/screenshots/logo.png",
"description": "Hoarder is an AI-powered bookmarking tool that helps you save and organize your digital content. It automatically tags your links, notes, and images, making them easy to find later. With features like auto-fetching, lists, and full-text search, Hoarder is the perfect tool for anyone who wants to keep track of their digital life.",
"install_methods": [
{
"type": "default",
"script": "/ct/hoarder.sh",
"resources": {
"cpu": 2,
"ram": 4096,
"hdd": 8,
"os": "Debian",
"version": "12"
}
}
],
"default_credentials": {
"username": null,
"password": null
},
"notes": []
}