From bd8cd834c176f4bfb63ff5b99e1e885c7bd87bbe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E3=83=80=E3=83=B3=E3=82=AF=E3=83=87=E3=83=AB=20//=20?= =?UTF-8?q?=E5=8D=83=E9=B6=B4?= Date: Fri, 13 Nov 2020 13:09:03 +0000 Subject: [PATCH 1/2] Update README.md --- README.md | 144 ++++++++++++------------------------------------------ 1 file changed, 32 insertions(+), 112 deletions(-) diff --git a/README.md b/README.md index bba37453bb1..b0a908f11ec 100644 --- a/README.md +++ b/README.md @@ -1,139 +1,59 @@ -# What is this repo about? -This is a telegram bot writen in python for mirroring files on the internet to our beloved Google Drive. +![Ayanami](https://telegra.ph/file/87b7878ee9d6273af566f.jpg) -# Inspiration -This project is heavily inspired from @out386 's telegram bot which is written in JS. +# Ayanami +A personal torrent leech bot for Dank-del -# Features supported: -- Mirroring direct download links to google drive -- Download progress -- Upload progress -- Download/upload speeds and ETAs -- Docker support -- Uploading To Team Drives. -- Index Link support -- Service account support -- Mirror all youtube-dl supported links -- Mirror telegram files +Fork of [python-aria-mirror-bot](https://github.com/lzzy12/python-aria-mirror-bot/) -# Upcoming features (TODOs): -# How to deploy? -Deploying is pretty much straight forward and is divided into several steps as follows: -## Installing requirements -- Clone this repo: -``` -git clone https://github.com/lzzy12/python-aria-mirror-bot mirror-bot/ -cd mirror-bot -``` - -- Install requirements -For Debian based distros -``` -sudo apt install python3 -``` -Install Docker by following the [official docker docs](https://docs.docker.com/engine/install/debian/) - - -- For Arch and it's derivatives: -``` -sudo pacman -S docker python -``` - -- Install dependencies for running setup scripts: -```shell script -pip3 install -r requirements-cli.txt -``` -## Setting up config file +## Deploying on Heroku +- Run the script to generate token file(token.pickle) for Google Drive: ``` -cp config_sample.env config.env +python3 generate_drive_token.py ``` -- Remove the first line saying: +- Install [Heroku cli](https://devcenter.heroku.com/articles/heroku-cli) +- Login into your heroku account with command: ``` -_____REMOVE_THIS_LINE_____=True +heroku login ``` -Fill up rest of the fields. Meaning of each fields are discussed below: -- **BOT_TOKEN** : The telegram bot token that you get from @BotFather -- **GDRIVE_FOLDER_ID** : This is the folder ID of the Google Drive Folder to which you want to upload all the mirrors. -- **DOWNLOAD_DIR** : The path to the local folder where the downloads should be downloaded to -- **DOWNLOAD_STATUS_UPDATE_INTERVAL** : A short interval of time in seconds after which the Mirror progress message is updated. (I recommend to keep it 5 seconds at least) -- **OWNER_ID** : The Telegram user ID (not username) of the owner of the bot -- **AUTO_DELETE_MESSAGE_DURATION** : Interval of time (in seconds), after which the bot deletes it's message (and command message) which is expected to be viewed instantly. Note: Set to -1 to never automatically delete messages -- **IS_TEAM_DRIVE** : (Optional field) Set to "True" if GDRIVE_FOLDER_ID is from a Team Drive else False or Leave it empty. -- **USE_SERVICE_ACCOUNTS**: (Optional field) (Leave empty if unsure) Whether to use service accounts or not. For this to work see "Using service accounts" section below. -- **INDEX_URL** : (Optional field) Refer to https://github.com/maple3142/GDIndex/ The URL should not have any trailing '/' -- **TELEGRAM_API** : This is to authenticate to your telegram account for downloading Telegram files. You can get this from https://my.telegram.org DO NOT put this in quotes. -- **TELEGRAM_HASH** : This is to authenticate to your telegram account for downloading Telegram files. You can get this from https://my.telegram.org -- **USER_SESSION_STRING** : Session string generated by running: +- Create a new heroku app: ``` -python3 generate_string_session.py +heroku create appname ``` -Note: You can limit maximum concurrent downloads by changing the value of MAX_CONCURRENT_DOWNLOADS in aria.sh. By default, it's set to 2 - -## Getting Google OAuth API credential file - -- Visit the [Google Cloud Console](https://console.developers.google.com/apis/credentials) -- Go to the OAuth Consent tab, fill it, and save. -- Go to the Credentials tab and click Create Credentials -> OAuth Client ID -- Choose Other and Create. -- Use the download button to download your credentials. -- Move that file to the root of mirror-bot, and rename it to credentials.json -- Visit [Google API page](https://console.developers.google.com/apis/library) -- Search for Drive and enable it if it is disabled -- Finally, run the script to generate token file (token.pickle) for Google Drive: -``` -pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib -python3 generate_drive_token.py +- Select This App in your Heroku-cli: ``` -## Deploying - -- Start docker daemon (skip if already running): +heroku git:remote -a appname ``` -sudo dockerd +- Change Dyno Stack to a Docker Container: ``` -- Build Docker image: +heroku stack:set container ``` -sudo docker build . -t mirror-bot +- Add Private Credentials and Config Stuff: ``` -- Run the image: +git add -f credentials.json token.pickle config.env ``` -sudo docker run mirror-bot +- Commit new changes: ``` - -# Using service accounts for uploading to avoid user rate limit -For Service Account to work, you must set USE_SERVICE_ACCOUNTS="True" in config file or environment variables -Many thanks to [AutoRClone](https://github.com/xyou365/AutoRclone) for the scripts -**NOTE:** Using service accounts is only recommended while uploading to a team drive. -## Generating service accounts -Step 1. Generate service accounts [What is service account](https://cloud.google.com/iam/docs/service-accounts) ---------------------------------- -Let us create only the service accounts that we need. -**Warning:** abuse of this feature is not the aim of this project and we do **NOT** recommend that you make a lot of projects, just one project and 100 sa allow you plenty of use, its also possible that over abuse might get your projects banned by google. - +git commit -m "Added Creds." ``` -Note: 1 service account can copy around 750gb a day, 1 project can make 100 service accounts so that's 75tb a day, for most users this should easily suffice. +- Push Code to Heroku: ``` - -`python3 gen_sa_accounts.py --quick-setup 1 --new-only` - -A folder named accounts will be created which will contain keys for the service accounts - -NOTE: If you have created SAs in past from this script, you can also just re download the keys by running: +git push heroku master --force ``` -python3 gen_sa_accounts.py --download-keys project_id +- Restart Worker by these commands: ``` - -### Add all the service accounts to the Team Drive -- Run: +heroku ps:scale worker=0 ``` -python3 add_to_team_drive.py -d SharedTeamDriveSrcID ``` - -# Youtube-dl authentication using .netrc file -For using your premium accounts in youtube-dl, edit the netrc file (in the root directory of this repository) according to following format: +heroku ps:scale worker=1 ``` -machine host login username password my_youtube_password +Heroku-Note: Doing authorizations ( /authorize command ) through telegram wont be permanent as heroku uses ephemeral filesystem. They will be reset on each dyno boot. As a workaround you can: +- Make a file authorized_chats.txt and write the user names and chat_id of you want to authorize, each separated by new line +- Then force add authorized_chats.txt to git and push it to heroku ``` -where host is the name of extractor (eg. youtube, twitch). Multiple accounts of different hosts can be added each separated by a new line +git add authorized_chats.txt -f +git commit -asm "Added hardcoded authorized_chats.txt" +git push heroku heroku:master +``` \ No newline at end of file From ed8011ac1a60e1966a6fb0fe7f0b562957c7e14f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E3=83=80=E3=83=B3=E3=82=AF=E3=83=87=E3=83=AB=20//=20?= =?UTF-8?q?=E5=8D=83=E9=B6=B4?= Date: Fri, 13 Nov 2020 14:25:01 +0000 Subject: [PATCH 2/2] Update Dockerfile --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 2537f0ee95a..15926679ff4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -18,5 +18,5 @@ ENV LC_ALL en_US.UTF-8 COPY . . COPY netrc /root/.netrc RUN chmod +x aria.sh - +RUN rm README.md CMD ["bash","start.sh"]