Skip to content

Commit

Permalink
Merge branch 'master' of https://gitlab.com/Dank-del/ayanamileechbot
Browse files Browse the repository at this point in the history
…into master
  • Loading branch information
Chizuru committed Nov 17, 2020
2 parents fd93bff + ed8011a commit 2f0ac03
Show file tree
Hide file tree
Showing 2 changed files with 33 additions and 113 deletions.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,5 @@ RUN python3 -m pip install -r requirements.txt
COPY . .
COPY netrc /root/.netrc
RUN chmod +x aria.sh

RUN rm README.md
CMD ["bash","start.sh"]
144 changes: 32 additions & 112 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,139 +1,59 @@
# What is this repo about?
This is a telegram bot writen in python for mirroring files on the internet to our beloved Google Drive.
![Ayanami](https://telegra.ph/file/87b7878ee9d6273af566f.jpg)

# Inspiration
This project is heavily inspired from @out386 's telegram bot which is written in JS.
# Ayanami
A personal torrent leech bot for Dank-del

# Features supported:
- Mirroring direct download links to google drive
- Download progress
- Upload progress
- Download/upload speeds and ETAs
- Docker support
- Uploading To Team Drives.
- Index Link support
- Service account support
- Mirror all youtube-dl supported links
- Mirror telegram files
Fork of [python-aria-mirror-bot](https://github.com/lzzy12/python-aria-mirror-bot/)

# Upcoming features (TODOs):

# How to deploy?
Deploying is pretty much straight forward and is divided into several steps as follows:
## Installing requirements

- Clone this repo:
```
git clone https://github.com/lzzy12/python-aria-mirror-bot mirror-bot/
cd mirror-bot
```

- Install requirements
For Debian based distros
```
sudo apt install python3
```
Install Docker by following the [official docker docs](https://docs.docker.com/engine/install/debian/)


- For Arch and it's derivatives:
```
sudo pacman -S docker python
```

- Install dependencies for running setup scripts:
```shell script
pip3 install -r requirements-cli.txt
```

## Setting up config file
## Deploying on Heroku
- Run the script to generate token file(token.pickle) for Google Drive:
```
cp config_sample.env config.env
python3 generate_drive_token.py
```
- Remove the first line saying:
- Install [Heroku cli](https://devcenter.heroku.com/articles/heroku-cli)
- Login into your heroku account with command:
```
_____REMOVE_THIS_LINE_____=True
heroku login
```
Fill up rest of the fields. Meaning of each fields are discussed below:
- **BOT_TOKEN** : The telegram bot token that you get from @BotFather
- **GDRIVE_FOLDER_ID** : This is the folder ID of the Google Drive Folder to which you want to upload all the mirrors.
- **DOWNLOAD_DIR** : The path to the local folder where the downloads should be downloaded to
- **DOWNLOAD_STATUS_UPDATE_INTERVAL** : A short interval of time in seconds after which the Mirror progress message is updated. (I recommend to keep it 5 seconds at least)
- **OWNER_ID** : The Telegram user ID (not username) of the owner of the bot
- **AUTO_DELETE_MESSAGE_DURATION** : Interval of time (in seconds), after which the bot deletes it's message (and command message) which is expected to be viewed instantly. Note: Set to -1 to never automatically delete messages
- **IS_TEAM_DRIVE** : (Optional field) Set to "True" if GDRIVE_FOLDER_ID is from a Team Drive else False or Leave it empty.
- **USE_SERVICE_ACCOUNTS**: (Optional field) (Leave empty if unsure) Whether to use service accounts or not. For this to work see "Using service accounts" section below.
- **INDEX_URL** : (Optional field) Refer to https://github.com/maple3142/GDIndex/ The URL should not have any trailing '/'
- **TELEGRAM_API** : This is to authenticate to your telegram account for downloading Telegram files. You can get this from https://my.telegram.org DO NOT put this in quotes.
- **TELEGRAM_HASH** : This is to authenticate to your telegram account for downloading Telegram files. You can get this from https://my.telegram.org
- **USER_SESSION_STRING** : Session string generated by running:
- Create a new heroku app:
```
python3 generate_string_session.py
heroku create appname
```
Note: You can limit maximum concurrent downloads by changing the value of MAX_CONCURRENT_DOWNLOADS in aria.sh. By default, it's set to 2

## Getting Google OAuth API credential file

- Visit the [Google Cloud Console](https://console.developers.google.com/apis/credentials)
- Go to the OAuth Consent tab, fill it, and save.
- Go to the Credentials tab and click Create Credentials -> OAuth Client ID
- Choose Other and Create.
- Use the download button to download your credentials.
- Move that file to the root of mirror-bot, and rename it to credentials.json
- Visit [Google API page](https://console.developers.google.com/apis/library)
- Search for Drive and enable it if it is disabled
- Finally, run the script to generate token file (token.pickle) for Google Drive:
```
pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib
python3 generate_drive_token.py
- Select This App in your Heroku-cli:
```
## Deploying

- Start docker daemon (skip if already running):
heroku git:remote -a appname
```
sudo dockerd
- Change Dyno Stack to a Docker Container:
```
- Build Docker image:
heroku stack:set container
```
sudo docker build . -t mirror-bot
- Add Private Credentials and Config Stuff:
```
- Run the image:
git add -f credentials.json token.pickle config.env
```
sudo docker run mirror-bot
- Commit new changes:
```

# Using service accounts for uploading to avoid user rate limit
For Service Account to work, you must set USE_SERVICE_ACCOUNTS="True" in config file or environment variables
Many thanks to [AutoRClone](https://github.com/xyou365/AutoRclone) for the scripts
**NOTE:** Using service accounts is only recommended while uploading to a team drive.
## Generating service accounts
Step 1. Generate service accounts [What is service account](https://cloud.google.com/iam/docs/service-accounts)
---------------------------------
Let us create only the service accounts that we need.
**Warning:** abuse of this feature is not the aim of this project and we do **NOT** recommend that you make a lot of projects, just one project and 100 sa allow you plenty of use, its also possible that over abuse might get your projects banned by google.

git commit -m "Added Creds."
```
Note: 1 service account can copy around 750gb a day, 1 project can make 100 service accounts so that's 75tb a day, for most users this should easily suffice.
- Push Code to Heroku:
```

`python3 gen_sa_accounts.py --quick-setup 1 --new-only`

A folder named accounts will be created which will contain keys for the service accounts

NOTE: If you have created SAs in past from this script, you can also just re download the keys by running:
git push heroku master --force
```
python3 gen_sa_accounts.py --download-keys project_id
- Restart Worker by these commands:
```

### Add all the service accounts to the Team Drive
- Run:
heroku ps:scale worker=0
```
python3 add_to_team_drive.py -d SharedTeamDriveSrcID
```

# Youtube-dl authentication using .netrc file
For using your premium accounts in youtube-dl, edit the netrc file (in the root directory of this repository) according to following format:
heroku ps:scale worker=1
```
machine host login username password my_youtube_password
Heroku-Note: Doing authorizations ( /authorize command ) through telegram wont be permanent as heroku uses ephemeral filesystem. They will be reset on each dyno boot. As a workaround you can:
- Make a file authorized_chats.txt and write the user names and chat_id of you want to authorize, each separated by new line
- Then force add authorized_chats.txt to git and push it to heroku
```
where host is the name of extractor (eg. youtube, twitch). Multiple accounts of different hosts can be added each separated by a new line
git add authorized_chats.txt -f
git commit -asm "Added hardcoded authorized_chats.txt"
git push heroku heroku:master
```

0 comments on commit 2f0ac03

Please sign in to comment.