Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 readme #13

Merged
merged 1 commit into from
Jan 23, 2024
Merged

🐛 readme #13

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 38 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,43 @@ Some skilled engineers even have a blog site where they push some gold content,

This program aims to create the following features:

- Given a list of websites, that are located in a MongoDB collection, scrape the content of each website and save it in another MongoDB collection.
- Given a list of websites, that are located in a MongoDB collection, scrape the content of each website and save it in another MongoDB collection. ✅
- After the scraping, calculate the similarity between the new content and the previous content of each website, and update the MongoDB collection
with this information.
- All the registered users will receive an email according to the URL that they have registered notifying them about news in their favorite engineers websites.
with this information. ✅
- All the registered users will receive an email according to the URL that they have registered notifying them about news in their favorite engineers websites. ✅

Obs: All these flows will be trigerred by a cron job. ✅


## Environement Variables

The following environment variables are required to run the program:

- `LOG_LEVEL`: The level of the logs that will be printed. The values could be `DEBUG`, `INFO`, `WARNING` or `ERROR`.
- `LOG_TYPE`: The format of the logs that will be printed. The values could be `json` or `text`.
- `NL_MONGO_URI`: The URI of the MongoDB database that will be used to store the data.
- `NL_EMAIL_PASSWORD`: The password of the email that will be used to send the emails.
- `NL_EMAIL_USERNAME`: The user of the email that will be used to send the emails.

## Commands

`make help` - Show the available commands of this project. Using it, it's enoght to play around the project.


## Integration & Unit Tests

To run the integration tests, you need to have a MongoDB instance running in your machine. To do it, you can run the following command:

```bash
make dev/start
```

Access the dev container and run the tests:

```bash
make dev
make test
make integration-test
```


Obs: All these flows will be trigerred by a cron job.