Skip to content

Commit

Permalink
Merge branch 'main' into New-prompt-
Browse files Browse the repository at this point in the history
  • Loading branch information
@dartpain
dartpain authored Oct 22, 2023
2 parents d7b28a3 + e3e8e67 commit 9303746
Show file tree
Hide file tree
Showing 41 changed files with 985 additions and 428 deletions.
Binary file added Assets/DocsGPT tee-back.jpeg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Assets/DocsGPT tee-front.jpeg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
99 changes: 79 additions & 20 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
@@ -1,52 +1,111 @@
# Welcome to DocsGPT Contributing Guidelines

Thank you for choosing this project to contribute to. We are all very grateful!
Thank you for choosing to contribute to DocsGPT! We are all very grateful!

### [🎉 Join the Hacktoberfest with DocsGPT and Earn a Free T-shirt! 🎉](https://github.com/arc53/DocsGPT/blob/main/HACKTOBERFEST.md)

# We accept different types of contributions

📣 **Discussions** - where you can start a new topic or answer some questions
📣 **Discussions** - Engage in conversations, start new topics, or help answer questions.

🐞 **Issues** - This is how we track tasks, sometimes it is bugs that need fixing, and sometimes it is new features
🐞 **Issues** - This is where we keep track of tasks. It could be bugs,fixes or suggestions for new features.

🛠️ **Pull requests** - This is how you can suggest changes to our repository, to work on existing issues or add new features
🛠️ **Pull requests** - Suggest changes to our repository, either by working on existing issues or adding new features.

📚 **Wiki** - where we have our documentation
📚 **Wiki** - This is where our documentation resides.


## 🐞 Issues and Pull requests

We value contributions to our issues in the form of discussion or suggestions. We recommend that you check out existing issues and our [roadmap](https://github.com/orgs/arc53/projects/2).
We value contributions in the form of discussions or suggestions. We recommend taking a look at existing issues and our [roadmap](https://github.com/orgs/arc53/projects/2).

If you want to contribute by writing code, there are a few things that you should know before doing it:
Before creating issues, please check out how the latest version of our app looks and works by launching it via [Quickstart](https://github.com/arc53/DocsGPT#quickstart) the version on our live demo is slightly modified with login. Your issues should relate to the version that you can launch via [Quickstart](https://github.com/arc53/DocsGPT#quickstart).

We have a frontend in React (Vite) and backend in Python.
### 👨‍💻 If you're interested in contributing code, here are some important things to know:

### If you are looking to contribute to frontend (⚛️React, Vite):
Tech Stack Overview:

- The current frontend is being migrated from `/application` to `/frontend` with a new design, so please contribute to the new one.
- 🌐 Frontend: Built with React (Vite) ⚛️,

- 🖥 Backend: Developed in Python 🐍

### 🌐 If you are looking to contribute to frontend (⚛️React, Vite):

- The current frontend is being migrated from [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) to [`/frontend`](https://github.com/arc53/DocsGPT/tree/main/frontend) with a new design, so please contribute to the new one.
- Check out this [milestone](https://github.com/arc53/DocsGPT/milestone/1) and its issues.
- The Figma design can be found [here](https://www.figma.com/file/OXLtrl1EAy885to6S69554/DocsGPT?node-id=0%3A1&t=hjWVuxRg9yi5YkJ9-1).
- The updated Figma design can be found [here](https://www.figma.com/file/OXLtrl1EAy885to6S69554/DocsGPT?node-id=0%3A1&t=hjWVuxRg9yi5YkJ9-1).

Please try to follow the guidelines.

### If you are looking to contribute to Backend (🐍 Python):
- Check out our issues and contribute to `/application` or `/scripts` (ignore old `ingest_rst.py` `ingest_rst_sphinx.py` files; they will be deprecated soon).
- All new code should be covered with unit tests ([pytest](https://github.com/pytest-dev/pytest)). Please find tests under [`/tests`](https://github.com/arc53/DocsGPT/tree/main/tests) folder.
- Before submitting your PR, ensure it is queryable after ingesting some test data.
### 🖥 If you are looking to contribute to Backend (🐍 Python):

- Review our issues and contribute to [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) or [`/scripts`](https://github.com/arc53/DocsGPT/tree/main/scripts) (please disregard old [`ingest_rst.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst.py) [`ingest_rst_sphinx.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst_sphinx.py) files; they will be deprecated soon).
- All new code should be covered with unit tests ([pytest](https://github.com/pytest-dev/pytest)). Please find tests under [`/tests`](https://github.com/arc53/DocsGPT/tree/main/tests) folder.
- Before submitting your Pull Request, ensure it can be queried after ingesting some test data.

### Testing

To run unit tests from the root of the repository, execute:
```
python -m pytest
```

### Workflow:
Create a fork, make changes on your forked repository, and submit changes as a pull request.
## Workflow 📈

## Questions/collaboration
Please join our [Discord](https://discord.gg/n5BX8dh8rU). Don't hesitate; we are very friendly and welcoming to new contributors.
Here's a step-by-step guide on how to contribute to DocsGPT:

1. **Fork the Repository:**
- Click the "Fork" button at the top-right of this repository to create your fork.

2. **Create and Switch to a New Branch:**
- Create a new branch for your contribution using:
```shell
git checkout -b your-branch-name
```

3. **Make Changes:**
- Make the required changes in your branch.

4. **Add Changes to the Staging Area:**
- Add your changes to the staging area using:
```shell
git add .
```

# Thank you so much for considering contributing to DocsGPT!🙏
5. **Commit Your Changes:**
- Commit your changes with a descriptive commit message using:
```shell
git commit -m "Your descriptive commit message"
```

6. **Push Your Changes to the Remote Repository:**
- Push your branch with changes to your fork on GitHub using:
```shell
git push origin your-branch-name
```

7. **Submit a Pull Request (PR):**
- Create a Pull Request from your branch to the main repository. Make sure to include a detailed description of your changes and reference any related issues.

8. **Collaborate:**
- Be responsive to comments and feedback on your PR.
- Make necessary updates as suggested.
- Once your PR is approved, it will be merged into the main repository.

9. **Testing:**
- Before submitting a Pull Request, ensure your code passes all unit tests.
- To run unit tests from the root of the repository, execute:
```shell
python -m pytest
```

*Note: You should run the unit test only after making the changes to the backend code.*

10. **Questions and Collaboration:**
- Feel free to join our Discord. We're very friendly and welcoming to new contributors, so don't hesitate to reach out.

Thank you for considering contributing to DocsGPT! 🙏

## Questions/collaboration
Feel free to join our [Discord](https://discord.gg/n5BX8dh8rU). We're very friendly and welcoming to new contributors, so don't hesitate to reach out.
# Thank you so much for considering to contribute DocsGPT!🙏
14 changes: 10 additions & 4 deletions HACKTOBERFEST.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,25 @@ Familiarize yourself with the current contributions and our [Roadmap](https://gi
Deciding to contribute with code? Here are some insights based on the area of your interest:

- Frontend (⚛️React, Vite):
- Most of the code is located in `/frontend` folder. You can also check out our React extension in /extensions/react-widget.
- Most of the code is located in [`/frontend`](https://github.com/arc53/DocsGPT/tree/main/frontend) folder. You can also check out our React extension in [`/extensions/react-widget`](https://github.com/arc53/DocsGPT/tree/main/extensions/react-widget).
- For design references, here's the [Figma](https://www.figma.com/file/OXLtrl1EAy885to6S69554/DocsGPT?node-id=0%3A1&t=hjWVuxRg9yi5YkJ9-1).
- Ensure you adhere to the established guidelines.

- Backend (🐍Python):
- Focus on `/application` or `/scripts`. However, avoid the files ingest_rst.py and ingest_rst_sphinx.py, as they will soon be deprecated.
- Focus on [`/application`](https://github.com/arc53/DocsGPT/tree/main/application) or [`/scripts`](https://github.com/arc53/DocsGPT/tree/main/scripts). However, avoid the files [`ingest_rst.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst.py) and [`ingest_rst_sphinx.py`](https://github.com/arc53/DocsGPT/blob/main/scripts/old/ingest_rst_sphinx.py), as they will soon be deprecated.
- Newly added code should come with relevant unit tests (pytest).
- Refer to the `/tests` folder for test suites.
- Refer to the [`/tests`](https://github.com/arc53/DocsGPT/tree/main/tests) folder for test suites.

Check out our [Contributing Guidelines](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md)

Once you have created your PR and our maintainers have merged it, please fill in this [form](https://airtable.com/appfkqFVjB0RpYCJh/shrXXM98xgRsbjO7s).

Feel free to join our Discord server. We're here to help newcomers, so don't hesitate to jump in! [Join us here](https://discord.gg/n5BX8dh8rU).

Thank you very much for considering contributing to DocsGPT during Hacktoberfest! 🙏 Your contributions could earn you a stylish new t-shirt as a token of our appreciation. 🎁 Join us, and let's code together! 🚀
Thank you very much for considering contributing to DocsGPT during Hacktoberfest! 🙏 Your contributions (not just simple typo) could earn you a stylish new t-shirt as a token of our appreciation. 🎁 Join us, and let's code together! 🚀

Here is a preview of the shirts:
<p align="center">
<img src="Assets/DocsGPT tee-front.jpeg" width="350" />
<img src="Assets/DocsGPT tee-back.jpeg" width="350" />
</p>
92 changes: 54 additions & 38 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
</p>

<p align="left">
<strong>DocsGPT</strong> is a cutting-edge open-source solution that streamlines the process of finding information in project documentation. With its integration of the powerful <strong>GPT</strong> models, developers can easily ask questions about a project and receive accurate answers.
<strong><a href="https://docsgpt.arc53.com/">DocsGPT</a></strong> is a cutting-edge open-source solution that streamlines the process of finding information in the project documentation. With its integration of the powerful <strong>GPT</strong> models, developers can easily ask questions about a project and receive accurate answers.

Say goodbye to time-consuming manual searches, and let <strong>DocsGPT</strong> help you quickly find the information you need. Try it out and see how it revolutionizes your project documentation experience. Contribute to its development and be a part of the future of AI-powered assistance.
Say goodbye to time-consuming manual searches, and let <strong><a href="https://docsgpt.arc53.com/">DocsGPT</a></strong> help you quickly find the information you need. Try it out and see how it revolutionizes your project documentation experience. Contribute to its development and be a part of the future of AI-powered assistance.
</p>

<div align="center">
Expand All @@ -21,58 +21,56 @@ Say goodbye to time-consuming manual searches, and let <strong>DocsGPT</strong>

</div>

### Production Support / Help for companies:
### Production Support / Help for companies:

We're eager to provide personalized assistance when deploying your DocsGPT to a live environment.
- [Get Support 👋](https://airtable.com/appdeaL0F1qV8Bl2C/shrrJF1Ll7btCJRbP)

- [Book Demo 👋](https://airtable.com/appdeaL0F1qV8Bl2C/shrrJF1Ll7btCJRbP)
- [Send Email ✉️](mailto:[email protected]?subject=DocsGPT%20support%2Fsolutions)

### [🎉 Join the Hacktoberfest with DocsGPT and Earn a Free T-shirt! 🎉](https://github.com/arc53/DocsGPT/blob/main/HACKTOBERFEST.md)

![video-example-of-docs-gpt](https://d3dg1063dc54p9.cloudfront.net/videos/demov3.gif)


## Roadmap

You can find our roadmap [here](https://github.com/orgs/arc53/projects/2). Please don't hesitate to contribute or create issues, it helps us improve DocsGPT!

## Our Open-Source models optimized for DocsGPT:

| Name | Base Model | Requirements (or similar) |
|-------------------|------------|----------------------------------------------------------|
| [Docsgpt-7b-falcon](https://huggingface.co/Arc53/docsgpt-7b-falcon) | Falcon-7b | 1xA10G gpu |
| [Docsgpt-14b](https://huggingface.co/Arc53/docsgpt-14b) | llama-2-14b | 2xA10 gpu's |
| [Docsgpt-40b-falcon](https://huggingface.co/Arc53/docsgpt-40b-falcon) | falcon-40b | 8xA10G gpu's |

| Name | Base Model | Requirements (or similar) |
| --------------------------------------------------------------------- | ----------- | ------------------------- |
| [Docsgpt-7b-falcon](https://huggingface.co/Arc53/docsgpt-7b-falcon) | Falcon-7b | 1xA10G gpu |
| [Docsgpt-14b](https://huggingface.co/Arc53/docsgpt-14b) | llama-2-14b | 2xA10 gpu's |
| [Docsgpt-40b-falcon](https://huggingface.co/Arc53/docsgpt-40b-falcon) | falcon-40b | 8xA10G gpu's |

If you don't have enough resources to run it, you can use bitsnbytes to quantize.


## Features

![Group 9](https://user-images.githubusercontent.com/17906039/220427472-2644cff4-7666-46a5-819f-fc4a521f63c7.png)


## Useful links
[Live preview](https://docsgpt.arc53.com/)

[Join our Discord](https://discord.gg/n5BX8dh8rU)

[Guides](https://docs.docsgpt.co.uk/)

[Interested in contributing?](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md)
- 🔍🔥 [Live preview](https://docsgpt.arc53.com/)

- 💬🎉[Join our Discord](https://discord.gg/n5BX8dh8rU)

- 📚😎 [Guides](https://docs.docsgpt.co.uk/)

[How to use any other documentation](https://docs.docsgpt.co.uk/Guides/How-to-train-on-other-documentation)
- 👩‍💻👨‍💻 [Interested in contributing?](https://github.com/arc53/DocsGPT/blob/main/CONTRIBUTING.md)

[How to host it locally (so all data will stay on-premises)](https://docs.docsgpt.co.uk/Guides/How-to-use-different-LLM)
- 🗂️🚀 [How to use any other documentation](https://docs.docsgpt.co.uk/Guides/How-to-train-on-other-documentation)

- 🏠🔐 [How to host it locally (so all data will stay on-premises)](https://docs.docsgpt.co.uk/Guides/How-to-use-different-LLM)

## Project structure

- Application - Flask app (main application).

- Extensions - Chrome extension.

- Scripts - Script that creates similarity search index and stores for other libraries.
- Scripts - Script that creates similarity search index for other libraries.

- Frontend - Frontend uses Vite and React.

Expand All @@ -89,26 +87,30 @@ It will install all the dependencies and allow you to download the local model o
Otherwise, refer to this Guide:

1. Download and open this repository with `git clone https://github.com/arc53/DocsGPT.git`
2. Create a `.env` file in your root directory and set the env variable `OPENAI_API_KEY` with your OpenAI API key and `VITE_API_STREAMING` to true or false, depending on if you want streaming answers or not.
2. Create a `.env` file in your root directory and set the env variable `OPENAI_API_KEY` with your [OpenAI API key](https://platform.openai.com/account/api-keys) and `VITE_API_STREAMING` to true or false, depending on whether you want streaming answers or not.
It should look like this inside:

```
API_KEY=Yourkey
VITE_API_STREAMING=true
```
See optional environment variables in the `/.env-template` and `/application/.env_sample` files.
3. Run `./run-with-docker-compose.sh`.

See optional environment variables in the [/.env-template](https://github.com/arc53/DocsGPT/blob/main/.env-template) and [/application/.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) files.

3. Run [./run-with-docker-compose.sh](https://github.com/arc53/DocsGPT/blob/main/run-with-docker-compose.sh).
4. Navigate to http://localhost:5173/.

To stop, just run `Ctrl + C`.

## Development environments

### Spin up mongo and redis
For development, only two containers are used from `docker-compose.yaml` (by deleting all services except for Redis and Mongo).

For development, only two containers are used from [docker-compose.yaml](https://github.com/arc53/DocsGPT/blob/main/docker-compose.yaml) (by deleting all services except for Redis and Mongo).
See file [docker-compose-dev.yaml](./docker-compose-dev.yaml).

Run

```
docker compose -f docker-compose-dev.yaml build
docker compose -f docker-compose-dev.yaml up -d
Expand All @@ -119,44 +121,57 @@ docker compose -f docker-compose-dev.yaml up -d
Make sure you have Python 3.10 or 3.11 installed.

1. Export required environment variables or prepare a `.env` file in the `/application` folder:
- Copy `.env_sample` and create `.env` with your OpenAI API token for the `API_KEY` and `EMBEDDINGS_KEY` fields.
- Copy [.env_sample](https://github.com/arc53/DocsGPT/blob/main/application/.env_sample) and create `.env` with your OpenAI API token for the `API_KEY` and `EMBEDDINGS_KEY` fields.

(check out [`application/core/settings.py`](application/core/settings.py) if you want to see more config options.)

2. (optional) Create a Python virtual environment:
You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments .
You can follow the [Python official documentation](https://docs.python.org/3/tutorial/venv.html) for virtual environments.

a) On Mac OS and Linux

```commandline
python -m venv venv
. venv/bin/activate
```

b) On Windows

```commandline
python -m venv venv
venv/Scripts/activate
```

3. Change to the `application/` subdir and install dependencies for the backend:
3. Change to the `application/` subdir by the command `cd application/` and install dependencies for the backend:

```commandline
pip install -r application/requirements.txt
pip install -r requirements.txt
```

4. Run the app using `flask run --host=0.0.0.0 --port=7091`.
5. Start worker with `celery -A application.app.celery worker -l INFO`.

### Start frontend
### Start frontend

Make sure you have Node version 16 or higher.

1. Navigate to the `/frontend` folder.
2. Install dependencies by running `npm install`.
3. Run the app using `npm run dev`.
1. Navigate to the [/frontend](https://github.com/arc53/DocsGPT/tree/main/frontend) folder.
2. Install required packages `husky` and `vite` (ignore if installed).

```commandline
npm install husky -g
npm install vite -g
```

3. Install dependencies by running `npm install --include=dev`.
4. Run the app using `npm run dev`.

## Contributing
Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for information about how to get involved. We welcome issues, questions, and pull requests.

Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for information about how to get involved. We welcome issues, questions, and pull requests.

## Code Of Conduct

We as members, contributors, and leaders, pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. Please refer to the [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) file for more information about contributing.

## Many Thanks To Our Contributors
Expand All @@ -166,6 +181,7 @@ We as members, contributors, and leaders, pledge to make participation in our co
</a>

## License
The source code license is MIT, as described in the LICENSE file.

The source code license is [MIT](https://opensource.org/license/mit/), as described in the [LICENSE](LICENSE) file.

Built with [🦜️🔗 LangChain](https://github.com/hwchase17/langchain)
Loading

0 comments on commit 9303746

Please sign in to comment.