Skip to content

Commit

Permalink
Merge branch 'main' into fix/precommit-tests
Browse files Browse the repository at this point in the history
  • Loading branch information
tyfiero authored Apr 8, 2024
2 parents 130c0ac + a40c804 commit 0eca547
Show file tree
Hide file tree
Showing 65 changed files with 14,772 additions and 1,104 deletions.
7 changes: 2 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,8 @@ We want to help you build. [Apply for 1-on-1 support.](https://0ggfznkwh4j.typef

<br>

---

⚠️ **WARNING:** This experimental project is under rapid development and lacks basic safeguards. Until a stable `1.0` release, **ONLY** run this repository on devices without sensitive information or access to paid services. ⚠️

---
> [!IMPORTANT]
> This experimental project is under rapid development and lacks basic safeguards. Until a stable `1.0` release, only run this repository on devices without sensitive information or access to paid services.
<br>

Expand Down
5 changes: 2 additions & 3 deletions ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
# Roadmap

Our goal is to power a billion devices with the 01OS over the next 10 years. The Cambrian explosion of AI devices.
Our goal is to power a billion devices with the 01OS over the next 10 years. The Cambrian explosion of AI devices.

We can do that with your help. Help extend the 01OS to run on new hardware, to connect with new peripherals like GPS and cameras, and add new locally running language models to unlock use-cases for this technology that no-one has even imagined yet.
We can do that with your help. Help extend the 01OS to run on new hardware, to connect with new peripherals like GPS and cameras, and add new locally running language models to unlock use-cases for this technology that no-one has even imagined yet.

In the coming months, we're going to release:

- [ ] Add support for Azure and PlayHT for fast latency
- [ ] An open-source language model for computer control
- [ ] A react-native app for your phone
- [ ] A hand-held device that runs fully offline.

4 changes: 2 additions & 2 deletions TASKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
- [ ] Sends to describe API
- [ ] prints and returns description
- [ ] Llamafile for phi-2 + moondream
- [ ] test on rPi + Jetson (+android mini phone?)
- [ ] test on rPi + Jetson (+android mini phone?)

**OS**

Expand Down Expand Up @@ -66,7 +66,7 @@

**Hardware**

- [ ] (Hardware and software) Get the 01OS working on the **Jetson** or Pi. Pick one to move forward with.
- [ ] (Hardware and software) Get the 01OS working on the **Jetson** or Pi. Pick one to move forward with.
- [ ] Connect the Seeed Sense (ESP32 with Wifi, Bluetooth and a mic) to a small DAC + amplifier + speaker.
- [ ] Connect the Seeed Sense to a battery.
- [ ] Configure the ESP32 to be a wireless mic + speaker for the Jetson or Pi.
Expand Down
15 changes: 8 additions & 7 deletions docs/README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,8 @@

<br>

---

⚠️ **警告:** 这个实验性项目正在快速开发中,并且缺乏基本的安全保障。在稳定的 1.0 版本发布之前, **仅在**没有敏感信息或访问付费服务的设备上运行此存储库。⚠️

---
> [!IMPORTANT]
> 这个实验性项目正在快速开发中,并且缺乏基本的安全保障。在稳定的 `1.0` 版本发布之前, 仅在没有敏感信息或访问付费服务的设备上运行此存储库。
<br>

Expand All @@ -39,7 +36,7 @@ git clone https://github.com/OpenInterpreter/01 # Clone the repository
cd 01/software # CD into the source directory
```

<!-- > Not working? Read our [setup guide](https://docs.openinterpreter.com/getting-started/setup). -->
<!-- > 不起作用?阅读我们的[安装指南](https://docs.openinterpreter.com/getting-started/setup) -->

```shell
brew install portaudio ffmpeg cmake # Install Mac OSX dependencies
Expand All @@ -48,6 +45,8 @@ export OPENAI_API_KEY=sk... # OR run `poetry run 01 --local` to run everything l
poetry run 01 # Runs the 01 Light simulator (hold your spacebar, speak, release)
```

<!-- > 对于Windows安装,请阅读我们的[专用安装指南](https://docs.openinterpreter.com/getting-started/setup#windows)。 -->

<br>

# 硬件
Expand Down Expand Up @@ -78,7 +77,9 @@ poetry run 01 # Runs the 01 Light simulator (hold your spacebar, speak, release)

## LMC 消息

为了与系统的不同组件进行通信,我们引入了 [LMC 消息](https://docs.openinterpreter.com/protocols/lmc-messages) 格式,它扩展了 OpenAI 的消息格式以包含一个 "computer" 角色。
为了与系统的不同组件进行通信,我们引入了 [LMC 消息](https://docs.openinterpreter.com/protocols/lmc-messages) 格式,它扩展了 OpenAI 的消息格式以包含一个 "computer" 角色:

https://github.com/OpenInterpreter/01/assets/63927363/8621b075-e052-46ba-8d2e-d64b9f2a5da9

## 动态系统消息

Expand Down
40 changes: 21 additions & 19 deletions docs/README_FR.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ git clone https://github.com/OpenInterpreter/01 # Clone le dépôt
cd 01/software # CD dans le répertoire source
```

<!-- > Cela ne fonctionne pas ? Lis notre [guide d'installation](https://docs.openinterpreter.com/getting-started/setup). -->
<!-- > Cela ne fonctionne pas ? Lisez notre [guide d'installation](https://docs.openinterpreter.com/getting-started/setup). -->

```shell
brew install portaudio ffmpeg cmake # Installe les dépendances Mac OSX
Expand All @@ -48,56 +48,58 @@ export OPENAI_API_KEY=sk... # OU exécute `poetry run 01 --local` pour tout exé
poetry run 01 # Exécute le simulateur 01 Light (maintenez votre barre d'espace, parlez, relâchez)
```

<!-- > Pour une installation sous Windows, lisez [le guide dédié](https://docs.openinterpreter.com/getting-started/setup#windows). -->

<br>

# Hardware

- Le **01 Light** est une interface vocale basée sur ESP32. Les instructions de construction sont [ici]. (https://github.com/OpenInterpreter/01/tree/main/hardware/light). Une liste de ce qu'il faut acheter [ici](https://github.com/OpenInterpreter/01/blob/main/hardware/light/BOM.md).
- Il fonctionne en tandem avec le **01 Server** ([guide d'installation ci-dessous](https://github.com/OpenInterpreter/01/blob/main/README.md#01-server)) fonctionnant sur votre ordinateur domestique.
- **Mac OSX** et **Ubuntu** sont pris en charge en exécutant `poetry run 01` (**Windows** Windows est pris en charge de manière expérimentale). Cela utilise votre barre d'espace pour simuler le 01 Light..
- Le **01 Light** est une interface vocale basée sur ESP32. Les instructions de construction sont [ici]. (https://github.com/OpenInterpreter/01/tree/main/hardware/light). Une liste de ce qu'il faut acheter se trouve [ici](https://github.com/OpenInterpreter/01/blob/main/hardware/light/BOM.md).
- Il fonctionne en tandem avec le **Server 01** ([guide d'installation ci-dessous](https://github.com/OpenInterpreter/01/blob/main/README.md#01-server)) fonctionnant sur votre ordinateur.
- **Mac OSX** et **Ubuntu** sont pris en charge en exécutant `poetry run 01` (**Windows** est pris en charge de manière expérimentale). Cela utilise votre barre d'espace pour simuler le 01 Light.
- (prochainement) Le **01 Heavy** est un dispositif autonome qui exécute tout localement.

**Nous avons besoin de votre aide pour soutenir et construire plus de hardware.** Le 01 devrait pouvoir fonctionner sur tout dispositif avec entrée (microphone, clavier, etc.), sortie (haut-parleurs, écrans, moteurs, etc.) et une connexion internet (ou suffisamment de puissance de calcul pour tout exécuter localement). [Contribution Guide →](https://github.com/OpenInterpreter/01/blob/main/CONTRIBUTING.md)
**Nous avons besoin de votre aide pour soutenir et construire plus de hardware.** Le 01 devrait pouvoir fonctionner sur tout dispositif avec entrée (microphone, clavier, etc.), sortie (haut-parleurs, écrans, moteurs, etc.) et connexion internet (ou suffisamment de puissance de calcul pour tout exécuter localement). [Guide de Contribution](https://github.com/OpenInterpreter/01/blob/main/CONTRIBUTING.md)

<br>

# Comment ça marche ?

Le 01 expose un websocket de speech-to-speech à l'adresse localhost:10001.
Le 01 expose un websocket de *speech-to-speech* à l'adresse `localhost:10001`.

Si vous diffusez des octets audio bruts vers `/` au format [Streaming LMC](https://docs.openinterpreter.com/guides/streaming-response), vous recevrez sa réponse dans le même format.
Si vous diffusez des octets audio bruts vers `/` au [format de streaming LMC](https://docs.openinterpreter.com/guides/streaming-response), vous recevrez sa réponse dans le même format.

Inspiré en partie par [Andrej Karpathy's LLM OS](https://twitter.com/karpathy/status/1723140519554105733), nous utilisons un [un modèle de langage inteprétant du code](https://github.com/OpenInterpreter/open-interpreter), et le sollicitons lorsque certains événements se produisent dans le [noyau de votre ordinateur](https://github.com/OpenInterpreter/01/blob/main/software/source/server/utils/kernel.py).
Inspiré en partie par [l'idée d'un OS LLM d'Andrej Karpathy](https://twitter.com/karpathy/status/1723140519554105733), nous utilisons un [un modèle de langage inteprétant du code](https://github.com/OpenInterpreter/open-interpreter), et le sollicitons lorsque certains événements se produisent dans le [noyau de votre ordinateur](https://github.com/OpenInterpreter/01/blob/main/software/source/server/utils/kernel.py).

Le 01 l'encapsule dans une interface vocale :

<br>

<img width="100%" alt="LMC" src="https://github.com/OpenInterpreter/01/assets/63927363/52417006-a2ca-4379-b309-ffee3509f5d4"><br><br>

# Protocols
# Protocoles

## LMC Messages
## Messages LMC

To communicate with different components of this system, we introduce [LMC Messages](https://docs.openinterpreter.com/protocols/lmc-messages) format, which extends OpenAI’s messages format to include a "computer" role:
Pour communiquer avec les différents composants du système, nous introduisons le [format de messages LMC](https://docs.openinterpreter.com/protocols/lmc-messages), une extension du format de message d'OpenAI qui inclut un nouveau rôle "*computer*":

https://github.com/OpenInterpreter/01/assets/63927363/8621b075-e052-46ba-8d2e-d64b9f2a5da9

## Dynamic System Messages
## Messages Systèmes Dynamiques (Dynamic System Messages)

Les Dynamic System Messages vous permettent d'exécuter du code à l'intérieur du message système du LLM, juste avant qu'il n'apparaisse à l'IA.
Les Messages Systèmes Dynamiques vous permettent d'exécuter du code à l'intérieur du message système du LLM, juste avant qu'il n'apparaisse à l'IA.

```python
# Modifiez les paramètres suivants dans i.py
interpreter.system_message = r" The time is {{time.time()}}. " # Tout ce qui est entre doubles crochets sera exécuté comme du Python
interpreter.chat("What time is it?") # Il le saura, sans faire appel à un outil/API
interpreter.chat("What time is it?") # L'interpréteur connaitre la réponse, sans faire appel à un outil ou une API
```

# Guides

## 01 Server

Pour exécuter le serveur sur votre ordinateur de bureau et le connecter à votre 01 Light, exécutez les commandes suivantes :
Pour exécuter le serveur sur votre ordinateur et le connecter à votre 01 Light, exécutez les commandes suivantes :

```shell
brew install ngrok/ngrok/ngrok
Expand All @@ -107,7 +109,7 @@ poetry run 01 --server --expose

La dernière commande affichera une URL de serveur. Vous pouvez saisir ceci dans le portail WiFi captif de votre 01 Light pour le connecter à votre serveur 01.

## Local Mode
## Mode Local

```
poetry run 01 --local
Expand All @@ -117,9 +119,9 @@ Si vous souhaitez exécuter localement du speech-to-text en utilisant Whisper, v

## Personnalisation

Pour personnaliser le comportement du système, modifie le [system message, model, skills library path,](https://docs.openinterpreter.com/settings/all-settings) etc. in `i.py`. Ce fichier configure un interprète et est alimenté par Open Interpreter.
Pour personnaliser le comportement du système, modifie [`system message`, `model`, `skills library path`,](https://docs.openinterpreter.com/settings/all-settings) etc. in `i.py`. Ce fichier configure un interprète alimenté par Open Interpreter.

## Ubuntu Dependencies
## Dépendances Ubuntu

```bash
sudo apt-get install portaudio19-dev ffmpeg cmake
Expand All @@ -135,7 +137,7 @@ Veuillez consulter nos [directives de contribution](CONTRIBUTING.md) pour plus d

# Roadmap

Visite [notre roadmap](/ROADMAP.md) pour voir le futur du 01.
Visitez [notre roadmap](/ROADMAP.md) pour connaitre le futur du 01.

<br>

Expand Down
6 changes: 3 additions & 3 deletions docs/client/setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ poetry run 01 --client

### Flags

- `--client`
- `--client`
Run client.

- `--client-type TEXT`
Specify the client type.
- `--client-type TEXT`
Specify the client type.
Default: `auto`.
6 changes: 5 additions & 1 deletion docs/getting-started/setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,16 @@ On Windows you will need to install the following:
- [Git for Windows](https://git-scm.com/download/win).
- [virtualenv](https://virtualenv.pypa.io/en/latest/installation.html) or [MiniConda](https://docs.anaconda.com/free/miniconda/miniconda-install/) to manage virtual environments.
- [Chocolatey](https://chocolatey.org/install#individual) to install the required packages.
- [Microsoft C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools):
- Choose [**Download Build Tools**](https://visualstudio.microsoft.com/visual-cpp-build-tools/).
- Run the downloaded file **vs_BuildTools.exe**.
- In the installer, select **Workloads** > **Desktop & Mobile** > **Desktop Development with C++**.

With these installed, you can run the following commands in a **PowerShell terminal as an administrator**:

```powershell
# Install the required packages
choco install -y ffmpeg cmake
choco install -y ffmpeg
```

## Install 01
Expand Down
66 changes: 33 additions & 33 deletions docs/server/setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,73 +44,73 @@ For more information, please read about <a href="/services/speech-to-text">speec

## CLI Flags

- `--server`
- `--server`
Run server.

- `--server-host TEXT`
Specify the server host where the server will deploy.
- `--server-host TEXT`
Specify the server host where the server will deploy.
Default: `0.0.0.0`.

- `--server-port INTEGER`
Specify the server port where the server will deploy.
- `--server-port INTEGER`
Specify the server port where the server will deploy.
Default: `10001`.

- `--tunnel-service TEXT`
Specify the tunnel service.
- `--tunnel-service TEXT`
Specify the tunnel service.
Default: `ngrok`.

- `--expose`
- `--expose`
Expose server to internet.

- `--server-url TEXT`
Specify the server URL that the client should expect.
Defaults to server-host and server-port.
- `--server-url TEXT`
Specify the server URL that the client should expect.
Defaults to server-host and server-port.
Default: `None`.

- `--llm-service TEXT`
Specify the LLM service.
- `--llm-service TEXT`
Specify the LLM service.
Default: `litellm`.

- `--model TEXT`
Specify the model.
- `--model TEXT`
Specify the model.
Default: `gpt-4`.

- `--llm-supports-vision`
- `--llm-supports-vision`
Specify if the LLM service supports vision.

- `--llm-supports-functions`
- `--llm-supports-functions`
Specify if the LLM service supports functions.

- `--context-window INTEGER`
Specify the context window size.
- `--context-window INTEGER`
Specify the context window size.
Default: `2048`.

- `--max-tokens INTEGER`
Specify the maximum number of tokens.
- `--max-tokens INTEGER`
Specify the maximum number of tokens.
Default: `4096`.

- `--temperature FLOAT`
Specify the temperature for generation.
- `--temperature FLOAT`
Specify the temperature for generation.
Default: `0.8`.

- `--tts-service TEXT`
Specify the TTS service.
- `--tts-service TEXT`
Specify the TTS service.
Default: `openai`.

- `--stt-service TEXT`
Specify the STT service.
- `--stt-service TEXT`
Specify the STT service.
Default: `openai`.

- `--local`
- `--local`
Use recommended local services for LLM, STT, and TTS.

- `--install-completion [bash|zsh|fish|powershell|pwsh]`
Install completion for the specified shell.
- `--install-completion [bash|zsh|fish|powershell|pwsh]`
Install completion for the specified shell.
Default: `None`.

- `--show-completion [bash|zsh|fish|powershell|pwsh]`
Show completion for the specified shell, to copy it or customize the installation.
- `--show-completion [bash|zsh|fish|powershell|pwsh]`
Show completion for the specified shell, to copy it or customize the installation.
Default: `None`.

- `--help`
- `--help`
Show this message and exit.
2 changes: 1 addition & 1 deletion docs/style.css
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,4 @@

.body {
font-weight: normal;
}
}
4 changes: 2 additions & 2 deletions hardware/light/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ Please install first [PlatformIO](http://platformio.org/) open source ecosystem

```bash
cd software/source/clients/esp32/src/client/
```
```

And build and upload the firmware with a simple command:

```bash
pio run --target upload
```
```

## Wifi

Expand Down
2 changes: 1 addition & 1 deletion project_management/hardware/devices/jetson-nano/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@

![](mac-share-internet-v2.png)

d. Now the Jetson should have connectivity!
d. Now the Jetson should have connectivity!
2 changes: 1 addition & 1 deletion project_management/software/os/TASKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@
# For later

- [ ] We could have `/i` which other interpreter's hit. That behaves more like the OpenAI POST endpoint with stream=True by default (i think this is important for users to see the exchange happening in real time, streaming `event/stream` or whatever). You could imagine some kind of handshake — another interpreter → my interpreter's /i → the sender is unrecognized → computer message is sent to /, prompting AI to ask the user to have the sending interpreter send a specific code → the user tells the sending interpreter to use that specific code → the sender is recognized and added to friends-list (`computer.inetwork.friends()`) → now they can hit eachother's i endpoints freely with `computer.inetwork.friend(id).message("hey")`.
- [ ] (OS team: this will require coordination with the OI core team, so let's talk about it / I'll explain at the next meetup.) When transfering skills that require OS control, the sender can replace those skills with that command, with one input "natural language query" (?) preceeded by the skill function name or something like that. Basically so if you ask it to do something you set up as a skill, it actually asks your computer to do it. If you ask your computer to do it directly, it's more direct.
- [ ] (OS team: this will require coordination with the OI core team, so let's talk about it / I'll explain at the next meetup.) When transfering skills that require OS control, the sender can replace those skills with that command, with one input "natural language query" (?) proceeded by the skill function name or something like that. Basically so if you ask it to do something you set up as a skill, it actually asks your computer to do it. If you ask your computer to do it directly, it's more direct.
1 change: 0 additions & 1 deletion software/.cursorignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
_archive
__pycache__
.idea

Loading

0 comments on commit 0eca547

Please sign in to comment.