Skip to content

Latest commit

 

History

History
15 lines (10 loc) · 1.32 KB

README.md

File metadata and controls

15 lines (10 loc) · 1.32 KB
Raw

Ollama + Mixtral + Langchain example (deployed to Cloudflare)

You can use Cloudflare Tunnel to open a secure HTTPS tunnel to a running Ollama instance on your local computer.

Using Cloudflare Workers, you can connect the Ollama instance to Langchain and use cutting-edge models like Mixtral.

Usage

  1. Configure your Workers project and deploy it (see "Get started" if you don't know how)
  2. Install cloudflared.
  3. Start a new tunnel with tunnel.sh.
  4. Find the output of your new Cloudflare Tunnel and set it as the TUNNEL secret in your Workers project: echo "YOURURL" | npx wrangler secret put TUNNEL.
  5. Request GET $workersURL/ with an optional query param query to directly query the model on your local machine. The response is streaming!

This is not recommended for production use, but shows how you can use local tooling to generate embeddings instead of relying on a third-party API.