Local NSFW Model Router

Introduction

The Local NSFW Model Router is a specialized tool designed for efficient comparison and utilization of AI language models optimized for NSFW (Not Safe For Work) content and role-playing scenarios. This project provides a local, privacy-focused solution for users to interact with and evaluate various AI models without relying on external servers or cloud-based services.

Key features include:

• Run models with Nexa SDK entirely on device for ultimate privacy
• Chat interface with Streamlit to chat with preset or customize character
• Built in model switcher
• Selection of uncensored language models at Nexa Model Hub with RAM and disk size suggestions

Installation

Prerequisites

1. Set up Miniconda and create a new conda virtual environment.
2. Ensure you have Git installed on your system.

Step-by-Step Installation

1. Clone the repository:

   git clone https://github.com/MaokunZhang/local-nsfw-model-router.git
   cd local-nsfw-model-router

2. Create and activate a new Conda environment:

   conda create --name nsfw_model_router python=3.12
   conda activate nsfw_model_router

3. Install Nexa SDK:
   • For CPU:

     pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/cpu --extra-index-url https://pypi.org/simple --no-cache-dir

   • For GPU (Metal - macOS):

     CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --prefer-binary --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple --no-cache-dir

   • For CUDA and AMD GPU support, refer to the [Nexa SDK Installation Guide](GitHub - NexaAI/nexa-sdk: Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. I).
4. Install other dependencies:

   pip install -r requirements.txt

Usage

Run the Streamlit app:

streamlit run app.py

Follow the on-screen instructions to start chatting with the default model or switch between different models.

File Structure

• app.py: Main Streamlit application
• utils/:
  • initialize.py: Initializes chat and loads models
  • gen_response.py: Handles output generation
  • customize.py: Allows users to customize character roles

Resources

• NexaAI Model Hub
• Nexa-SDK GitHub Repository

Disclaimer

This project is intended for adult users and contains NSFW content. Use responsibly and in accordance with local laws and regulations.