Make llama-cpp-python an optional dependency

[sinopeus]

I get the following when running poetry add raglite

raglite_benchmark on  main [!?] is 📦 v0.0.0 via 🐍 v3.10.16 (raglite-benchmark-env) 
⬢ [Docker] ❯ poetry add raglite   
Using version ^0.6.2 for raglite
Updating dependencies
Resolving dependencies... (6.7s)
Package operations: 2 installs, 9 updates, 0 removals
  - Updating fsspec (2024.12.0 -> 2025.2.0)
  - Updating huggingface-hub (0.27.1 -> 0.28.1)
  - Updating mdformat (0.7.21 -> 0.7.22)
  - Updating aiohttp (3.11.11 -> 3.11.12)
  - Updating mcp (1.2.0 -> 1.2.1)
  - Updating openai (1.60.1 -> 1.61.1)
  - Updating pytz (2024.2 -> 2025.1)
  - Updating sqlalchemy (2.0.37 -> 2.0.38)
  - Installing llama-cpp-python (0.3.7): Failed
PEP517 build of a dependency failed
Backend subprocess exited when trying to invoke build_wheel
    | Command '['/tmp/tmphngka48n/.venv/bin/python', '/opt/poetry-env/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py', 'build_wheel', '/tmp/tmph1gnx_xq']' returned non-zero exit status 1.
    | 
    | *** scikit-build-core 0.10.7 using CMake 3.25.1 (wheel)
    | *** Configuring CMake...
    | loading initial cache file /tmp/tmpykr8ztv_/build/CMakeInit.txt
    | -- The C compiler identification is GNU 12.2.0
    | -- The CXX compiler identification is GNU 12.2.0
    | -- Detecting C compiler ABI info
    | -- Detecting C compiler ABI info - done
    | -- Check for working C compiler: /usr/bin/gcc - skipped
    | -- Detecting C compile features
    | -- Detecting C compile features - done
    | -- Detecting CXX compiler ABI info
    | -- Detecting CXX compiler ABI info - done
    | -- Check for working CXX compiler: /usr/bin/g++ - skipped
    | -- Detecting CXX compile features
    | -- Detecting CXX compile features - done
    | -- Found Git: /usr/bin/git (found version "2.39.5") 
    | -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
    | -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
    | -- Found Threads: TRUE  
    | -- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
    | -- CMAKE_SYSTEM_PROCESSOR: aarch64
    | -- Including CPU backend
    | -- Found OpenMP_C: -fopenmp (found version "4.5") 
    | -- Found OpenMP_CXX: -fopenmp (found version "4.5") 
    | -- Found OpenMP: TRUE (found version "4.5")  
    | -- ARM detected
    | -- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
    | -- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
    | -- ARM -mcpu not found, -mcpu=native will be used
    | -- .... /ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o -c /tmp/tmpnwvx4foz/llama_cpp_python-0.3.7/vendor/llama.cpp/ggml/src/ggml-quants.c
    | ninja: build stopped: subcommand failed.
    | 
    | *** CMake build failed
Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with llama-cpp-python (0.3.7)
  - not supporting PEP 517 builds
  - not specifying PEP 517 build requirements correctly
  - the build requirements are incompatible with your operating system or Python version
  - the build requirements are missing system dependencies (eg: compilers, libraries, headers).
You can verify this by running pip wheel --no-cache-dir --use-pep517 "llama-cpp-python (==0.3.7)".
  - Updating pdftext (0.5.0 -> 0.5.1)
raglite_benchmark on  main [!?] is :package: v0.0.0 via :snake: v3.10.16 (raglite-benchmark-env) took 20s 
⬢ [Docker] ❯

[sipalstermans]

A workaround I found, is to pin llama-cpp-python=0.3.2 and install raglite. To actually make llama-cpp-python optional, @lsorber proposed to make the import conditional and avoid importing it until the point you actually need it like we do for chainlit:

@cli.command()
def chainlit(ctx: typer.Context) -> None:
    """Serve a Chainlit frontend."""
    # Set the environment variables for the Chainlit frontend.
    os.environ["RAGLITE_DB_URL"] = ctx.obj["db_url"]
    os.environ["RAGLITE_LLM"] = ctx.obj["llm"]
    os.environ["RAGLITE_EMBEDDER"] = ctx.obj["embedder"]
    # Import Chainlit here as it's an optional dependency.
    try:
        from chainlit.cli import run_chainlit
    except ImportError as error:
        error_message = "To serve a Chainlit frontend, please install the `chainlit` extra."
        raise ImportError(error_message) from error
    # Serve the frontend.
    run_chainlit(__file__.replace("_cli.py", "_chainlit.py"))

This is trickier with llama-cpp-python though because it's imports are currently more tightly integrated in RAGLite.

Note:
RAGLite should be optional on all local models. In the past I encountered issues when ms-marco-MiniLM-L-12-v2 was downloaded on import because _cli.py was part of the package's __init__.py and it's default values were evaluated on import:

"""RAGLite CLI."""

import json
import os
from typing import ClassVar

import typer
from pydantic_settings import BaseSettings, SettingsConfigDict

from raglite._config import RAGLiteConfig


class RAGLiteCLIConfig(BaseSettings):
    """RAGLite CLI config."""

    model_config: ClassVar[SettingsConfigDict] = SettingsConfigDict(
        env_prefix="RAGLITE_", env_file=".env", extra="allow"
    )

    mcp_server_name: str = "RAGLite"
    db_url: str = str(RAGLiteConfig().db_url)
    llm: str = RAGLiteConfig().llm
    embedder: str = RAGLiteConfig().embedder

This issue is resolved by #86 but we should make sure that this issue doesn't pop up somewhere else as well.

[rchretien]

@sipalstermans I will shortly work on making llama-cpp-python a runtime dependency instead of an installation-time dependency.