This project runs a local llm agent based RAG model on LlamaIndex.
Upload files and ask questions over your documents.
Upload upto 10 files within 5 mb; max_size(5 mb) can be configured.
LLM Used : Ollama
Embedding : HuggingFaceEmbeddings with llama-index
User Interface : Chainlit
Note : Ollama runs on Linux or MacOS - automatically uses hardware accelerotor if available in this devices (GPU and MPS respectively), otherwise uses CPU
To install Ollama on a Mac, you need to have macOS 11 Big Sur or later. The installation process can be done in a few steps:
-
Download Ollama: You can download Ollama for macOS from the official website.
-
Install with Homebrew: If you prefer using Homebrew, you can install Ollama with the following command: \
brew install ollama
-
Start Ollama services on your mac, which runs on a local server which will serve all the api calls used in any llm application which will use Ollama:
brew services start ollama
For Linux users, the installation process is straightforward:
- Install with a Single Command: You can install Ollama using a single command by running in your terminal:
curl https://ollama.ai/install.sh | sh
To find a model in Ollama, you can visit the Ollama library page. This page lists all the available models that you can pull and run locally using Ollama.
ollama pull mistral:latest
Now, we are all set for writing our rag application in LlamaIndex.
Note : we can use downloaded gguf files from huggingface and use LLamaCPP or any other llm loader classes for gguf files instead of Ollama.
-
Clone the repo using git:
git clone https://github.com/rauni-iitr/llamaindex_ollama_chainlit.git
-
Create a virtual enviornment, with 'venv' or with 'conda' and activate.
python3 -m venv .venv source .venv/bin/activate
-
Install dependencies within the created virtual enviornment using:
- requirements.txt:
pip install -r requirements.txt
- Manually installing packages:
pip install llama-index chromadb chainlit pyyaml pip install llama-index-embeddings-huggingface pip install llama-index-llms-ollama pip install llama-index-vector-stores-chroma
- requirements.txt:
To run the model:
chainlit run app.py