Learning Llamaindex

September, 2023

LlamaIndex is a data framework for LLM applications to ingest, structure, and access private or domain-specific data. LLM applications sometimes require augmenting these models with private or domain-specific data distributed across siloed applications and data stores, behind APIs, in SQL databases, or trapped in PDFs and slide decks.

Installing Llamaindex. You should do a pip install, and install the packages for the llm you want. LlamaIndex defaults to openai for basic functioning

In the below example, I focus on ctransformers and run any models it can run. Download one of the models which ctransformers support and use it instead of `model.bin` below.

pip install llama_index torch
pip install ctransformers
            

Import the relevant stuff in your python file.

import logging
import sys
import torch
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from langchain.llms import CTransformers
            

I put some text files in my `./data` folder and run the below. Now I can ask some questions about the texts I put there.

logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

documents = SimpleDirectoryReader("./data").load_data()

llm = CTransformers(model='./model.bin', model_type='llama')

service_context = ServiceContext.from_defaults(chunk_size=1024, llm=llm)
index = VectorStoreIndex.from_documents(documents, service_context=service_context)

query_engine = index.as_query_engine()

response = query_engine.query("What did the author do growing up?")
print(response)