Add initial files

.gitignore

@@ -0,0 +1,9 @@
+.venv/
+
+.env
+
+
+__pycache__/
+
+# Jupyter Notebook checkpoints
+.ipynb_checkpoints/

README.md

@@ -1,11 +1,124 @@
----
-title: Sarvam Assignment
-emoji: 👁
-colorFrom: pink
-colorTo: yellow
-sdk: docker
-pinned: false
-short_description: Q&A using Rag and Agent
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# RAG and Agent-based Q&A System
+
+## Table of Contents
+- [Introduction](#introduction)
+- [Features](#features)
+- [System Architecture](#system-architecture)
+- [Prerequisites](#prerequisites)
+- [Installation](#installation)
+- [Usage](#usage)
+- [API Endpoints](#api-endpoints)
+- [Frontend Interface](#frontend-interface)
+
+This project implements a sophisticated Question-Answering system that combines Retrieval-Augmented Generation (RAG) with an intelligent agent. The system is designed to answer queries related to NCERT textbooks chapters, specifically focusing on the Sound chapter, while also handling general queries using web search capabilities. Do not: **Don't forget to specify your open-ai key in the .env file**
+
+The application serves two main functionalities:
+
+- A RAG system that retrieves relevant information from a vector database containing NCERT textbook content.
+- An agent-based system that can perform smart actions based on the user's query, including invoking the RAG system when appropriate and using additional tools like web search.
+
+## Introduction
+
+This project implements a sophisticated Question-Answering system that combines Retrieval-Augmented Generation (RAG) with an intelligent agent. The system is designed to answer queries related to NCERT textbooks, specifically focusing on the Sound chapter, while also handling general queries using web search capabilities.
+
+The application serves two main functionalities:
+1. A RAG system that retrieves relevant information from a vector database containing NCERT textbook content.
+2. An agent-based system that can perform smart actions based on the user's query, including invoking the RAG system when appropriate and using additional tools like web search.
+
+## Features
+
+- **RAG System**: 
+  - Utilizes a vector database to store and retrieve relevant information from NCERT textbooks.
+  - Provides accurate and concise answers to questions related to the Sound chapter.
+
+- **Intelligent Agent**:
+  - Determines when to use the RAG system based on the query content.
+  - Incorporates additional tools, including web search for non-textbook related queries.
+  - Calculates word count of responses when requested.
+
+- **FastAPI Backend**:
+  - Serves both RAG and Agent functionalities via separate endpoints.
+  - Ensures efficient and scalable handling of requests.
+
+- **Gradio Frontend**:
+  - Provides an intuitive user interface for interacting with both the RAG and Agent systems.
+  - Allows easy testing and demonstration of the system's capabilities.
+
+
+## System Architecture
+
+The system is built using the following key components:
+
+1. **Vector Store**: Stores embeddings of NCERT textbook content for efficient retrieval.
+2. **LangChain**: Facilitates the creation of the RAG chain and the agent.
+3. **OpenAI's ChatGPT**: Powers the language model for generating responses.
+4. **DuckDuckGo Search API**: Enables web search capabilities for the agent.
+5. **FastAPI**: Provides the backend API framework.
+6. **Gradio**: Creates the frontend user interface.
+
+## Installation
+
+1. Clone the repository and create a virtual env
+```bash
+python -m venv venv
+source venv/bin/activate
+```
+
+2.Install the required packages:
+```bash
+pip install -r requirements.txt
+```
+
+3. Setup up .env with all the environment variables
+```bash
+OPEN_API_KEY=your_openai_api_key
+UVICORN_HOST = 127.0.0.1
+UVICORN_PORT = 7860
+SOURCE_DATA = "../pdf_data"
+VECTOR_STORE = "../chroma_langchain_db"
+```
+
+## Usage
+
+To start the application run:
+```python
+python3 main_app.py
+```
+
+This will start the FastAPI server and launch the Gradio interface. You can access the Gradio interface by navigating to `http://localhost:8000` in your web browser.
+
+## API Endpoints
+
+The application exposes two main endpoints:
+
+1. `/rag` (POST): For querying the RAG system
+   - Request body: `{ "question": "Your question here" }`
+   - Response: `{ "answer": "Generated answer" }`
+
+2. `/agent` (POST): For interacting with the intelligent agent
+   - Request body: `{ "question": "Your question here" }`
+   - Response: `{ "answer": "Agent's response" }`
+
+
+## Frontend Interface
+
+The Gradio interface provides two tabs:
+
+1. **RAG System**: For asking questions related to the NCERT Sound chapter.
+2. **Agent**: For general queries, including those that may require web search or other tools.
+
+Users can type their questions in the input box and receive answers in real-time.
+
+### RAG System Interface
+
+![RAG System Interface](images/RAG_app.png)
+
+### Agent Interface
+
+![Agent Interface](images/Agent_app.png)
+
+### API Documentation
+
+The FastAPI automatic interactive API documentation is available at `/docs` endpoint:
+
+![API Documentation](images/API_docs.png)

chroma_langchain_db/237d30d4-0d4f-4fa7-b4ff-2981bcd6b160/data_level0.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f18abd8c514282db82706e52b0a33ed659cd534e925a6f149deb7af9ce34bd8e
+size 6284000

chroma_langchain_db/237d30d4-0d4f-4fa7-b4ff-2981bcd6b160/header.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:effaa959ce2b30070fdafc2fe82096fc46e4ee7561b75920dd3ce43d09679b21
+size 100

chroma_langchain_db/237d30d4-0d4f-4fa7-b4ff-2981bcd6b160/length.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7e5e2d7ceaf351ff89d12523776511335dcc308f06544d4cbbc32efa59828a2
+size 4000

main_app.py

@@ -0,0 +1,72 @@
+import os
+from fastapi import FastAPI
+from pydantic import BaseModel
+from dotenv import load_dotenv
+from utils.document_loader import load_pdf, create_unique_ids
+from utils.embeddings import get_embeddings
+from utils.vector_store import create_vector_store, get_retriever, load_vector_store
+from utils.rag_chain import get_model, create_rag_chain, get_conversational_rag_chain
+from utils.gradio_interface import create_gradio_interface
+from utils.agent import init_agent, get_agent_response
+import gradio as gr
+
+load_dotenv()
+
+app = FastAPI()
+
+class QuestionRequest(BaseModel):
+    question: str
+
+class AnswerResponse(BaseModel):
+    answer: str
+
+def init_rag_system():
+    pdf_path = os.getenv("SOURCE_DATA")
+    vector_store_path = os.getenv("VECTOR_STORE")
+    # Load embeddings
+    embeddings = get_embeddings()
+    if os.path.exists(vector_store_path) and os.listdir(vector_store_path):
+        print("Loading existing vector store...")
+        vector_store = load_vector_store(embeddings)
+    else:
+        print("Creating new vector store...")
+        documents = load_pdf(pdf_path)
+        unique_ids = create_unique_ids(documents)
+        vector_store = create_vector_store(documents, unique_ids, embeddings)
+    retriever = get_retriever(vector_store)
+    model = get_model()
+    rag_chain = create_rag_chain(model, retriever)
+    return get_conversational_rag_chain(rag_chain)
+
+# Initialize conversational RAG chain
+conversational_rag_chain = init_rag_system()
+
+# Initialize agent
+agent = init_agent()
+
+@app.post("/rag", response_model=AnswerResponse)
+async def ask_rag_question(request: QuestionRequest):
+    print(f"RAG Question: {request.question}")
+    response = conversational_rag_chain.invoke(
+        {"input": request.question},
+        config={"configurable": {"session_id": "default_session"}}
+    )
+    return AnswerResponse(answer=response["answer"])
+
+@app.post("/agent", response_model=AnswerResponse)
+async def ask_agent_question(request: QuestionRequest):
+    print(f"Agent Question: {request.question}")
+    response = get_agent_response(agent, request.question)
+    return AnswerResponse(answer=response)
+
+interface = create_gradio_interface(app, conversational_rag_chain, agent)
+app = gr.mount_gradio_app(app, interface, path="/")
+
+if __name__ == "__main__":
+    import uvicorn
+    uvicorn.run(
+        app,
+        host=os.getenv("UVICORN_HOST"),
+        port=int(os.getenv("UVICORN_PORT")),
+        # reload=True
+    )

requirements.txt

@@ -0,0 +1,9 @@
+chromadb
+langchain
+langchain-community
+langchain-openai
+langchain-chroma
+gradio
+openai
+pydantic
+duckduckgo-search

utils/agent.py

@@ -0,0 +1,114 @@
+import os
+from dotenv import load_dotenv
+from utils.embeddings import get_embeddings
+from utils.vector_store import load_vector_store
+from langchain_community.utilities import DuckDuckGoSearchAPIWrapper
+from langchain_community.tools import DuckDuckGoSearchResults
+from langchain.chains import create_retrieval_chain
+from langchain.chains.combine_documents import create_stuff_documents_chain
+from langchain.tools import tool
+from langchain_openai import ChatOpenAI
+from langchain.agents import create_openai_functions_agent, AgentExecutor
+from langchain_community.chat_message_histories import ChatMessageHistory
+from langchain_core.messages import AIMessage
+from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
+from langchain_core.runnables import RunnableWithMessageHistory
+
+load_dotenv()
+
+wrapper = DuckDuckGoSearchAPIWrapper(max_results=2)
+search_web = DuckDuckGoSearchResults(api_wrapper=wrapper, source="news")
+
+@tool
+def rag_tool(query:str)->str:
+    """
+    The function queries the vector db and retrieves the answer
+    """
+    embeddings = get_embeddings()
+    vector_store = load_vector_store(embeddings)
+    retriver = vector_store.as_retriever(search_type='similarity',search_kwargs={"k": 2})
+    
+    system_prompt = (
+    "You are an assistant for question-answering tasks. "
+    "Use the following pieces of retrieved context to answer "
+    "the question. If you don't know the answer, say that you "
+    "don't know. Use three sentences maximum and keep the "
+    "answer concise."
+    "\n\n"
+    "{context}"
+    )
+
+    prompt = ChatPromptTemplate.from_messages(
+        [
+            ("system", system_prompt),
+            ("human", "{input}"),
+        ]
+    )
+
+    question_answer_chain = create_stuff_documents_chain(get_model_use(), prompt)
+    rag_chain = create_retrieval_chain(retriver, question_answer_chain)
+    response = rag_chain.invoke({"input":query})
+    return (response["answer"])
+
+
+@tool
+def calculate_word_count(words: str) -> int:
+    """
+    The function helps in calculating the number of words present in the responses
+    """
+    response = words.split()
+    return len(response)
+
+tools = [rag_tool,search_web, calculate_word_count]
+
+prompt = ChatPromptTemplate.from_messages([
+    ("system", """
+    You are an assistant helping the user with queries related to the NCERT Sound chapter and real-time events or factual information. Follow these instructions:
+
+    1. **NCERT Sound Chapter Queries**:
+       - Use your rag tool provide to answer any query regarding NCERT Sound chapter to answer questions such as:
+         - What is an echo?
+         - How is sound propagated?
+         - What are the applications of ultrasound?
+       - STRICT RULE: Do not use external tools after using rag_tool
+
+    2. **Non-Sound Chapter Queries**:
+       - For any questions unrelated to the Sound chapter, such as real-time events, news, or factual information not covered in the Sound chapter, use the search tool to provide the latest and most accurate information.
+
+    3. **Counting Words in a Response**:
+       - If the query involves counting the number of words in a response, use the `calculate_word_count` tool to determine the word count.
+
+    4. **Clarification**:
+       - If the query is unclear or ambiguous, clarify the user's intent before selecting the appropriate tool or providing a response.
+
+    Be concise, accurate, and use the appropriate tool or knowledge based on the query type. Do not confuse the tools or mix the instructions for different query types.
+    """),
+    MessagesPlaceholder(variable_name="chat_history"),
+    ("user", "Form input details: {input}"),
+    MessagesPlaceholder(variable_name="agent_scratchpad"),
+])
+
+def get_model_use():
+    return ChatOpenAI(api_key=os.getenv("OPEN_API_KEY"),temperature=0)
+
+def init_agent():
+    llm = get_model_use()
+    agent = create_openai_functions_agent(llm, tools, prompt)
+    agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
+    message_history = ChatMessageHistory()
+    agent_with_chat_history = RunnableWithMessageHistory(
+        agent_executor,
+        lambda session_id: message_history,
+        input_messages_key="input",
+        history_messages_key="chat_history",
+    )
+    return agent_with_chat_history
+
+def get_agent_response(agent, user_input, session_id="agentic_trial"):
+    response = agent.invoke(
+        {
+            "input": user_input
+        },
+        config={"configurable": {"session_id": session_id}}
+    )
+    return response['output']

utils/document_loader.py

@@ -0,0 +1,14 @@
+import os
+from langchain_community.document_loaders import PDFPlumberLoader
+
+def load_pdf(directory):
+    
+    for file in os.listdir(directory):
+        file_path = os.path.join(directory,file)
+        loader = PDFPlumberLoader(file_path)
+        document = loader.load()
+
+    return document
+
+def create_unique_ids(documents):
+    return [f"{doc.metadata['source']}_page_{doc.metadata['page']}" for doc in documents]

utils/embeddings.py

@@ -0,0 +1,8 @@
+import os
+from langchain_openai import OpenAIEmbeddings
+from dotenv import load_dotenv
+
+load_dotenv()
+
+def get_embeddings():
+    return OpenAIEmbeddings(model='text-embedding-3-small', api_key=os.getenv("OPEN_API_KEY"))

utils/gradio_interface.py

@@ -0,0 +1,46 @@
+import gradio as gr
+from fastapi import FastAPI
+from fastapi.responses import JSONResponse
+
+def create_gradio_interface(app: FastAPI, conversational_rag_chain, agent):
+    def qa_function(message, history, system):
+        if system == "RAG":
+            response = conversational_rag_chain.invoke(
+                {"input": message},
+                config={"configurable": {"session_id": "abc123"}}
+            )
+            return response["answer"]
+        elif system == "Agent":
+            response = agent.invoke(
+                {"input": message},
+                config={"configurable": {"session_id": "agent_session"}}
+            )
+            return response['output']
+
+    gr_app = gr.Blocks()
+
+    with gr_app:
+        gr.Markdown("# NCERT Q&A System")
+        gr.Markdown("Ask questions based on the NCERT Sound chapter or use the Agent for broader queries.")
+
+        chatbot = gr.Chatbot()
+        msg = gr.Textbox()
+        clear = gr.Button("Clear")
+
+        system_choice = gr.Radio(["RAG", "Agent"], label="Choose System", value="RAG")
+
+        def user(user_message, history, system):
+            return "", history + [[user_message, None]]
+
+        def bot(history, system):
+            user_message = history[-1][0]
+            bot_message = qa_function(user_message, history, system)
+            history[-1][1] = bot_message
+            return history
+
+        msg.submit(user, [msg, chatbot, system_choice], [msg, chatbot], queue=False).then(
+            bot, [chatbot, system_choice], chatbot
+        )
+        clear.click(lambda: None, None, chatbot, queue=False)
+
+    return gr_app

utils/rag_chain.py

@@ -0,0 +1,62 @@
+import os
+from langchain_openai import ChatOpenAI
+from langchain.chains import create_retrieval_chain, create_history_aware_retriever
+from langchain.chains.combine_documents import create_stuff_documents_chain
+from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
+from langchain_core.runnables.history import RunnableWithMessageHistory
+from langchain_community.chat_message_histories import ChatMessageHistory
+
+def get_model():
+    return ChatOpenAI(api_key=os.getenv("OPEN_API_KEY"))
+
+def create_contextualize_q_prompt():
+    contextualize_q_system_prompt = (
+        "Given a chat history and the latest user question "
+        "which might reference context in the chat history, "
+        "formulate a standalone question which can be understood "
+        "without the chat history. Do NOT answer the question, "
+        "just reformulate it if needed and otherwise return it as is."
+    )
+    return ChatPromptTemplate.from_messages([
+        ("system", contextualize_q_system_prompt),
+        MessagesPlaceholder("chat_history"),
+        ("human", "{input}"),
+    ])
+
+def create_qa_prompt():
+    qa_system_prompt = """You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. \
+    The retrieved content belongs to subject textbooks present. You will receive different chunks, each of which belongs to a single page of a textbook. \
+    Using the chunk given, think logically and answer the questions from the user. \
+    If you are not able to identify the relevant information regarding to the user's question in the retrieved chunks, then just return 'No data found'.\
+    Use three sentences maximum and keep the answer concise. \
+
+    {context}"""
+    return ChatPromptTemplate.from_messages([
+        ("system", qa_system_prompt),
+        MessagesPlaceholder("chat_history"),
+        ("human", "{input}"),
+    ])
+
+def create_rag_chain(model, retriever):
+    contextualize_q_prompt = create_contextualize_q_prompt()
+    qa_prompt = create_qa_prompt()
+
+    history_aware_retriever = create_history_aware_retriever(model, retriever, contextualize_q_prompt)
+    question_answer_chain = create_stuff_documents_chain(model, qa_prompt)
+    return create_retrieval_chain(history_aware_retriever, question_answer_chain)
+
+def get_conversational_rag_chain(rag_chain):
+    store = {}
+
+    def get_session_history(session_id: str):
+        if session_id not in store:
+            store[session_id] = ChatMessageHistory()
+        return store[session_id]
+
+    return RunnableWithMessageHistory(
+        rag_chain,
+        get_session_history,
+        input_messages_key="input",
+        history_messages_key="chat_history",
+        output_messages_key="answer",
+    )

utils/vector_store.py

@@ -0,0 +1,36 @@
+import os
+from dotenv import load_dotenv
+from langchain_chroma import Chroma
+
+load_dotenv()
+
+persist_directory = os.getenv("VECTOR_STORE")
+
+def create_vector_store(documents, unique_ids, embeddings):
+    """
+    Creates a new vector store with the given documents, unique IDs, and embeddings.
+    """
+    vector_store = Chroma(
+        collection_name="NCERT-Chapters",
+        embedding_function=embeddings,
+        persist_directory=persist_directory  
+    )
+    vector_store.add_documents(documents=documents, ids=unique_ids)
+    vector_store.persist() 
+    return vector_store
+
+def load_vector_store(embeddings):
+    """
+    Loads an existing vector store using the embeddings provided.
+    """
+    return Chroma(
+        collection_name="NCERT-Chapters",
+        persist_directory=persist_directory, 
+        embedding_function=embeddings
+    )
+
+def get_retriever(vector_store, k=5):
+    """
+    Returns a retriever object to search through the vector store.
+    """
+    return vector_store.as_retriever(search_kwargs={"k": k})