Upload 7 files

.gitignore

@@ -0,0 +1,160 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/

Dockerfile

@@ -0,0 +1,11 @@
+FROM python:3.9
+RUN useradd -m -u 1000 user
+USER user
+ENV HOME=/home/user \
+    PATH=/home/user/.local/bin:$PATH
+WORKDIR $HOME/app
+COPY --chown=user . $HOME/app
+COPY ./requirements.txt ~/app/requirements.txt
+RUN pip install -r requirements.txt
+COPY . .
+CMD ["chainlit", "run", "app.py", "--port", "7860"]

README.md

@@ -1,11 +1,63 @@
+# SafeMate
+
+Welcome to **SafeMate**, an occupational health and safety assistant designed to serve construction field workers. This project is currently in the proof of concept stage and aims to provide immediate, accessible safety guidelines and answers derived from authoritative sources. The core of this assistant is built upon a chat interface that draws its responses from the safety manual provided by the Port of Seattle, offering a practical and user-friendly approach to workplace safety.
+
+## Project Overview
+
+SafeMate is crafted to transform the way construction workers interact with safety manuals, making it easier than ever to get answers to crucial safety questions on the job. By leveraging state-of-the-art machine learning models and natural language processing techniques, SafeMate delivers relevant safety information through a simple chat interface.
+
+### Current State of the Project
+
+This project is in its early stages, and the current implementation serves as a proof of concept. The backbone of SafeMate is a chat interface that allows users to query a safety manual, specifically the one provided by the Port of Seattle, which is publicly available online.
+
+### Project Structure
+
+The project is organized as follows:
+
+D:.
+├───.chainlit
+│   └───translations
+├───.files
+├───app
+├───data
+│   ├───processed
+│   ├───raw
+│   └───results
+├───models
+├───notebooks
+│   └───.ipynb_checkpoints
+├───scripts
+├───utils
+└───__pycache__
+
+Note: Some folders have been created to accommodate future developments and are not in use at the moment.
+
+### Achievements So Far
+
+1. **Synthetic QA Development**: Utilization of the RAGAS library to assess embeddings and models for effective question-answering.
+2. **Data Processing**: Transformation of the raw PDF manual into a structured format, with embeddings stored in a vector store for quick retrieval.
+3. **User Interface**: Implementation of ChainLit as the user interface, facilitating a seamless interaction where users can prompt questions, and the system retrieves context to provide informed responses.
+4. **Accessibility**: The project's code is hosted on GitHub, with a live demo available on a Hugging Face space, demonstrating the capabilities and potential of SafeMate.
+
+## Future Directions
+
+As SafeMate evolves, we plan to expand the knowledge base beyond the Port of Seattle's safety manual, incorporate feedback mechanisms for continuous improvement, and refine our models for better accuracy and user experience.
+
+## Contact
+
+Cesáreo González Alvarez
+Github: cga-telice
+Linkedin: (https://www.linkedin.com/in/caesaripse/)
+
+## License
+
+SafeMate is released under the [MIT License](LICENSE).
+
+## Acknowledgments
+
+A special thank you to the Port of Seattle for making their safety manual publicly accessible and serving as the foundation for this project.
+
 ---
-title: SafeMate
-emoji: 🦀
-colorFrom: yellow
-colorTo: gray
-sdk: docker
-pinned: false
-license: openrail
----
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+For more information, updates, and to try out the SafeMate proof of concept, please visit our [GitHub repository](link-to-repo) and [Hugging Face demo space](link-to-demo).
+

app.py

@@ -0,0 +1,132 @@
+# You can find this code for Chainlit python streaming here (https://docs.chainlit.io/concepts/streaming/python)
+
+# OpenAI Chat completion
+import os
+from dotenv import load_dotenv
+from getpass import getpass
+from operator import itemgetter
+
+import openai
+from openai import AsyncOpenAI  # importing openai for API usage
+
+import chainlit as cl  # importing chainlit for our app
+from chainlit.prompt import Prompt, PromptMessage  # importing prompt tools
+from chainlit.playground.providers import ChatOpenAI  # importing ChatOpenAI tools
+
+from langchain_community.document_loaders import PyMuPDFLoader
+from langchain.text_splitter import RecursiveCharacterTextSplitter
+from langchain_openai import OpenAIEmbeddings
+from langchain_community.vectorstores import FAISS
+from langchain import hub
+from langchain.prompts import ChatPromptTemplate
+import faiss
+
+# Working directory
+# Get the absolute path to the directory containing the script
+script_dir = os.path.dirname(os.path.abspath(__file__))
+
+# Change the current working directory to the script directory
+os.chdir(script_dir)
+
+# Set the enviroment variables
+
+load_dotenv()
+
+openai.api_key = os.getenv("OPENAI_API_KEY")
+
+# Load the document vector store
+file_name = "Seattle" 
+path_raw = "data//raw//" + file_name + ".pdf" # The path where the raw documents are stored
+path_processed = "data//processed//" + file_name + ".faiss" # The path where we will store the index
+
+embeddings = OpenAIEmbeddings(
+    model="text-embedding-3-small"
+)
+
+vector_store = FAISS.from_texts([""], embeddings)
+
+vs=vector_store.load_local(path_processed, embeddings, allow_dangerous_deserialization=True)
+
+# Build the retriever
+
+retriever = vs.as_retriever()
+
+# Configure the Prompt templates
+
+system_template = """You are a helpful but prudent occupational health and
+safety assistant. Your anwswers will be grounded on the context.
+If you don´t know an answer you will say that you don´t know.
+"""
+
+user_template = """
+# Context:
+{context}
+
+# Question:
+{question}
+"""
+
+@cl.on_chat_start  # marks a function that will be executed at the start of a user session
+async def start_chat():
+    settings = {
+        "model": "gpt-3.5-turbo",
+        "temperature": 0,
+        "max_tokens": 500,
+        "top_p": 1,
+        "frequency_penalty": 0,
+        "presence_penalty": 0,
+    }
+
+    cl.user_session.set("settings", settings)
+
+
+@cl.on_message  # marks a function that should be run each time the chatbot receives a message from a user
+async def main(message: cl.Message):
+    settings = cl.user_session.get("settings") # gets the settings of the started session
+
+    client = AsyncOpenAI()
+
+    document_list = retriever.invoke(message.content)
+    context = " ".join([doc.page_content for doc in document_list])
+    
+    print(message.content)
+
+    print(context)
+
+    prompt = Prompt(
+        provider=ChatOpenAI.id,
+        messages=[
+            PromptMessage(
+                role="system",
+                template=system_template,
+                formatted=system_template,
+            ),
+            PromptMessage(
+                role="user",
+                template=user_template,
+                formatted=user_template.format(question=message.content, context=context),
+            ),
+        ],
+        inputs={"question": message.content, "context": context},
+        settings=settings,
+    )
+
+    print([m.to_openai() for m in prompt.messages])
+
+    msg = cl.Message(content="")
+
+    # Call OpenAI
+    async for stream_resp in await client.chat.completions.create(
+        messages=[m.to_openai() for m in prompt.messages], stream=True, **settings
+    ):
+        token = stream_resp.choices[0].delta.content
+        if not token:
+            token = ""
+        await msg.stream_token(token)
+
+    # Update the prompt object with the completion
+    prompt.completion = msg.content
+    msg.prompt = prompt
+
+    # Send and close the message stream
+    await msg.send()

chainlit.md

@@ -0,0 +1,9 @@
+# Welcome to SafeMate! 🚀🤖
+
+Hi there! 👋 We're excited to have you on board. 
+
+**SafeMate** is proof of concept app designed to assist construction field workers with occupational health and safety documentation.
+
+At this point the app is a chat with a document, a [Construction Safety Manual from the Port of seattle](https://www.portseattle.org/sites/default/files/2018-03/Construction_Safety_Manual.pdf)
+
+**Go ahead and chat with the Safety Manual!**

requirements.txt

@@ -0,0 +1,11 @@
+chainlit==0.7.700
+cohere==4.37
+openai
+tiktoken==0.5.2
+python-dotenv==1.0.0
+langchain
+langchain-core
+langchain-community
+langchain-openai
+langchainhub
+faiss_cpu