metadata

title: LLMServer
emoji: 👹
colorFrom: indigo
colorTo: purple
sdk: docker
pinned: false

LLM Server

This repository contains a FastAPI-based server that serves open-source Large Language Models from Hugging Face.

Getting Started

These instructions will help you set up and run the project on your local machine.

Prerequisites

Python 3.10 or higher
Git

Cloning the Repository

Choose one of the following methods to clone the repository:

HTTPS

git clone https://huggingface.co/spaces/TeamGenKI/LLMServer
cd project-name

SSH

git clone git@hf.co:spaces/TeamGenKI/LLMServer
cd project-name

Setting Up the Virtual Environment

Windows

# Create virtual environment
python -m venv myenv

# Activate virtual environment
myenv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Linux

# Create virtual environment
python -m venv myenv

# Activate virtual environment
source myenv/bin/activate

# Install dependencies
pip install -r requirements.txt

macOS

# Create virtual environment
python3 -m venv myenv

# Activate virtual environment
source myenv/bin/activate

# Install dependencies
pip3 install -r requirements.txt

Running the Application

Once you have set up your environment and installed the dependencies, you can start the FastAPI application:

uvicorn main.app:app --reload

The API will be available at http://localhost:8001

API Documentation

Once the application is running, you can access:

Interactive API documentation (Swagger UI) at http://localhost:8000/docs
Alternative API documentation (ReDoc) at http://localhost:8000/redoc

Deactivating the Virtual Environment

When you're done working on the project, you can deactivate the virtual environment:

deactivate

Contributing

[Add contributing guidelines here]

License

[Add license information here]

Project Structure

.
├── Dockerfile
├── main
│   ├── api.py
│   ├── app.py
│   ├── config.yaml
│   ├── env_template
│   ├── __init__.py
│   ├── logs
│   │   └── llm_api.log
│   ├── models
│   ├── __pycache__
│   │   ├── api.cpython-39.pyc
│   │   ├── app.cpython-39.pyc
│   │   ├── __init__.cpython-39.pyc
│   │   └── routes.cpython-39.pyc
│   ├── routes.py
│   ├── test_locally.py
│   └── utils
│       ├── errors.py
│       ├── helpers.py
│       ├── __init__.py
│       ├── logging.py
│       ├── __pycache__
│       │   ├── helpers.cpython-39.pyc
│       │   ├── __init__.cpython-39.pyc
│       │   ├── logging.cpython-39.pyc
│       │   └── validation.cpython-39.pyc
│       └── validation.py
├── README.md
└── requirements.txt