Upload 5 files

LEGAL

@@ -0,0 +1,26 @@
+Legal Notice
+Reverse Engineering Disclaimer
+
+DeepInfra-Wrapper is a project developed for educational purposes and is not affiliated with or endorsed by DeepInfra or any associated entities. The use of this project is subject to the following legal notice.
+Educational Purposes Only
+
+DeepInfra-Wrapper was created solely for educational and research purposes to explore and understand the functionality of the DeepInfra API. The reverse-engineering efforts undertaken in this project are intended to enhance knowledge and skills related to software development and API interaction.
+Not Authorized by DeepInfra
+
+This project, including its codebase and associated documentation, is not authorized by DeepInfra or any affiliated parties. It is an independent initiative that aims to provide a learning resource for individuals interested in working with APIs and Flask applications.
+Ethical Usage
+
+Users are expected to adhere to ethical standards when using DeepInfra-Wrapper. The project should be utilized responsibly and in compliance with applicable laws and regulations. Any use of DeepInfra-Wrapper for malicious or unauthorized activities is strictly prohibited.
+No Warranty
+
+DeepInfra-Wrapper is distributed "as is" and without any warranty. The authors and contributors of this project make no representations or warranties regarding the accuracy, functionality, or reliability of the code. Users are solely responsible for their use of the project.
+Legal Implications
+
+Users of DeepInfra-Wrapper are responsible for ensuring that their use complies with all relevant laws and regulations. Any unauthorized or improper use of the project may result in legal consequences. The authors and contributors disclaim any liability arising from the use of DeepInfra-Wrapper.
+Open Source License
+
+DeepInfra-Wrapper is released under the MIT License. Users are encouraged to review and comply with the terms of this license.
+
+By using DeepInfra-Wrapper, you acknowledge that you have read and understood this legal notice and agree to use the project responsibly and in accordance with applicable laws.
+
+For any inquiries or concerns related to this legal notice, please contact the project maintainers.

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 4sh
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,20 +1,98 @@
----
-title: Test Space
-emoji: 🌍
-colorFrom: yellow
-colorTo: indigo
-sdk: docker
-pinned: false
-license: mit
----
+# DeepInfra-Wrapper
 
-This is a templated Space for [Shiny for Python](https://shiny.rstudio.com/py/). 
+DeepInfra-Wrapper is a Python Flask project designed to provide a convenient and free interface for utilizing the DeepInfra API through reverse-engineering. It serves as a local and global server host, allowing users to interact with the DeepInfra chat completion models using Python requests.
 
+## Features
 
-To get started with a new app do the following:
+- **Local and Global Server**: Choose between a local server or utilize a global server with Cloudflare integration for enhanced performance.
 
-1) Install Shiny with `pip install shiny`
-2) Create a new app with `shiny create .`
-3) Then run the app with `shiny run --reload`
+- **Chat Completion**: Easily generate chat completions by sending messages to the DeepInfra API.
 
-To learn more about this framework please see the [Documentation](https://shiny.rstudio.com/py/docs/overview.html).
+- **Model Selection**: Access a variety of models for different use cases.
+
+- **Streaming Support**: Enable real-time streaming for dynamic chat interactions.
+
+## Getting Started
+
+### Prerequisites
+
+- Python 3.6 or higher
+- Flask
+- Flask-CORS
+- Flask-Cloudflared
+- Requests
+- Fake User Agent
+
+### Installation
+
+1. Clone the repository:
+
+   ```bash
+   git clone https://github.com/Recentaly/DeepInfra-Wrapper.git
+   ```
+
+2. Install dependencies:
+
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+3. Run the Flask application:
+
+   ```bash
+   python app.py
+   ```
+
+## Configuration
+
+Adjust the configuration settings in the `assets/config.json` file to customize your DeepInfra-Wrapper experience.
+
+```json
+{
+    "use_global": true
+}
+```
+
+## Usage
+
+### Chat Completion
+
+Send a POST request to `/chat/completions` with the following JSON payload (messages must be in OpenAI format):
+
+```json
+{
+    "messages": [{"role": "user", "content": "Hello, World!"}],
+    "model": "meta-llama/Llama-2-70b-chat-hf",
+    "max_tokens": 150,
+    "top_p": 1,
+    "stream": true
+}
+```
+
+### Get Models
+
+Retrieve the available models by sending a GET request to `/models`.
+
+### Check API Status
+
+Verify the API status by accessing the root route `/`.
+
+## Error Handling
+
+The API gracefully handles errors, such as forbidden requests, providing meaningful error messages.
+
+## Google Colab
+
+The server is also usable on [This Google Colab Link](https://colab.research.google.com/drive/15sQ6sjZJYUincL3otypxfH96aCfLQ7HZ?usp=sharing)
+
+## License
+
+This project is licensed under the [MIT License](LICENSE).
+
+## Acknowledgments
+
+- Special thanks to the DeepInfra team for providing the chat completion models.
+
+## Contact
+
+For issues and inquiries, please open an [issue](https://github.com/Recentaly/DeepInfra-Wrapper/issues).

app.py

@@ -1,151 +1,197 @@
-from pathlib import Path
-from typing import List, Dict, Tuple
-import matplotlib.colors as mpl_colors
-
-import pandas as pd
-import seaborn as sns
-import shinyswatch
-
-from shiny import App, Inputs, Outputs, Session, reactive, render, req, ui
-
-sns.set_theme()
-
-www_dir = Path(__file__).parent.resolve() / "www"
-
-df = pd.read_csv(Path(__file__).parent / "penguins.csv", na_values="NA")
-numeric_cols: List[str] = df.select_dtypes(include=["float64"]).columns.tolist()
-species: List[str] = df["Species"].unique().tolist()
-species.sort()
-
-app_ui = ui.page_fillable(
-    shinyswatch.theme.minty(),
-    ui.layout_sidebar(
-        ui.sidebar(
-            # Artwork by @allison_horst
-            ui.input_selectize(
-                "xvar",
-                "X variable",
-                numeric_cols,
-                selected="Bill Length (mm)",
-            ),
-            ui.input_selectize(
-                "yvar",
-                "Y variable",
-                numeric_cols,
-                selected="Bill Depth (mm)",
-            ),
-            ui.input_checkbox_group(
-                "species", "Filter by species", species, selected=species
-            ),
-            ui.hr(),
-            ui.input_switch("by_species", "Show species", value=True),
-            ui.input_switch("show_margins", "Show marginal plots", value=True),
-        ),
-        ui.output_ui("value_boxes"),
-        ui.output_plot("scatter", fill=True),
-        ui.help_text(
-            "Artwork by ",
-            ui.a("@allison_horst", href="https://twitter.com/allison_horst"),
-            class_="text-end",
-        ),
-    ),
-)
-
-
-def server(input: Inputs, output: Outputs, session: Session):
-    @reactive.Calc
-    def filtered_df() -> pd.DataFrame:
-        """Returns a Pandas data frame that includes only the desired rows"""
-
-        # This calculation "req"uires that at least one species is selected
-        req(len(input.species()) > 0)
-
-        # Filter the rows so we only include the desired species
-        return df[df["Species"].isin(input.species())]
-
-    @output
-    @render.plot
-    def scatter():
-        """Generates a plot for Shiny to display to the user"""
-
-        # The plotting function to use depends on whether margins are desired
-        plotfunc = sns.jointplot if input.show_margins() else sns.scatterplot
-
-        plotfunc(
-            data=filtered_df(),
-            x=input.xvar(),
-            y=input.yvar(),
-            palette=palette,
-            hue="Species" if input.by_species() else None,
-            hue_order=species,
-            legend=False,
-        )
-
-    @output
-    @render.ui
-    def value_boxes():
-        df = filtered_df()
-
-        def penguin_value_box(title: str, count: int, bgcol: str, showcase_img: str):
-            return ui.value_box(
-                title,
-                count,
-                {"class_": "pt-1 pb-0"},
-                showcase=ui.fill.as_fill_item(
-                    ui.tags.img(
-                        {"style": "object-fit:contain;"},
-                        src=showcase_img,
-                    )
-                ),
-                theme_color=None,
-                style=f"background-color: {bgcol};",
-            )
-
-        if not input.by_species():
-            return penguin_value_box(
-                "Penguins",
-                len(df.index),
-                bg_palette["default"],
-                # Artwork by @allison_horst
-                showcase_img="penguins.png",
-            )
-
-        value_boxes = [
-            penguin_value_box(
-                name,
-                len(df[df["Species"] == name]),
-                bg_palette[name],
-                # Artwork by @allison_horst
-                showcase_img=f"{name}.png",
-            )
-            for name in species
-            # Only include boxes for _selected_ species
-            if name in input.species()
-        ]
-
-        return ui.layout_column_wrap(*value_boxes, width = 1 / len(value_boxes))
-
-
-# "darkorange", "purple", "cyan4"
-colors = [[255, 140, 0], [160, 32, 240], [0, 139, 139]]
-colors = [(r / 255.0, g / 255.0, b / 255.0) for r, g, b in colors]
-
-palette: Dict[str, Tuple[float, float, float]] = {
-    "Adelie": colors[0],
-    "Chinstrap": colors[1],
-    "Gentoo": colors[2],
-    "default": sns.color_palette()[0],  # type: ignore
-}
-
-bg_palette = {}
-# Use `sns.set_style("whitegrid")` to help find approx alpha value
-for name, col in palette.items():
-    # Adjusted n_colors until `axe` accessibility did not complain about color contrast
-    bg_palette[name] = mpl_colors.to_hex(sns.light_palette(col, n_colors=7)[1])  # type: ignore
-
-
-app = App(
-    app_ui,
-    server,
-    static_assets=str(www_dir),
-)
+# ---------------------------------------- IMPORTS ---------------------------------------- #
+
+# import flask and flask_cors to host the api
+from flask import Flask, request, jsonify, render_template
+from flask_cors import CORS
+
+# import the api class
+from assets.source import api, non_streamed_format
+
+# import addon
+from assets.source.addons import * # here we only use 'create_cloudflare_tunnel' and 'translate' from the addons
+
+# logging module for debugging
+import logging
+
+# json module to parse json
+from json import loads
+
+# ---------------------------------------- CONFIGURE LOCAL SERVER ---------------------------------------- #
+
+# create flask app
+app = Flask(__name__)
+app.template_folder = "assets/templates"
+
+# enable cors
+CORS(app)
+
+# ---------------------------------------- READ FROM CONFIG FILE ---------------------------------------- #
+with (open("assets/config.json", "r")) as f:
+
+    config_file = loads(f.read())
+
+    # copy constants over
+    DEBUG: bool = config_file.get("DEBUG", False)
+    PORT: int = config_file.get("PORT", 5000)
+    HOST: str = config_file.get("HOST", "0.0.0.0")
+
+    # check if user wants to use a global server too
+    if config_file["use_global"]:
+
+        # create a cloudflare tunnel
+        create_cloudflare_tunnel(PORT)
+
+# ---------------------------------------- LOGGING CONFIG ---------------------------------------- #
+
+# set logging level
+logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(message)s')
+
+# ---------------------------------------- ROUTES ---------------------------------------- #
+
+# chat generaiton route
+@app.route("/chat/completions", methods=["POST"])
+def chat():
+
+    # get request data
+    data = request.get_json()
+
+    # get messages
+    messages = message_translation(data["messages"]) if config_file["use_addons"] else data["messages"]
+        
+    # get model
+    model = translate(data["model"]) if config_file["use_addons"] else data["model"]
+
+    # get max tokens
+    max_tokens = data.get("max_tokens", 150)
+
+    # top p and top k
+    top_p = data.get("top_p", 0.99)
+    top_k = data.get("top_k", 50)
+
+    # temperature, frequency penalty and presence penalty
+    temperature = data.get("temperature", 0.6)
+
+    # frequency penalty
+    frequency_penalty = data.get("frequency_penalty", 1)
+
+    # presence penalty
+    presence_penalty = data.get("presence_penalty", 1)
+
+    # streaming function. uses text/event-stream instead of application/json
+    def stream():
+
+        # generate chat
+        for chunk in api.chat(messages, 
+                              model, 
+                              stream=True, 
+                              max_tokens=max_tokens, 
+                              top_p=top_p, 
+                              temperature=temperature, 
+                              frequency_penalty=frequency_penalty, 
+                              presence_penalty=presence_penalty,
+                                top_k=top_k
+        ):
+
+            # yield chat
+            #print(chunk)
+            yield chunk + b'\n\n'
+
+        # in the end, return done
+        yield b'data: [DONE]'
+
+    # check if user wants to stream
+    if data.get("stream"):
+
+        # log
+        logging.info(f"Streaming requested for model {model}\n")
+
+        # return stream
+        return app.response_class(stream(), mimetype='text/event-stream')
+    
+    # even if not, stream but collect all data to a full string
+    else:
+
+        # log
+        logging.info(f"Non-streaming requested for model {model}\n")
+
+        # pre-init
+        full: str = ""
+
+        # generate chat
+        for chunk in api.chat(messages, 
+                              model, 
+                              stream=True, 
+                              max_tokens=max_tokens, 
+                              top_p=top_p, 
+                              temperature=temperature, 
+                              frequency_penalty=frequency_penalty, 
+                              presence_penalty=presence_penalty,
+                              top_k=top_k
+        ):
+
+            try:
+
+                # append chunk
+                full += loads(chunk.decode("utf-8").removeprefix('data: '))["choices"][0]["delta"]["content"]
+
+            except: pass
+
+        # return full
+        return jsonify(non_streamed_format(model, full))
+
+
+# route to get all models
+@app.route("/models", methods=["GET"])
+def get_models():
+
+    # return models
+    return jsonify(api.get_models())
+
+# root route to check if api is online
+@app.route("/", methods=["GET"])
+def root():
+    
+        # return root
+        return render_template("index.html")
+
+# ---------------------------------------- ERROR HANDLING ---------------------------------------- #
+@app.errorhandler(403)
+def forbidden(error):
+
+    # return 403
+    return jsonify(
+
+        {"status": False},
+        {'error': [
+
+            {'message': 'Something went wrong, the API was blocked from sending a request to the DeepInfra API. Please try again later.'},
+            {'tpye': 'forbidden'},
+            {'error': f'{error}'}
+        ]},
+        {'hint': 'please report issues on the github page'}
+    ), 403
+
+@app.errorhandler(500)
+def internal_server_error(error):
+    
+        # return 500
+        return jsonify(
+    
+            {"status": False},
+            {'error': [
+    
+                {'message': 'Something went wrong, the API was unable to complete your request. Please try again later.'},
+                {'tpye': 'internal server error'},
+                {'error': f'{error}'}
+            ]},
+            {'hint': 'please report issues on the github page'}
+        ), 500
+
+# ---------------------------------------- START API ---------------------------------------- #
+
+# start the api
+if __name__ == "__main__":
+
+    app.run(debug=DEBUG, port=PORT, host=HOST)
+
+# Path: app.py

requirements.txt

@@ -1,4 +1,4 @@
-shiny==0.10.2
-shinyswatch==0.6.1
-seaborn==0.12.2
-matplotlib==3.7.1
+flask
+flask_cors
+fake_useragent
+requests