First initial commit

All notebooks + python script + readme + requirements

README.md

@@ -1,3 +1,130 @@
----
-license: apache-2.0
----
+# EssAI: AI-generated essays detector
+
+## Table of Contents
+
+1. [Overview](#overview)
+2. [Features](#features)
+3. [Files](#files)
+4. [Installation](#installation)
+5. [Usage](#usage)
+6. [Model Details](#model-details)
+7. [Dataset](#dataset)
+8. [Fine-tuning](#fine-tuning)
+9. [Results](#results)
+10. [Additional Resources](#additional-resources)
+11. [License](#license)
+12. [Contact](#contact)
+
+## Overview
+
+This project fine-tunes a Large Language Model (LLM) in order to detect AI-generated essays. The model aims to help educators, researchers or individuals identify text that has been generated by AI, ensuring the authenticity of written content.
+
+## Features
+
+- Detects AI-generated essays with very high accuracy (over 95%).
+- Fine-tuned on massive dataset combining ~500K human-written and AI-generated essays.
+
+## Files
+
+### `requirements.txt`
+This file lists all the Python packages required to run the project. It ensures that all necessary dependencies are installed for the project to function correctly.
+
+### `essai_user_input.py`
+This script is responsible for handling user inputs. Just copy in your essay and run it to get the prediction.
+
+### `training.py`
+This script has handled the training process of the model. It includes code for loading the dataset, fine-tuning it and saving the trained model.
+
+### `testing.py`
+This script is used to evaluate the performance of the trained model. It loads the test dataset, performs predictions, and calculates performance metrics such as accuracy and F1-score.
+
+### `data_insights.py`
+This script generates insights and visualizations from the data used in this project. It includes functions for analyzing dataset statistics, plotting graphs, and summarizing key data points to help understand the dataset better.
+
+## Installation
+
+To install the required dependencies, clone the repository and install the necessary Python packages in the **requirements.txt** file:
+
+```bash
+git clone https://github.com/diegovelilla/EssAI
+cd EssAI
+pip install -r requirements.txt
+```
+
+## Usage
+
+You can use the model to check your own essays by running the **essai_user_input.py** file and coping the text into the input part right after the imports:
+
+```python
+# --- INPUT ---
+
+input_list = [""" WRITE HERE YOUR FIRST ESSAY """,
+              """ WRITE HERE YOUR SECOND ESSAY """]
+
+# -------------
+```
+As you can see, you can check more than one essay at a time. This model has been trained with 350-400 word long essays, so just keep that in mind when using it. Learn more about the data used in the [data_insights](https://github.com/diegovelilla/EssAI/blob/main/essai_data_insights.ipynb) notebook.
+
+## Model details
+The base model selected for this project was the [bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) model from hugging face. BERT (Bidirectional Encoder Representations from Transformers) is a transformer model developed and published in 2018 by Google's AI Reasearch Team. This is an open-source model with 110M parameters pretrained on a large corpus of English written data with the objectives of:
+
+- Predicting missing words in a sentence.
+- Guessing if two sentences were next to each other in the original text.
+
+Which makes it a really competent text classification model and a great candidate for our project.
+
+## Dataset
+The dataset used was taken from Kaggle and can be found [here](https://www.kaggle.com/datasets/shanegerami/ai-vs-human-text). It contains about 500K different essays with around 60% being human written and the 40% left AI-generated. For further data info, check out the [data_insights](https://github.com/diegovelilla/EssAI/blob/main/essai_data_insights.ipynb) notebook. Also check out the [training](https://github.com/diegovelilla/EssAI/blob/main/essai_training.ipynb) and [testing](https://github.com/diegovelilla/EssAI/blob/main/essai_testing.ipynb) notebooks if interested in how the model was fine-tuned or want to check the model's performance (instructions inside).
+
+## Fine-tuning
+For resource issues and since this was intended as a learning project, only 1% from de full 500K dataset has been used which would still mean a training dataset of 4.000 essays and a testing dataset of 1.000 essays. 
+
+I encourage anyone reading this to try to further train this model increasing the data used with the [training](https://github.com/diegovelilla/EssAI/blob/main/essai_training.ipynb) notebook.
+
+## Results
+For the first 1.000 datasets tested, the model showed a 98% accuracy. For the second one, and with a testing sample of 20.000 essays, the accuracy shown was 97%.
+Further testing can be done using the [testing](https://github.com/diegovelilla/EssAI/blob/main/essai_testing.ipynb) notebook
+
+In the initial testing phase with a sample of 1.000 essays, the model demonstrated an impressive accuracy of 98%. In a subsequent, more extensive test involving 20.000 essays, the model maintained a high accuracy of 97%. 
+
+For more detailed evaluation and further testing, please refer to the [testing](https://github.com/diegovelilla/EssAI/blob/main/essai_testing.ipynb) notebook.
+
+## Additional Resources
+
+Throughout the development, I've found some resources very useful that I would like to share apart from others related to the project.
+
+### Tutorials and Documentation
+
+- **[Hugging Face NLP Course](huggingface.co/learn/nlp-course/)**: Comprehensive tutorials and documentation on what is NLP and how to use Hugging Face's libraries.
+- **[Hugging Face Transformers Documentation](https://huggingface.co/transformers/)**: The official documentation for the Transformers library.
+
+### Articles and Papers
+
+- **[BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805)**: The original research paper on BERT, which provided insights into the architecture and capabilities of the model.
+- **[A Comprehensive Guide to Fine-Tuning BERT](https://towardsdatascience.com/a-comprehensive-guide-to-fine-tuning-bert-for-nlp-tasks-39ef4a51c7d3)**: An article that outlines various techniques for fine-tuning BERT models for specific tasks.
+
+### Tools and Libraries
+
+- **[Kaggle Datasets](https://www.kaggle.com/datasets)**: Platform used to source the dataset for this project.
+- **[Git Large File Storage (LFS)](https://git-lfs.github.com/)**: Tool used for managing large files in the Git repository. Very useful for moving big files like the ones that form the model.
+
+### YouTube channels
+
+- **[Andrej Karpathy](https://www.youtube.com/@AndrejKarpathy)**: One of my favourite ML/DL YouTube channels with amazing videos. Can't stress enough how much I have learned from this man.
+- **[DotCSV](https://www.youtube.com/@DotCSV)**: The first AI related YouTube channel I did ever follow. Great spanish speaking channel to keep up with AI news.
+
+These resources provided valuable information and tools throughout the project's development. If you’re working on similar projects, they might be helpful to you as well.
+
+## License
+This project is licensed under the **Apache 2.0 License**. See the [LICENSE](https://github.com/diegovelilla/EssAI/blob/main/LICENSE) file for more details.
+
+## Contact
+
+For any questions or feedback please reach out to:
+
+- **Email**: [diegovelillarecio@gmail.com](mailto:diegovelillarecio@gmail.com)
+- **GitHub Profile**: [diegovelilla](https://github.com/diegovelilla)
+- **Hugging Face Profile**: [diegovelilla](https://huggingface.co/diegovelilla)
+- **LinkedIn**: [Diego Velilla Recio](https://www.linkedin.com/in/diego-velilla-recio/)
+
+Feel free to open an issue on GitHub or contact me in any way if you have any queries or suggestions.

essai_testing.ipynb

@@ -0,0 +1,280 @@
+{
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python",
+      "version": "3.10.13",
+      "mimetype": "text/x-python",
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "pygments_lexer": "ipython3",
+      "nbconvert_exporter": "python",
+      "file_extension": ".py"
+    },
+    "kaggle": {
+      "accelerator": "none",
+      "dataSources": [],
+      "dockerImageVersionId": 30746,
+      "isInternetEnabled": false,
+      "language": "python",
+      "sourceType": "notebook",
+      "isGpuEnabled": false
+    }
+  },
+  "nbformat_minor": 0,
+  "nbformat": 4,
+  "cells": [
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- INSTALLATION ---\n",
+        "\n",
+        "!pip install pandas numpy matplotlib nltk scikit-learn transformers datasets torch\n",
+        "!kaggle datasets download -d shanegerami/ai-vs-human-text\n",
+        "!unzip -n ai-vs-human-text.zip\n",
+        "!rm ai-vs-human-text.zip\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "XKWBDF8lir6o",
+        "execution": {
+          "iopub.status.busy": "2024-08-14T18:13:18.903225Z",
+          "iopub.execute_input": "2024-08-14T18:13:18.903635Z",
+          "iopub.status.idle": "2024-08-14T18:14:34.119173Z",
+          "shell.execute_reply.started": "2024-08-14T18:13:18.903599Z",
+          "shell.execute_reply": "2024-08-14T18:14:34.117649Z"
+        },
+        "trusted": true
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- IMPORTS ---\n",
+        "\n",
+        "import pandas as pd\n",
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt\n",
+        "import re\n",
+        "import nltk\n",
+        "from nltk.corpus import stopwords\n",
+        "nltk.download('stopwords')\n",
+        "stopwords = set(stopwords.words('english'))\n",
+        "from sklearn.model_selection import train_test_split\n",
+        "from sklearn.metrics import accuracy_score, precision_recall_fscore_support\n",
+        "from transformers import AutoTokenizer, AutoModelForSequenceClassification\n",
+        "from transformers import Trainer, TrainingArguments, DataCollatorWithPadding\n",
+        "from datasets import Dataset\n",
+        "import torch\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "q9TGKRUIiPMy"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- USEFUL FUNCTIONS ----\n",
+        "\n",
+        "def clean_text(text):\n",
+        "    \"\"\"\n",
+        "    This funtion get's rid of nonalphabetical characters, stopwords and lower cases the text.\n",
+        "\n",
+        "    Args:\n",
+        "    text (str): The text to be cleaned\n",
+        "\n",
+        "    Returns:\n",
+        "    text (str): The cleaned text\n",
+        "\n",
+        "    Example:\n",
+        "    df['text'] = df['text'].apply(clean_text)\n",
+        "    \"\"\"\n",
+        "    text = re.sub(r'[^a-zA-Z]', ' ', text)\n",
+        "    text = text.lower()\n",
+        "    words = text.split()\n",
+        "    text = [word for word in words if not word in stopwords]\n",
+        "    text = ' '.join(words)\n",
+        "    return text\n",
+        "\n",
+        "def tokenize_function(dataframe):\n",
+        "    \"\"\"\n",
+        "    This funtion tokenizes the 'text' field of the dataframe.\n",
+        "\n",
+        "    Args:\n",
+        "    dataframe (pandas.DataFrame): The dataframe to be tokenized\n",
+        "\n",
+        "    Returns:\n",
+        "    dataframe (pandas.DataFrame): The tokenized dataframe\n",
+        "\n",
+        "    Example and output:\n",
+        "    train_dataset_token = train_dataset.map(tokenize_function, batched=True)\n",
+        "    \"\"\"\n",
+        "    return tokenizer(dataframe[\"text\"], truncation=True)\n",
+        "\n",
+        "def compute_metrics(eval_pred):\n",
+        "    \"\"\"\n",
+        "    This funtion computes the accuracy, precision, recall and f1 score of the model.\n",
+        "\n",
+        "    It'is passed to the trainer and it outputs when evaluating the model.\n",
+        "\n",
+        "    Args:\n",
+        "    eval_pred (tuple): The predictions and labels of the model\n",
+        "\n",
+        "    Returns:\n",
+        "    dict: The accuracy, precision, recall and f1 score of the model\n",
+        "\n",
+        "    Example:\n",
+        "    >>> trainer.evaluate()\n",
+        "    {\n",
+        "        'accuracy': accuracy,\n",
+        "        'precision': precision,\n",
+        "        'recall': recall,\n",
+        "        'f1': f1\n",
+        "    }\n",
+        "    \"\"\"\n",
+        "    predictions, labels = eval_pred\n",
+        "    predictions = predictions.argmax(axis=-1)\n",
+        "    accuracy = accuracy_score(labels, predictions)\n",
+        "    precision, recall, f1, _ = precision_recall_fscore_support(labels, predictions, average='binary')\n",
+        "    return {\n",
+        "        'accuracy': accuracy,\n",
+        "        'precision': precision,\n",
+        "        'recall': recall,\n",
+        "        'f1': f1\n",
+        "    }\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "JtYsc4hJAnk3"
+      },
+      "execution_count": 41,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- LOADING THE MODEL ---\n",
+        "\n",
+        "# Load the initial tokenizer and model to set the number of labels its going to classify as 2\n",
+        "checkpoint = \"diegovelilla/EssAI\"\n",
+        "tokenizer = AutoTokenizer.from_pretrained(checkpoint)\n",
+        "model = AutoModelForSequenceClassification.from_pretrained(checkpoint)\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "P87A1UTgJIia"
+      },
+      "execution_count": 42,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- DATA PREPROCESSING ---\n",
+        "\n",
+        "df = pd.read_csv('AI_Human.csv')\n",
+        "\n",
+        "# Separate human from ai\n",
+        "df_human = df[df[\"generated\"] == 0]\n",
+        "df_ai = df[df[\"generated\"] == 1]\n",
+        "\n",
+        "# We take as many human written esssays as AI generate since the dataset is a bit unbalanced\n",
+        "df_ai_len = df_ai[\"text\"].count()\n",
+        "df_human = df_human.sample(n=df_ai_len)\n",
+        "\n",
+        "# We concatenate both dataframes, shuffle them and then we take 1% of them since those will be enough to fine tune the model\n",
+        "# and with my current resources I won't be able to process more. For better results increase the fraction of the data used.\n",
+        "df_unshuffled = pd.concat([df_human, df_ai])\n",
+        "df = df_unshuffled.sample(frac=0.01).reset_index(drop=True)\n",
+        "\n",
+        "# Get rid of nonalphatetical characters, stopwords and we lower case it.\n",
+        "df['text'] = df['text'].apply(clean_text)\n",
+        "\n",
+        "# According to the transformers library of hugging face the targets column name should be labels and ints\n",
+        "df = df.rename(columns={'generated': 'labels'})\n",
+        "df['labels'] = df['labels'].astype(int)\n",
+        "\n",
+        "# We convert the pandas dataframe into a hugging face dataset and tokenize both of them\n",
+        "ds = Dataset.from_pandas(df)\n",
+        "ds_token = ds.map(tokenize_function, batched=True)\n",
+        "\n",
+        "# Drop columns that are not necessary and set the dataset format to pytorch tensors\n",
+        "ds_token = ds_token.remove_columns([\"text\", \"token_type_ids\"])\n",
+        "ds_token.set_format(type='torch', columns=['input_ids', 'attention_mask', 'labels'])\n",
+        "\n",
+        "# -------------------------\n"
+      ],
+      "metadata": {
+        "id": "rYaHbUCDG7tf"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- INSTANTIATING TRAINER ----\n",
+        "\n",
+        "# We instantiate a DataCollatorWithPadding in order to pad the inputs in batches while training\n",
+        "data_collator = DataCollatorWithPadding(tokenizer=tokenizer)\n",
+        "\n",
+        "# Create the training arguments\n",
+        "training_args = TrainingArguments(\"./results\")\n",
+        "\n",
+        "# Create the trainer\n",
+        "trainer = Trainer(\n",
+        "    model,\n",
+        "    training_args,\n",
+        "    eval_dataset=ds_token,\n",
+        "    data_collator=data_collator,\n",
+        "    tokenizer=tokenizer,\n",
+        "    compute_metrics = compute_metrics\n",
+        ")\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "Golh92ee33aA"
+      },
+      "execution_count": 50,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- EVALUATION ---\n",
+        "\n",
+        "evaluation_results = trainer.evaluate()\n",
+        "\n",
+        "print(\"Accuracy:\", evaluation_results['eval_accuracy'])\n",
+        "print(\"Precision:\", evaluation_results['eval_precision'])\n",
+        "print(\"Recall:\", evaluation_results['eval_recall'])\n",
+        "print(\"F1:\", evaluation_results['eval_f1'])\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "WkQgrxgFPkpJ"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

essai_training.ipynb

@@ -0,0 +1,333 @@
+{
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python",
+      "version": "3.10.13",
+      "mimetype": "text/x-python",
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "pygments_lexer": "ipython3",
+      "nbconvert_exporter": "python",
+      "file_extension": ".py"
+    },
+    "kaggle": {
+      "accelerator": "none",
+      "dataSources": [],
+      "dockerImageVersionId": 30746,
+      "isInternetEnabled": false,
+      "language": "python",
+      "sourceType": "notebook",
+      "isGpuEnabled": false
+    }
+  },
+  "nbformat_minor": 0,
+  "nbformat": 4,
+  "cells": [
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- INSTALLATION ---\n",
+        "\n",
+        "!pip install pandas numpy matplotlib nltk scikit-learn transformers datasets torch\n",
+        "!kaggle datasets download -d shanegerami/ai-vs-human-text\n",
+        "!unzip -n ai-vs-human-text.zip\n",
+        "!rm ai-vs-human-text.zip\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "XKWBDF8lir6o",
+        "execution": {
+          "iopub.status.busy": "2024-08-14T18:13:18.903225Z",
+          "iopub.execute_input": "2024-08-14T18:13:18.903635Z",
+          "iopub.status.idle": "2024-08-14T18:14:34.119173Z",
+          "shell.execute_reply.started": "2024-08-14T18:13:18.903599Z",
+          "shell.execute_reply": "2024-08-14T18:14:34.117649Z"
+        },
+        "trusted": true
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- IMPORTS ---\n",
+        "\n",
+        "import pandas as pd\n",
+        "import numpy as np\n",
+        "import matplotlib.pyplot as plt\n",
+        "import re\n",
+        "import nltk\n",
+        "from nltk.corpus import stopwords\n",
+        "nltk.download('stopwords')\n",
+        "stopwords = set(stopwords.words('english'))\n",
+        "from sklearn.model_selection import train_test_split\n",
+        "from sklearn.metrics import accuracy_score, precision_recall_fscore_support\n",
+        "from transformers import AutoTokenizer, AutoModelForSequenceClassification\n",
+        "from transformers import Trainer, TrainingArguments, DataCollatorWithPadding\n",
+        "from datasets import Dataset\n",
+        "import torch\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "q9TGKRUIiPMy"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- USEFUL FUNCTIONS ----\n",
+        "\n",
+        "def clean_text(text):\n",
+        "    \"\"\"\n",
+        "    This funtion get's rid of nonalphabetical characters, stopwords and lower cases the text.\n",
+        "\n",
+        "    Args:\n",
+        "    text (str): The text to be cleaned\n",
+        "\n",
+        "    Returns:\n",
+        "    text (str): The cleaned text\n",
+        "\n",
+        "    Example:\n",
+        "    df['text'] = df['text'].apply(clean_text)\n",
+        "    \"\"\"\n",
+        "    text = re.sub(r'[^a-zA-Z]', ' ', text)\n",
+        "    text = text.lower()\n",
+        "    words = text.split()\n",
+        "    text = [word for word in words if not word in stopwords]\n",
+        "    text = ' '.join(words)\n",
+        "    return text\n",
+        "\n",
+        "def tokenize_function(dataframe):\n",
+        "    \"\"\"\n",
+        "    This funtion tokenizes the 'text' field of the dataframe.\n",
+        "\n",
+        "    Args:\n",
+        "    dataframe (pandas.DataFrame): The dataframe to be tokenized\n",
+        "\n",
+        "    Returns:\n",
+        "    dataframe (pandas.DataFrame): The tokenized dataframe\n",
+        "\n",
+        "    Example and output:\n",
+        "    train_dataset_token = train_dataset.map(tokenize_function, batched=True)\n",
+        "    \"\"\"\n",
+        "    return tokenizer(dataframe[\"text\"], truncation=True)\n",
+        "\n",
+        "def compute_metrics(eval_pred):\n",
+        "    \"\"\"\n",
+        "    This funtion computes the accuracy, precision, recall and f1 score of the model.\n",
+        "\n",
+        "    It'is passed to the trainer and it outputs when evaluating the model.\n",
+        "\n",
+        "    Args:\n",
+        "    eval_pred (tuple): The predictions and labels of the model\n",
+        "\n",
+        "    Returns:\n",
+        "    dict: The accuracy, precision, recall and f1 score of the model\n",
+        "\n",
+        "    Example:\n",
+        "    >>> trainer.evaluate()\n",
+        "    {\n",
+        "        'accuracy': accuracy,\n",
+        "        'precision': precision,\n",
+        "        'recall': recall,\n",
+        "        'f1': f1\n",
+        "    }\n",
+        "    \"\"\"\n",
+        "    predictions, labels = eval_pred\n",
+        "    predictions = predictions.argmax(axis=-1)\n",
+        "    accuracy = accuracy_score(labels, predictions)\n",
+        "    precision, recall, f1, _ = precision_recall_fscore_support(labels, predictions, average='binary')\n",
+        "    return {\n",
+        "        'accuracy': accuracy,\n",
+        "        'precision': precision,\n",
+        "        'recall': recall,\n",
+        "        'f1': f1\n",
+        "    }\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "JtYsc4hJAnk3"
+      },
+      "execution_count": 3,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- INSTANTIATING THE MODEL ---\n",
+        "\n",
+        "# Load the initial tokenizer and model to set the number of labels its going to classify as 2\n",
+        "checkpoint = \"bert-base-uncased\"\n",
+        "tokenizer = AutoTokenizer.from_pretrained(checkpoint)\n",
+        "model = AutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)\n",
+        "\n",
+        "# -------------------------------"
+      ],
+      "metadata": {
+        "id": "Golh92ee33aA"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- DATA PREPROCESSING ---\n",
+        "\n",
+        "df = pd.read_csv('AI_Human.csv')\n",
+        "\n",
+        "# Separate human from ai\n",
+        "df_human = df[df[\"generated\"] == 0]\n",
+        "df_ai = df[df[\"generated\"] == 1]\n",
+        "\n",
+        "# We take as many human written esssays as AI generate since the dataset is a bit unbalanced\n",
+        "df_ai_len = df_ai[\"text\"].count()\n",
+        "df_human = df_human.sample(n=df_ai_len)\n",
+        "\n",
+        "# We concatenate both dataframes, shuffle them and then we take 1% of them since those will be enough to fine tune the model\n",
+        "# and with my current resources I won't be able to process more. For better results increase the fraction of the data used.\n",
+        "df_unshuffled = pd.concat([df_human, df_ai])\n",
+        "df = df_unshuffled.sample(frac=0.01).reset_index(drop=True)\n",
+        "\n",
+        "# Get rid of nonalphatetical characters, stopwords and we lower case it.\n",
+        "df['text'] = df['text'].apply(clean_text)\n",
+        "\n",
+        "# Split in train/test (I used 80%/20%)\n",
+        "df_train, df_test = train_test_split(df, test_size=0.2)\n",
+        "\n",
+        "# According to the transformers library of hugging face the targets column name should be labels and ints\n",
+        "df_train = df_train.rename(columns={'generated': 'labels'})\n",
+        "df_test = df_test.rename(columns={'generated': 'labels'})\n",
+        "df_train['labels'] = df_train['labels'].astype(int)\n",
+        "df_test['labels'] = df_test['labels'].astype(int)\n",
+        "\n",
+        "# We convert the pandas dataframe into hugging face datasets and tokenize both of them\n",
+        "train_dataset = Dataset.from_pandas(df_train)\n",
+        "test_dataset = Dataset.from_pandas(df_test)\n",
+        "train_dataset_token = train_dataset.map(tokenize_function, batched=True)\n",
+        "test_dataset_token = test_dataset.map(tokenize_function, batched=True)\n",
+        "\n",
+        "# Drop columns that are not necessary and set the dataset format to pytorch tensors\n",
+        "train_dataset_token = train_dataset_token.remove_columns([\"text\", \"__index_level_0__\", \"token_type_ids\"])\n",
+        "test_dataset_token = test_dataset_token.remove_columns([\"text\", \"__index_level_0__\", \"token_type_ids\"])\n",
+        "train_dataset_token.set_format(type='torch', columns=['input_ids', 'attention_mask', 'labels'])\n",
+        "test_dataset_token.set_format(type='torch', columns=['input_ids', 'attention_mask', 'labels'])\n",
+        "\n",
+        "# -------------------------\n"
+      ],
+      "metadata": {
+        "id": "GUNv7d5lkg2z"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- INSTANTIATING TRAINER ---\n",
+        "\n",
+        "# We instantiate a DataCollatorWithPadding in order to pad the inputs in batches while training\n",
+        "data_collator = DataCollatorWithPadding(tokenizer=tokenizer)\n",
+        "\n",
+        "# Create the training arguments\n",
+        "training_args = TrainingArguments(\n",
+        "    output_dir=\"./results\",\n",
+        "    per_device_train_batch_size=16,  # Adjust based on GPU memory\n",
+        "    per_device_eval_batch_size=16,\n",
+        "    num_train_epochs=3,\n",
+        "    weight_decay=0.01,\n",
+        "    logging_dir=\"./logs\",\n",
+        "    logging_steps=100,\n",
+        ")\n",
+        "\n",
+        "# Create the trainer\n",
+        "trainer = Trainer(\n",
+        "    model,\n",
+        "    training_args,\n",
+        "    train_dataset=train_dataset_token,\n",
+        "    eval_dataset=test_dataset_token,\n",
+        "    data_collator=data_collator,\n",
+        "    tokenizer=tokenizer,\n",
+        "    compute_metrics = compute_metrics\n",
+        ")\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "FhqLhZv5HFot"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- TRAINING ---\n",
+        "\n",
+        "trainer.train()\n",
+        "\n",
+        "# ----------------"
+      ],
+      "metadata": {
+        "id": "T65B4LitLfsN"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- EVALUATION ---\n",
+        "\n",
+        "evaluation_results = trainer.evaluate()\n",
+        "\n",
+        "print(\"Accuracy:\", evaluation_results['eval_accuracy'])\n",
+        "print(\"Precision:\", evaluation_results['eval_precision'])\n",
+        "print(\"Recall:\", evaluation_results['eval_recall'])\n",
+        "print(\"F1:\", evaluation_results['eval_f1'])\n",
+        "\n",
+        "# -------------------------"
+      ],
+      "metadata": {
+        "id": "WkQgrxgFPkpJ"
+      },
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "# --- EXPORTING THE MODEL (optional) ---\n",
+        "\n",
+        "# Save the model and tokenizer\n",
+        "#model.save_pretrained(\"./AI-Detector-Model/Model\")\n",
+        "#tokenizer.save_pretrained(\"./AI-Detector-Model/Tokenizer\")\n",
+        "\n",
+        "# Zip the model\n",
+        "#!zip -r AI-Detector-Model.zip AI-Detector-Model\n",
+        "\n",
+        "# --------------------------"
+      ],
+      "metadata": {
+        "id": "DF-ZWbjHSxuE"
+      },
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

essai_user_input.py

@@ -0,0 +1,166 @@
+# --- IMPORTS ---
+
+import torch
+from datasets import Dataset
+from transformers import Trainer, TrainingArguments, DataCollatorWithPadding
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+from sklearn.metrics import accuracy_score, precision_recall_fscore_support
+from sklearn.model_selection import train_test_split
+import pandas as pd
+import numpy as np
+import matplotlib.pyplot as plt
+import re
+import nltk
+from nltk.corpus import stopwords
+nltk.download('stopwords')
+stopwords = set(stopwords.words('english'))
+
+# -------------------------
+
+# --- INPUT ---
+
+input_list = [""" WRITE HERE YOUR FIRST ESSAY """,
+              """ WRITE HERE YOUR SECOND ESSAY """]
+
+# -------------
+
+# --- USEFUL FUNCTIONS ----
+
+
+def clean_text(text):
+    """
+    This funtion get's rid of nonalphabetical characters, stopwords and lower cases the text.
+
+    Args:
+    text (str): The text to be cleaned
+
+    Returns:
+    text (str): The cleaned text
+
+    Example:
+    df['text'] = df['text'].apply(clean_text)
+    """
+    text = re.sub(r'[^a-zA-Z]', ' ', text)
+    text = text.lower()
+    words = text.split()
+    text = [word for word in words if not word in stopwords]
+    text = ' '.join(words)
+    return text
+
+
+def tokenize_function(dataframe):
+    """
+    This funtion tokenizes the 'text' field of the dataframe.
+
+    Args:
+    dataframe (pandas.DataFrame): The dataframe to be tokenized
+
+    Returns:
+    dataframe (pandas.DataFrame): The tokenized dataframe
+
+    Example and output:
+    train_dataset_token = train_dataset.map(tokenize_function, batched=True)
+    """
+    return tokenizer(dataframe["text"], truncation=True)
+
+
+def compute_metrics(eval_pred):
+    """
+    This funtion computes the accuracy, precision, recall and f1 score of the model.
+
+    It'is passed to the trainer and it outputs when evaluating the model.
+
+    Args:
+    eval_pred (tuple): The predictions and labels of the model
+
+    Returns:
+    dict: The accuracy, precision, recall and f1 score of the model
+
+    Example:
+    >>> trainer.evaluate()
+    {
+        'accuracy': accuracy,
+        'precision': precision,
+        'recall': recall,
+        'f1': f1
+    }
+    """
+    predictions, labels = eval_pred
+    predictions = predictions.argmax(axis=-1)
+    accuracy = accuracy_score(labels, predictions)
+    precision, recall, f1, _ = precision_recall_fscore_support(
+        labels, predictions, average='binary')
+    return {
+        'accuracy': accuracy,
+        'precision': precision,
+        'recall': recall,
+        'f1': f1
+    }
+
+# -------------------------
+
+# --- LOADING THE MODEL ---
+
+
+# Load the initial tokenizer and model to set the number of labels its going to classify as 2
+checkpoint = "diegovelilla/EssAI"
+tokenizer = AutoTokenizer.from_pretrained(checkpoint)
+model = AutoModelForSequenceClassification.from_pretrained(checkpoint)
+
+# -------------------------
+
+# --- DATA PREPROCESSING ---
+
+n_input = len(input_list)
+
+# Now we convert the input to a dataset
+df = pd.DataFrame({'text': input_list})
+
+
+# Get rid of nonalphatetical characters, stopwords and we lower case it.
+df['text'] = df['text'].apply(clean_text)
+
+# We convert the pandas dataframe into hugging face datasets and tokenize both of them
+ds = Dataset.from_pandas(df)
+ds_token = ds.map(tokenize_function, batched=True)
+
+# Drop columns that are not necessary and set the dataset format to pytorch tensors
+ds_token = ds_token.remove_columns(["text", "token_type_ids"])
+ds_token.set_format(type='torch', columns=['input_ids', 'attention_mask'])
+
+# -------------------------
+
+# --- INSTANTIATING TRAINER ----
+
+# We instantiate a DataCollatorWithPadding in order to pad the inputs in batches while training
+data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
+
+# Create the training arguments
+training_args = TrainingArguments(".")
+
+# Create the trainer
+trainer = Trainer(
+    model,
+    training_args,
+    eval_dataset=ds_token,
+    data_collator=data_collator,
+    tokenizer=tokenizer,
+    compute_metrics=compute_metrics
+)
+
+# -------------------------
+
+# --- PREDICT ---
+
+# We predict and then format the output
+predictions = trainer.predict(ds_token)
+predictions = torch.from_numpy(predictions.predictions)
+predictions = torch.nn.functional.softmax(predictions, dim=-1)
+
+print('\n\n')
+for i in range(n_input):
+    index = torch.argmax(predictions[i])
+    print(f'{i+1}: HUMAN {round(predictions[i][0].item() * 100, 2)}% of confidence.') if index == 0 else print(
+        f'{i+1}: AI {round(predictions[i][1].item() * 100, 2)}% of confidence.')
+
+# -------------------------

requirements.txt

@@ -0,0 +1,10 @@
+pandas
+numpy
+matplotlib
+nltk
+scikit-learn
+transformers[pytorch]
+accelerate>=0.21.0
+datasets
+torch
+seaborn