![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
]
},
{
"cell_type": "markdown",
"source": [
"# Assignment for Shifkey Labs Gen AI\n",
"\n",
"This project fine tune Flan-T5 Base model from Hugging Face using a lawyer interaction dataset.\n",
"\n",
"The goal is to make the model better at text summarization. The lawyer instruct dataset will be use for a very concise, short summarization.\n",
"\n",
"Link to dataset [here](https://huggingface.co/datasets/Alignment-Lab-AI/Lawyer-Instruct)\n",
"\n",
"Thank you Vansh Sood for this tutorial."
],
"metadata": {
"id": "zNmXJE8xr7JR"
}
},
{
"cell_type": "code",
"source": [
"!pip install datasets"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "vDWWqwd83adt",
"outputId": "3eeb3a14-76f9-4771-f2d3-7541f6a58871"
},
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Collecting datasets\n",
" Downloading datasets-3.0.1-py3-none-any.whl.metadata (20 kB)\n",
"Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from datasets) (3.16.1)\n",
"Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.26.4)\n",
"Requirement already satisfied: pyarrow>=15.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (16.1.0)\n",
"Collecting dill<0.3.9,>=0.3.0 (from datasets)\n",
" Downloading dill-0.3.8-py3-none-any.whl.metadata (10 kB)\n",
"Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (2.2.2)\n",
"Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.32.3)\n",
"Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.5)\n",
"Collecting xxhash (from datasets)\n",
" Downloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)\n",
"Collecting multiprocess (from datasets)\n",
" Downloading multiprocess-0.70.17-py310-none-any.whl.metadata (7.2 kB)\n",
"Requirement already satisfied: fsspec<=2024.6.1,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]<=2024.6.1,>=2023.1.0->datasets) (2024.6.1)\n",
"Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.10.6)\n",
"Requirement already satisfied: huggingface-hub>=0.22.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.24.7)\n",
"Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (24.1)\n",
"Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.2)\n",
"Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (2.4.0)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
"Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (24.2.0)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.1)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.1.0)\n",
"Requirement already satisfied: yarl<2.0,>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.12.1)\n",
"Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n",
"Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.22.0->datasets) (4.12.2)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (3.3.2)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (3.10)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (2.2.3)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (2024.8.30)\n",
"INFO: pip is looking at multiple versions of multiprocess to determine which version is compatible with other requirements. This could take a while.\n",
" Downloading multiprocess-0.70.16-py310-none-any.whl.metadata (7.2 kB)\n",
"Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
"Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.2)\n",
"Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.2)\n",
"Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)\n",
"Downloading datasets-3.0.1-py3-none-any.whl (471 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m471.6/471.6 kB\u001b[0m \u001b[31m19.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hDownloading dill-0.3.8-py3-none-any.whl (116 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.3/116.3 kB\u001b[0m \u001b[31m10.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hDownloading multiprocess-0.70.16-py310-none-any.whl (134 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m11.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hDownloading xxhash-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m17.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hInstalling collected packages: xxhash, dill, multiprocess, datasets\n",
"Successfully installed datasets-3.0.1 dill-0.3.8 multiprocess-0.70.16 xxhash-3.5.0\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"from datasets import load_dataset"
],
"metadata": {
"id": "BgdjVwzM3mS6"
},
"execution_count": 4,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Load Flan T5 model"
],
"metadata": {
"id": "9QVQ_AmB5YCy"
}
},
{
"cell_type": "code",
"source": [
"from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, pipeline\n",
"import torch\n",
"\n",
"# Check if a GPU is available\n",
"import torch\n",
"device = 0 if torch.cuda.is_available() else -1\n",
"\n",
"# Load the Flan-T5 base model for text summarization\n",
"model_name = \"google/flan-t5-base\"\n",
"tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
"model = AutoModelForSeq2SeqLM.from_pretrained(model_name)"
],
"metadata": {
"id": "h-U_tXu64XkY",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 296,
"referenced_widgets": [
"8a8782ebb56e4e379cc79988a53015b6",
"d5eb545fbdcf490a8946bac299fa6216",
"48946e0fb66a464d9459214bf4c68e04",
"562598d36c654e4fa8e721af23896879",
"2e7fe5aef77645bab04f5cb5b716f2ee",
"c937d862bfa147fcab699af2652af2b9",
"14ff1ad18c594015afd08ecbf972feac",
"78a9b0acb010477c8c5d25776f93ba8d",
"8c0244830b7b4251855d091f0f656174",
"f85944f545f547d59c84b33d7a085723",
"fbf7b92107fa4f70a365e97efc956097",
"d24ef25d70414916b156a38c322ecc67",
"12ea787974d24638a9771d9fc645e844",
"0478bf76ffa3410f8ab3af7a96a09798",
"29c1037182aa4da4aa1bbf81ddfaaa13",
"81f7fa6234f14a35b3706c5302cb5e0c",
"3db59d7d3d7d405ba6608de0abb702c4",
"e32572a336954f04b51604a34032a9c4",
"ad36f3568e9645d988f99c2886edbd7a",
"e5a354f6039c461c93d49b9208b1dfa5",
"89dc330f989643018101103347fb6fdb",
"c315a663249048a7a1d9367ce71fc3a7",
"5d5cd59483944c8bbed36fae5b4622ed",
"6449d862b7f24f85bc653ea10f7362c8",
"a64b309fe21345cf8f5ecf00d230374d",
"496f5717df794a10acc0ec8e5e41e177",
"e7ca6d7922e24d28a43c37a4545d5463",
"5af8c885a3c94ea9ab62fa01640f149c",
"4f88f3df452c498e9ea17198315aa8ec",
"b5da91b3eb004c85ad4f8c0ed0e1f38a",
"45ac0212a26d48ecba5c366579c5a71a",
"2443060c146b438ca0f15b552b8f8e00",
"8983badc0df0493d810c8c9474078936",
"33e4a502c3574b888997b3fe642b0447",
"59283afd7bd44bb89a218ad4416a2a5d",
"d914ac7218164e158c65340cd7215787",
"0abb7583a1a74349a42edbad9b21a945",
"81c8202ad7704bb89942759a61f40da6",
"73e06082380d463e8b03de9dc058b3bb",
"ebe38547115e4d9fa293602acd0ac135",
"97c979ef08054668a2806da115d69cfe",
"330fcd32fc8c45e3b4d833157d9708ad",
"e87186c702404afb87941f5f9d2cb462",
"3d46512289ae4b0eb03bc5978eefa8b5",
"07190327d89449448193dc878572a580",
"12efa7f1ce4249b5b9419deff93c8c3f",
"453b9ffb13a7459b9c090eb30a471992",
"798a55ae80cd443e8eb2120928290b42",
"5d7fde15d8234d4f81af0add36bc3ea8",
"07a5bb6f51b34c22aa84553aa718d5a0",
"25cd1fa18d1c4c77b57813273ca1e9f7",
"4a22a51c08a141af81119db700baccd2",
"dffb836461a34136bc99b2d839683bd6",
"4094af3fa69b43f782d5927bde35d2c8",
"42115aff89494f838003b2b1dfba4307",
"4397a1bc490c44a194a1832dcaaae597",
"50d6709b2c4640a9be47cc226d1cbcbb",
"5d85042a70264891aa7fee2f333d02e0",
"14591c3f72ed49aaab091a4c2e7fc14e",
"2beb9b7463394b27a6682e33e623e253",
"164701c29b9e468e93ea7ea852b7b80f",
"26da665f53a240a0978a5681f68ce898",
"6461299f745a4ce1ac6824ccf232e852",
"c37a516e706649a09dfe91d14a8f26ad",
"4e2d0427352245e6a8882f80a305ff81",
"74170ea26a1f423e83414df300bd0e83",
"f40c68f4090d41e2809b4886e20518af",
"77def4f162bd4e7e939b1b07cb4c4c38",
"b01d2aa8625040518a5a68bfc8add258",
"6406ac096b714a5887c2d4cb34f3c289",
"9911828e05114cb8be85f768a1b3f952",
"19bd3d31c97141c7a540f53296cb96fe",
"0303132bf535417b868f106a3be24007",
"0a11c7641b224382bb9dc6c72b08a164",
"40bd38a236594dce9cdab82afb9ad249",
"ce0238cec407408691f59ebbfd6d872b",
"71d399979e674b6b9431f8fbe10adcab"
]
},
"outputId": "f94cfe81-a150-4a30-802f-d9fa6729ea26"
},
"execution_count": 5,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"tokenizer_config.json: 0%| | 0.00/2.54k [00:00"
],
"text/html": [
"\n",
" | Epoch | \n", "Training Loss | \n", "Validation Loss | \n", "
|---|---|---|
| 1 | \n", "No log | \n", "21.553352 | \n", "
| 2 | \n", "No log | \n", "18.918873 | \n", "
| 3 | \n", "No log | \n", "17.439026 | \n", "
| 4 | \n", "No log | \n", "16.559143 | \n", "
| 5 | \n", "No log | \n", "16.236048 | \n", "
"
]
},
"metadata": {}
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"TrainOutput(global_step=55, training_loss=20.093738902698863, metrics={'train_runtime': 358.3321, 'train_samples_per_second': 1.158, 'train_steps_per_second': 0.153, 'total_flos': 284174301265920.0, 'train_loss': 20.093738902698863, 'epoch': 5.0})"
]
},
"metadata": {},
"execution_count": 15
}
]
},
{
"cell_type": "markdown",
"source": [
"# Evaluate"
],
"metadata": {
"id": "O9FOXSXcn-NQ"
}
},
{
"cell_type": "code",
"source": [
"\n",
"# Evaluate the model on the evaluation dataset\n",
"metrics = trainer.evaluate()\n",
"\n",
"# Print the evaluation metrics\n",
"print(metrics)\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 74
},
"id": "N3gC3Dpgn9_q",
"outputId": "5156ee46-ca9f-48ff-a700-8e8952576cd2"
},
"execution_count": 21,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"\n",
" \n",
" \n",
" [116/116 04:28]\n",
" \n",
" "
]
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"{'eval_loss': 16.236047744750977, 'eval_runtime': 53.0288, 'eval_samples_per_second': 17.443, 'eval_steps_per_second': 2.187, 'epoch': 5.0}\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"# Summary Function"
],
"metadata": {
"id": "Fj1gIkoUoUsH"
}
},
{
"cell_type": "code",
"source": [
"\n",
"def summarize(text):\n",
" # Tokenize the input text and move it to the correct device\n",
" inputs = tokenizer(text, return_tensors=\"pt\", max_length=512, truncation=True).to(device)\n",
"\n",
" # Generate the summary using the fine-tuned model\n",
" summary_ids = model.generate(inputs[\"input_ids\"], max_length=128, num_beams=4, early_stopping=True)\n",
"\n",
" # Decode the generated summary back into text and return it\n",
" return tokenizer.decode(summary_ids[0], skip_special_tokens=True)"
],
"metadata": {
"id": "egbdnsPOoWgW"
},
"execution_count": 22,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Test"
],
"metadata": {
"id": "p8Cg3fuooXkk"
}
},
{
"cell_type": "code",
"source": [
"print(summarize(\n",
" \"\"\"\n",
"Person A: Hey, did you hear about the new project management software our company is planning to implement?\n",
"\n",
"Person B: Yeah, I heard a bit about it. What’s the deal with it?\n",
"\n",
"Person A: It’s called \"TaskFlow.\" The management thinks it’s going to streamline our workflow, especially with remote teams. It’s supposed to integrate all the tools we use, like Slack, Trello, and Google Drive, into one platform.\n",
"\n",
"Person B: That sounds interesting. But I’m a bit concerned about the learning curve. Is it user-friendly?\n",
"\n",
"Person A: From what I’ve seen, it looks pretty intuitive. They’re also planning to run a couple of training sessions to get everyone up to speed. The first one is next Monday.\n",
"\n",
"Person B: Okay, that helps. I guess I’ll have to attend that session. How does it compare to what we’re using now?\n",
"\n",
"Person A: It’s supposed to be much more efficient. We’ll be able to track project progress more easily and get real-time updates. Plus, it has built-in analytics to help us with performance tracking.\n",
"\n",
"Person B: That sounds promising. I just hope it doesn’t come with too many bugs at launch.\n",
"\n",
"Person A: Yeah, that’s always a concern with new software. But they’ve been testing it for a while now, so fingers crossed it goes smoothly.\n",
"\n",
"Person B: Let’s hope for the best. Thanks for the info!\n",
"\n",
"Person A: No problem. See you at the training!\n",
"\"\"\"\n",
"))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "199TVy5xoYtW",
"outputId": "386857dd-d19a-4488-8b5e-3de84b84160c"
},
"execution_count": 23,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Person B is excited about the new project management software \"TaskFlow\" that the company is planning to implement.\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"As expected, the model is very concise on the summarization but woud leave some details."
],
"metadata": {
"id": "n_wDFtFQ2rn0"
}
}
]
}