notebooks/TermProject.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "695a0d97-6b1d-4448-b16c-3ba95bb60c30",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# If these install, restart the kernal so it has access to the recently installed files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22de09b1-3c40-4195-8b5a-926d5391313f",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip3 install --user transformers datasets evaluate rouge_score\n",
    "!pip3 install --user datasets\n",
    "!pip install --user transformers[torch]\n",
    "!pip install --user accelerate -U\n",
    "!pip install --user wandb"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1687d09-6b14-47fb-9819-dbed0d8ae3b3",
   "metadata": {},
   "source": [
    "# Logging into huggingface hub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f2f10d5e-9655-4554-abb0-511044640854",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bc57319ea4544e0cbe8fd4f2c10ee2e9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from huggingface_hub import notebook_login\n",
    "\n",
    "notebook_login()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05b9e8c7-b52d-4a1f-8464-fb5f7e78e579",
   "metadata": {},
   "source": [
    "# Naming the WANDB project and notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6b4bad30-76c7-49dc-b563-a27ed9465eee",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m WANDB_NOTEBOOK_NAME should be a path to a notebook file, couldn't find ./TermProject.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mtah6k\u001b[0m (\u001b[33mcheaptrix\u001b[0m). Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "Tracking run with wandb version 0.16.6"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Run data is saved locally in <code>/scratch/user/u.th161689/AI_NLP/TermProject/wandb/run-20240412_232000-ls1byqtm</code>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Syncing run <strong><a href='https://wandb.ai/cheaptrix/congress_bill_sumamry_model/runs/ls1byqtm' target=\"_blank\">youthful-fire-1</a></strong> to <a href='https://wandb.ai/cheaptrix/congress_bill_sumamry_model' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View project at <a href='https://wandb.ai/cheaptrix/congress_bill_sumamry_model' target=\"_blank\">https://wandb.ai/cheaptrix/congress_bill_sumamry_model</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View run at <a href='https://wandb.ai/cheaptrix/congress_bill_sumamry_model/runs/ls1byqtm' target=\"_blank\">https://wandb.ai/cheaptrix/congress_bill_sumamry_model/runs/ls1byqtm</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<button onClick=\"this.nextSibling.style.display='block';this.style.display='none';\">Display W&B run</button><iframe src='https://wandb.ai/cheaptrix/congress_bill_sumamry_model/runs/ls1byqtm?jupyter=true' style='border:none;width:100%;height:420px;display:none;'></iframe>"
      ],
      "text/plain": [
       "<wandb.sdk.wandb_run.Run at 0x14deaee5bb90>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Must set the global variable WANDB_NOTEBOOK_NAME to the name of the notebook\n",
    "import os\n",
    "import wandb\n",
    "# Name is found through the os incase the name changes\n",
    "# WANDB_NOTEBOOK_NAME = os.path.basename(os.getcwd())\n",
    "notebook_name = str(os.path.basename(os.getcwd()))\n",
    "os.environ[\"WANDB_NOTEBOOK_NAME\"] = \"./\" + notebook_name\n",
    "os.environ[\"WANDB_PROJECT\"] = \"congress_bill_summary_model\"\n",
    "wandb.init(project='congress_bill_sumamry_model') "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69fbba70-2fb2-47c9-abc9-1824a3ab10d7",
   "metadata": {},
   "source": [
    "# Importing and loading the Congressional Bills dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fd3b62f1-5bd2-44ff-87f3-1d48ccb74459",
   "metadata": {},
   "outputs": [],
   "source": [
    "import datasets\n",
    "from datasets import load_dataset\n",
    "import pandas as pd\n",
    "from datasets import Dataset\n",
    "import json\n",
    "\n",
    "# Opening JSON file\n",
    "with open(\"cleaned_bill_sum_data.json\") as json_file:\n",
    "    cleaned_data = json.load(json_file)\n",
    "data = pd.DataFrame.from_dict(cleaned_data)\n",
    "dataset = Dataset.from_pandas(pd.DataFrame(data=data))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c87b4f7-bdbd-4869-8975-609373000ab0",
   "metadata": {},
   "source": [
    "# Splitting the data to training and testing split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9f08c14e-820e-497d-a979-6332bce67bc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "billsum = dataset.train_test_split(test_size=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9949be74-ed88-49a8-8465-8290af7abe6f",
   "metadata": {},
   "source": [
    "# Example of data within the training set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2f7551a5-631d-4de3-afb7-99d923d18ccd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'summary': '(This measure has not been amended since it was introduced. The summary of that version is repeated here.) Declares that when the House adjourns on any legislative day from Friday, July 15, 2016, through Friday, September 2, 2016, it stand adjourned until 2 p.m. on Tuesday, September 6, 2016.',\n",
       " 'text': 'A concurrent resolution providing for an adjournment of the House of Representatives. 2016-07-14 Passed Senate without amendment (This measure has not been amended since it was introduced. The summary of that version is repeated here.) Declares that when the House adjourns on any legislative day from Friday, July 15, 2016, through Friday, September 2, 2016, it stand adjourned until 2 p.m. on Tuesday, September 6, 2016. 2016-07-14 Introduced in Senate Declares that when the House adjourns on any legislative day from Friday, July 15, 2016, through Friday, September 2, 2016, it stand adjourned until 2 p.m. on Tuesday, September 6, 2016. text/xml EN Pursuant to Title 17 Section 105 of the United States Code, this file is not subject to copyright protection and is in the public domain. Congressional Research Service, Library of Congress This file contains bill summaries for federal legislation. A bill summary describes the most significant provisions of a piece of legislation and details the effects the legislative text may have on current law and federal programs. Bill summaries are authored by the Congressional Research Service (CRS) of the Library of Congress. As stated in Public Law 91-510 (2 USC 166 (d)(6)), one of the duties of CRS is \"to prepare summaries and digests of bills and resolutions of a public general nature introduced in the Senate or House of Representatives\". For more information, refer to the User Guide that accompanies this file.',\n",
       " 'title': 'A concurrent resolution providing for an adjournment of the House of Representatives.'}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "billsum[\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c2e2c67-3d0b-4d4b-b3b6-e6d9afb60c8a",
   "metadata": {},
   "source": [
    "# Load T5 tokenizer to process text and summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b3d4454e-e7f3-4020-9ecc-4c692c6da5c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoTokenizer\n",
    "\n",
    "checkpoint = \"google-t5/t5-small\"\n",
    "tokenizer = AutoTokenizer.from_pretrained(checkpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15e4537b-4a72-434a-8af7-1cef702581d0",
   "metadata": {},
   "source": [
    "# Perprocessing Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2feec165-87ed-4258-a332-14c3b7dd9867",
   "metadata": {},
   "outputs": [],
   "source": [
    "prefix = \"summarize: \"\n",
    "\n",
    "\n",
    "def preprocess_function(examples):\n",
    "    inputs = [prefix + doc for doc in examples[\"text\"]]\n",
    "    model_inputs = tokenizer(inputs, max_length=1024, truncation=True)\n",
    "\n",
    "    labels = tokenizer(text_target=examples[\"summary\"], max_length=128, truncation=True)\n",
    "\n",
    "    model_inputs[\"labels\"] = labels[\"input_ids\"]\n",
    "    return model_inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "814f0888-e5c0-4841-9084-ed4f6762ecdd",
   "metadata": {},
   "source": [
    "# Tokenize the dataset using the map function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "65c755b9-33ae-4d95-b0d2-b7b7950f4f71",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "23e819666e29433196ea0a7457855dbf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/212 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "64e22b879f724c4cbe294ac1418910ef",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/53 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokenized_billsum = billsum.map(preprocess_function, batched=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "294ec0e7-5120-4c13-9517-6e4407dd12e7",
   "metadata": {},
   "source": [
    "# Creating a batch of examples using DataCollatorForSeq2Seq with dynamic padding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a1e9d07d-7cbf-46c2-87c6-9ebc97fcbfcf",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-04-12 23:20:17.382485: I tensorflow/core/util/port.cc:111] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
      "2024-04-12 23:20:17.422130: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "2024-04-12 23:20:17.422161: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "2024-04-12 23:20:17.422185: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
      "2024-04-12 23:20:17.429463: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
      "To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2024-04-12 23:20:18.470755: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n"
     ]
    }
   ],
   "source": [
    "from transformers import DataCollatorForSeq2Seq\n",
    "\n",
    "data_collator = DataCollatorForSeq2Seq(tokenizer=tokenizer, model=checkpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd435fc8-af8b-419b-b41b-384f6b412119",
   "metadata": {},
   "source": [
    "# Importing the evaluator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "bc7159a1-3997-47bd-8d17-48c2e09f7ace",
   "metadata": {},
   "outputs": [],
   "source": [
    "import evaluate\n",
    "\n",
    "rouge = evaluate.load(\"rouge\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11f1979d-91e3-414f-b76b-36034e97b8b0",
   "metadata": {},
   "source": [
    "# Function that passes predictions and labels to compute to calculate the Rouge metric."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e301045f-eb8d-4c2d-b2f5-59acf2f053b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_metrics(eval_pred):\n",
    "    predictions, labels = eval_pred\n",
    "    decoded_preds = tokenizer.batch_decode(predictions, skip_special_tokens=True)\n",
    "    labels = np.where(labels != -100, labels, tokenizer.pad_token_id)\n",
    "    decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)\n",
    "\n",
    "    result = rouge.compute(predictions=decoded_preds, references=decoded_labels, use_stemmer=True)\n",
    "\n",
    "    prediction_lens = [np.count_nonzero(pred != tokenizer.pad_token_id) for pred in predictions]\n",
    "    result[\"gen_len\"] = np.mean(prediction_lens)\n",
    "\n",
    "    return {k: round(v, 4) for k, v in result.items()}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3833f68-20da-4343-88c6-4b5c144bfb7c",
   "metadata": {},
   "source": [
    "# Training - Loading T5 with AutoModelForSeq2SeqLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "8a9100fb-4ebe-469d-a1a3-b0a00c54528a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoModelForSeq2SeqLM, Seq2SeqTrainingArguments, Seq2SeqTrainer\n",
    "\n",
    "model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9722c5f5-a9ed-44d1-b677-9fe558890218",
   "metadata": {},
   "source": [
    "# Defining the training hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e9d5eb20-6e52-44c7-aa09-5ad6c09576d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ORIGINAL\n",
    "# training_args = Seq2SeqTrainingArguments(\n",
    "#     output_dir=\"my_awesome_billsum_model\",\n",
    "#     evaluation_strategy=\"epoch\",\n",
    "#     learning_rate=2e-5,\n",
    "#     per_device_train_batch_size=16,\n",
    "#     per_device_eval_batch_size=16,\n",
    "#     weight_decay=0.01,\n",
    "#     save_total_limit=3,\n",
    "#     num_train_epochs=4,\n",
    "#     predict_with_generate=True,\n",
    "#     fp16=True,\n",
    "#     push_to_hub=True,\n",
    "# )\n",
    "\n",
    "training_args = Seq2SeqTrainingArguments(\n",
    "    output_dir=\"congress_bill_summary_model\",\n",
    "    evaluation_strategy=\"epoch\",\n",
    "    learning_rate=2e-4,\n",
    "    per_device_train_batch_size=16,\n",
    "    per_device_eval_batch_size=16,\n",
    "    weight_decay=0.01,\n",
    "    save_total_limit=3,\n",
    "    num_train_epochs=4,\n",
    "    predict_with_generate=True,\n",
    "    fp16=True,\n",
    "    push_to_hub=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa5635ee-8d6a-45ce-8976-90df52eacd8a",
   "metadata": {},
   "source": [
    "# Passing the arguments, model, dataset, tokenizer, data collator, and compute_metrics function to Seq2SeqTrainer for training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3476e539-1f86-41b2-beec-f9441406a692",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/u.th161689/.local/lib/python3.11/site-packages/accelerate/accelerator.py:436: FutureWarning: Passing the following arguments to `Accelerator` is deprecated and will be removed in version 1.0 of Accelerate: dict_keys(['dispatch_batches', 'split_batches']). Please pass an `accelerate.DataLoaderConfiguration` instead: \n",
      "dataloader_config = DataLoaderConfiguration(dispatch_batches=None, split_batches=False)\n",
      "  warnings.warn(\n",
      "Detected kernel version 4.18.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.\n"
     ]
    }
   ],
   "source": [
    "trainer = Seq2SeqTrainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    train_dataset=tokenized_billsum[\"train\"],\n",
    "    eval_dataset=tokenized_billsum[\"test\"],\n",
    "    tokenizer=tokenizer,\n",
    "    data_collator=data_collator,\n",
    "    compute_metrics=compute_metrics,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91c564a2-6a8c-4073-ac83-aa45b2ae28fa",
   "metadata": {},
   "source": [
    "# Fine-tuning the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "d3f06124-0e43-4cf6-9af8-af0b42fa1b78",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "You're using a T5TokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='56' max='56' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [56/56 00:38, Epoch 4/4]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "      <th>Rouge1</th>\n",
       "      <th>Rouge2</th>\n",
       "      <th>Rougel</th>\n",
       "      <th>Rougelsum</th>\n",
       "      <th>Gen Len</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.108650</td>\n",
       "      <td>0.443800</td>\n",
       "      <td>0.427400</td>\n",
       "      <td>0.442700</td>\n",
       "      <td>0.442200</td>\n",
       "      <td>19.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.060382</td>\n",
       "      <td>0.455200</td>\n",
       "      <td>0.439900</td>\n",
       "      <td>0.455100</td>\n",
       "      <td>0.454600</td>\n",
       "      <td>19.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.053176</td>\n",
       "      <td>0.461700</td>\n",
       "      <td>0.448200</td>\n",
       "      <td>0.462300</td>\n",
       "      <td>0.462000</td>\n",
       "      <td>19.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>No log</td>\n",
       "      <td>0.049938</td>\n",
       "      <td>0.463300</td>\n",
       "      <td>0.449800</td>\n",
       "      <td>0.463500</td>\n",
       "      <td>0.463100</td>\n",
       "      <td>19.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.11/site-packages/transformers/generation/utils.py:1355: UserWarning: Using the model-agnostic default `max_length` (=20) to control the generation length. We recommend setting `max_new_tokens` to control the maximum length of the generation.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "TrainOutput(global_step=56, training_loss=0.17396153722490584, metrics={'train_runtime': 39.9457, 'train_samples_per_second': 21.229, 'train_steps_per_second': 1.402, 'total_flos': 177655146872832.0, 'train_loss': 0.17396153722490584, 'epoch': 4.0})"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Training needs numpy\n",
    "import numpy as np\n",
    "\n",
    "# Fine-tuning the model\n",
    "trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1e694f0-394b-4a68-8183-59fdf76d30a3",
   "metadata": {},
   "source": [
    "# Saving the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "b4d1a48a-55cb-4456-aa3f-861e86645cae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2483f039d1834b40a8f0904f0207907e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/242M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "64cf8e2edd09426d84c75d1bdbaeacee",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "events.out.tfevents.1712982022.fc003.3364345.0:   0%|          | 0.00/7.79k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c7cfea8e6b0a44e49a7158d0a81edcab",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "training_args.bin:   0%|          | 0.00/4.86k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4c29d1060f564100a4f3021ff3fc30ca",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "events.out.tfevents.1712981439.fc003.3322740.0:   0%|          | 0.00/7.79k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "59c0974429b74163a23bc5750116c026",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload 4 LFS files:   0%|          | 0/4 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "trainer.save_model(\"./congress_bill_summary_model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a93edb3-cc87-43fa-86af-7d77c66ddaae",
   "metadata": {},
   "source": [
    "# Pushing Model to Personal HuggingFace Hub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "19fb405b-5413-4bfa-a365-66e8729a1559",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CommitInfo(commit_url='https://huggingface.co/cheaptrix/congress_bill_summary_model/commit/dcdfc898778d0665ae87f330cd063a2a27fee3f4', commit_message='cheaptrix/congress_bill_summary_model', commit_description='', oid='dcdfc898778d0665ae87f330cd063a2a27fee3f4', pr_url=None, pr_revision=None, pr_num=None)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.push_to_hub(\"cheaptrix/congress_bill_summary_model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "166d4903-8809-4a77-9f43-45316cbccc91",
   "metadata": {},
   "source": [
    "# Interface"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "c960268a-411e-4abc-8f95-f88cf5b11055",
   "metadata": {},
   "outputs": [],
   "source": [
    "cleaned_test_data = []\n",
    "# Opening JSON file\n",
    "with open(\"cleaned_bill_sum_test_data.json\") as json_file:\n",
    "    cleaned_test_data = json.load(json_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "484d784c-2700-4e2e-8c09-99293468487d",
   "metadata": {},
   "source": [
    "# Creating a pipeline with the transformers module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "7298e9e4-4a53-4494-8f8a-0a69dbb55d52",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e989111988064fa799f348449f05235f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/242M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from transformers import pipeline\n",
    "\n",
    "summarizer = pipeline(\"summarization\", model=\"cheaptrix/congress_bill_summary_model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "fbde06cc-a162-437e-bda4-89f5de65c808",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'A concurrent resolution establishing deadlines for the Joint Committee of Congress on the Library to approve or deny the statue of the Reverend William Franklin \"Billy\" Graham, Jr., for placement in the National Statuary Hall. 2023-02-16 Introduced in Senate This concurrent resolution requires the Joint Committee on the Library to approve or deny the statue of Rev. Billy Graham for placement in the National Statuary Hall within 30 days after North Carolina submits specific information about the statute, including its dimensions and final weight. text/xml EN Pursuant to Title 17 Section 105 of the United States Code, this file is not subject to copyright protection and is in the public domain. Congressional Research Service, Library of Congress This file contains bill summaries for federal legislation. A bill summary describes the most significant provisions of a piece of legislation and details the effects the legislative text may have on current law and federal programs. Bill summaries are authored by the Congressional Research Service (CRS) of the Library of Congress. As stated in Public Law 91-510 (2 USC 166 (d)(6)), one of the duties of CRS is \"to prepare summaries and digests of bills and resolutions of a public general nature introduced in the Senate or House of Representatives\". For more information, refer to the User Guide that accompanies this file.'"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text = cleaned_test_data[3]['text']\n",
    "text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "72b8b7cb-f4cd-4282-9343-d8ec1be2d0d5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'summary_text': 'This concurrent resolution requires the Joint Committee on the Library to approve or deny the statue of Rev. Billy Graham for placement in the National Statuary Hall within 30 days after North Carolina submits specific information about the statute, including its dimensions and final weight.'}]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "summarizer(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "f3744109-c882-454a-9ecb-ff9fc7468e0a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'This concurrent resolution requires the Joint Committee on the Library to approve or deny the statue of Rev. Billy Graham for placement in the National Statuary Hall within 30 days after North Carolina submits specific information about the statute, including its dimensions and final weight.'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_test_data[3]['summary']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "c1b02b6c-5d8d-421e-9225-17802b183304",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Token indices sequence length is longer than the specified maximum sequence length for this model (548 > 512). Running this sequence through the model will result in indexing errors\n"
     ]
    }
   ],
   "source": [
    "generated_summaries = []\n",
    "for i in range(len(cleaned_test_data)):\n",
    "    summary = summarizer(cleaned_test_data[i]['text'])\n",
    "    generated_summaries.append({\"generated\" : summary[0]['summary_text'], \"summary\" : cleaned_test_data[i]['summary']})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "1791b057-3684-4e51-8d4e-3649a7811f20",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This resolution requires the Architect of the Capitol, the Secretary of the Senate, and the Chief Administrative Officer of the House of Representatives to encourage Capitol gift shops to accept cryptocurrency and to enter into contracts with vendors that accept cryptocurrency to provide food service and vending machines in the Capitol.',\n",
       " 'summary': 'Adopting Cryptocurrency in Congress as an Exchange of Payment for Transactions Resolution or the ACCEPT Resolution This resolution requires the Architect of the Capitol, the Secretary of the Senate, and the Chief Administrative Officer of the House of Representatives to encourage Capitol gift shops to accept cryptocurrency and to enter into contracts with vendors that accept cryptocurrency to provide food service and vending machines in the Capitol.'}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "ecc976d1-27f0-4094-94a2-ac1d025b9595",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution commends the bravery, courage, and resolve of the women and men of Iran who are (1) participating in the current protests to defend their fundamental human rights, and (2) risking their safety to speak out against the human rights abuses committed by the Iranian regime.',\n",
       " 'summary': 'This concurrent resolution commends the bravery, courage, and resolve of the women and men of Iran who are (1) participating in the current protests to defend their fundamental human rights, and (2) risking their safety to speak out against the human rights abuses committed by the Iranian regime. The resolution condemns (1) the brutal beating and death of Mahsa Amini; and (2) the violent suppression by the Iranian regime of women and men participating in the current demonstrations, including children, and calls for transparent accountability for all killings of protesters by Iranian security forces. Finally, the resolution encourages continued efforts by the Biden Administration to respond to the protests, including the recent sanctioning of the Iranian morality police.'}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "f3739cf1-846a-4f1e-acac-be1f629001da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution calls for honoring the 237th anniversary of the enactment of the Virginia Statute for Religious Freedom on Religious Freedom Day, January 16, 2023. The resolution affirms that religious freedom includes the right of individuals of any faith and individuals of no faith to live, work, associate, and worship in accordance with their beliefs; all people of the United States can be unified in supporting religious freedom because it is a fundamental human right; and the American people will remain forever unshackled in matters of faith.',\n",
       " 'summary': 'This concurrent resolution calls for honoring the 237th anniversary of the enactment of the Virginia Statute for Religious Freedom on Religious Freedom Day, January 16, 2023. The resolution affirms that religious freedom includes the right of individuals of any faith and individuals of no faith to live, work, associate, and worship in accordance with their beliefs; all people of the United States can be unified in supporting religious freedom because it is a fundamental human right; and the American people will remain forever unshackled in matters of faith.'}"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "ad90c77a-bc5c-48e9-a22d-9f7f95a34625",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution requires the Joint Committee on the Library to approve or deny the statue of Rev. Billy Graham for placement in the National Statuary Hall within 30 days after North Carolina submits specific information about the statute, including its dimensions and final weight.',\n",
       " 'summary': 'This concurrent resolution requires the Joint Committee on the Library to approve or deny the statue of Rev. Billy Graham for placement in the National Statuary Hall within 30 days after North Carolina submits specific information about the statute, including its dimensions and final weight.'}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "e15cdcb1-10e4-4a0b-8921-b9f56bfb63de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution declares that Congress should not impose any new performance fee, tax, royalty, or other charge relating to the public performance of sound recordings on a local radio station for broadcasting sound recordings over the air or on any business for such public performance .',\n",
       " 'summary': 'This concurrent resolution declares that Congress should not impose any new performance fee, tax, royalty, or other charge relating to the public performance of sound recordings on a local radio station for broadcasting sound recordings over the air or on any business for such public performance of sound recordings.'}"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "fa5ad2a1-5dae-44c6-b442-d632d37dd0e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution recognizes Abortion Provider Appreciation Day. Text/xml EN Pursuant to Title 17 Section 105 of the United States Code, this file is not subject to copyright protection and is in the public domain.',\n",
       " 'summary': 'This concurrent resolution recognizes Abortion Provider Appreciation Day.'}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "7ac4cad6-e406-4160-a1c4-dede5d30db1a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': \"This concurrent resolution condemns Russia's unjust and arbitrary detention of Russian democratic opposition leader Vladimir Kara-Murza and calls for his immediate release and the release of all other Russian opposition leaders.\",\n",
       " 'summary': \"This concurrent resolution condemns Russia's unjust and arbitrary detention of Russian democratic opposition leader Vladimir Kara-Murza and calls for his immediate release and the release of all other Russian opposition leaders. It also calls for the release of all political prisoners in Russia and in Belarus, as well as for the release of Ukrainian citizens held by Russia, and calls on the President and leaders of the free world to work tirelessly for the release of political prisoners in Russia.\"}"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[6]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "5f197b89-be47-48bd-a1ab-926e95294392",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution expresses the sense of Congress that tax-exempt fraternal benefit societies provide critical benefits to the people and communities of the United States and their work should continue to be promoted.',\n",
       " 'summary': 'This concurrent resolution expresses the sense of Congress that tax-exempt fraternal benefit societies provide critical benefits to the people and communities of the United States and their work should continue to be promoted.'}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[7]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "6f6e69b9-8e2e-4daf-9809-40bb835456fa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution expresses the sense of Congress that climate change caused by human activities constitutes a climate emergency, which demands the President use existing authorities and emergency powers to mitigate and prepare for the consequences of the emergency.',\n",
       " 'summary': 'This concurrent resolution expresses the sense of Congress that climate change caused by human activities constitutes a climate emergency, which demands the President use existing authorities and emergency powers to mitigate and prepare for the consequences of the emergency.'}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[8]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "6400f171-c491-406f-85a6-aab41d159e59",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': \"Fiscal State of the Nation Resolution This concurrent resolution requires the congressional budget committees to conduct an annual joint hearing to receive a presentation from the Comptroller General regarding (1) the Government Accountability Office's audit of the financial statement of the executive branch, and (2) the financial position and condition of the federal government.\",\n",
       " 'summary': \"Fiscal State of the Nation Resolution This concurrent resolution requires the congressional budget committees to conduct an annual joint hearing to receive a presentation from the Comptroller General regarding (1) the Government Accountability Office's audit of the financial statement of the executive branch, and (2) the financial position and condition of the federal government.\"}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "912101c6-66fb-4250-9320-1885e64f61c2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution expresses the sense that the Senate should provide its advice and consent to ratification of the Convention on Biological Diversity.',\n",
       " 'summary': 'This concurrent resolution expresses the sense that the Senate should provide its advice and consent to ratification of the Convention on Biological Diversity.'}"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "31a9a3d8-0ecd-47d2-95d3-855490f824cc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution calls on the media to voluntarily adopt certain practices to prevent further harm from its coverage of mass public murders. Specifically, it calls for coverage that (1) denies murderers a public platform; (2) minimizes the potential for media reporting to increase the likelihood of future mass public killings (i.e., the media contagion effect); and (3) prioritizes the victims of, and heroism in the response to, mass public killers.',\n",
       " 'summary': 'This concurrent resolution calls on the media to voluntarily adopt certain practices to prevent further harm from its coverage of mass public murders. Specifically, it calls for coverage that (1) denies murderers a public platform; (2) minimizes the potential for media reporting to increase the likelihood of future mass public murders (i.e., the media contagion effect); and (3) prioritizes the victims of, and heroism in the response to, mass public murders.'}"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[11]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "ece694d3-1870-4f9a-902d-346b97e54f8d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution affirms, more than 400 years after the arrival of the first slave ship, that the United States owes a debt of remembrance to those who lived through slavery or other historical injustices against people of color, as well as to their descendants. It also urges the establishment of a U.S. Commission on Truth, Racial Healing, and Transformation',\n",
       " 'summary': 'This concurrent resolution affirms, more than 400 years after the arrival of the first slave ship, that the United States owes a debt of remembrance to those who lived through slavery or other historical injustices against people of color, as well as to their descendants. It also urges the establishment of a U.S. Commission on Truth, Racial Healing, and Transformation.'}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[12]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "346509c4-c31e-4e6e-a5df-b3b470a3dc8a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': \"This concurrent resolution recognizes the disparity between wages paid to Latina women in comparison to men and reaffirms Congress's support for ensuring equal pay and closing the gender wage gap.\",\n",
       " 'summary': \"This concurrent resolution recognizes the disparity between wages paid to Latina women in comparison to men and reaffirms Congress's support for ensuring equal pay and closing the gender wage gap.\"}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[13]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "3c2d0904-4b8c-45ab-99dd-f661e55438b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution expresses the sense of Congress that a carbon tax would be detrimental to families and businesses and would severely harm the economic and national security of the country.',\n",
       " 'summary': 'This concurrent resolution expresses the sense of Congress that a carbon tax would be detrimental to families and businesses and would severely harm the economic and national security of the country.'}"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[14]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "23e3e126-ae9f-4e5e-94ed-9ab55b9b4dec",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'This concurrent resolution makes a correction to the official title of H.R. 815 (Making emergency supplemental appropriations for the fiscal year ending September 30, 2024, and for other purposes).',\n",
       " 'summary': 'This concurrent resolution makes a correction to the official title of H.R. 815 (Making emergency supplemental appropriations for the fiscal year ending September 30, 2024, and for other purposes).'}"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generated_summaries[15]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "f3c08476-c1d6-4fd3-9bb1-1590d4e1ec65",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_cleaned_test_data = []\n",
    "# Opening JSON file\n",
    "with open(\"new_cleaned_bill_sum_test_data.json\") as json_file:\n",
    "    new_cleaned_test_data = json.load(json_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "324fa648-056b-4a5d-8990-48fb75ea032b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'summary': 'Saving Access to Laboratory Services Act This bill modifies provisions relating to Medicare payment rates for clinical diagnostic laboratory services, including by requiring payment rates for certain widely available clinical diagnostic laboratory tests to be based on a statistical sampling of private sector rates.',\n",
       "  'text': 'Saving Access to Laboratory Services Act 2023-03-28 Introduced in Senate Saving Access to Laboratory Services Act This bill modifies provisions relating to Medicare payment rates for clinical diagnostic laboratory services, including by requiring payment rates for certain widely available clinical diagnostic laboratory tests to be based on a statistical sampling of private sector rates. text/xml EN Pursuant to Title 17 Section 105 of the United States Code, this file is not subject to copyright protection and is in the public domain. Congressional Research Service, Library of Congress This file contains bill summaries for federal legislation. A bill summary describes the most significant provisions of a piece of legislation and details the effects the legislative text may have on current law and federal programs. Bill summaries are authored by the Congressional Research Service (CRS) of the Library of Congress. As stated in Public Law 91-510 (2 USC 166 (d)(6)), one of the duties of CRS is \"to prepare summaries and digests of bills and resolutions of a public general nature introduced in the Senate or House of Representatives\". For more information, refer to the User Guide that accompanies this file.',\n",
       "  'title': 'Saving Access to Laboratory Services Act'},\n",
       " {'summary': 'Agriculture Resilience Act of 2023 This bill establishes, expands, and revises multiple programs and activities of the Department of Agriculture (USDA) primarily to reduce carbon emissions from the agriculture sector. Specifically, USDA must finalize and implement a plan to achieve net-zero emissions from the sector by 2040. USDA must periodically review and revise the plan, as necessary, and annually report on its implementation. Additionally, the bill expands the scope of various USDA research, extension, and education programs; conservation programs; and livestock programs to incorporate climate change adaptation and mitigation. Expanded activities include efforts to improve soil health and preserve farmland and grassland. Further, the bill changes programs that support renewable energy in rural areas to address carbon emissions in the agriculture sector. Among these changes, the bill provides statutory authority for the AgSTAR program for reducing methane emissions from livestock waste and requires the program to be moved from the Environmental Protection Agency to USDA. The bill also addresses food waste, for example, by (1) standardizing the voluntary labels used by food producers to indicate the date by which food should be used or discarded, and (2) making composting activities eligible for support through USDA conservation programs. Moreover, the bill establishes grants to reduce and prevent food waste in landfills and in schools.',\n",
       "  'text': 'Agriculture Resilience Act of 2023 2023-03-28 Introduced in Senate Agriculture Resilience Act of 2023 This bill establishes, expands, and revises multiple programs and activities of the Department of Agriculture (USDA) primarily to reduce carbon emissions from the agriculture sector. Specifically, USDA must finalize and implement a plan to achieve net-zero emissions from the sector by 2040. USDA must periodically review and revise the plan, as necessary, and annually report on its implementation. Additionally, the bill expands the scope of various USDA research, extension, and education programs; conservation programs; and livestock programs to incorporate climate change adaptation and mitigation. Expanded activities include efforts to improve soil health and preserve farmland and grassland. Further, the bill changes programs that support renewable energy in rural areas to address carbon emissions in the agriculture sector. Among these changes, the bill provides statutory authority for the AgSTAR program for reducing methane emissions from livestock waste and requires the program to be moved from the Environmental Protection Agency to USDA. The bill also addresses food waste, for example, by (1) standardizing the voluntary labels used by food producers to indicate the date by which food should be used or discarded, and (2) making composting activities eligible for support through USDA conservation programs. Moreover, the bill establishes grants to reduce and prevent food waste in landfills and in schools. text/xml EN Pursuant to Title 17 Section 105 of the United States Code, this file is not subject to copyright protection and is in the public domain. Congressional Research Service, Library of Congress This file contains bill summaries for federal legislation. A bill summary describes the most significant provisions of a piece of legislation and details the effects the legislative text may have on current law and federal programs. Bill summaries are authored by the Congressional Research Service (CRS) of the Library of Congress. As stated in Public Law 91-510 (2 USC 166 (d)(6)), one of the duties of CRS is \"to prepare summaries and digests of bills and resolutions of a public general nature introduced in the Senate or House of Representatives\". For more information, refer to the User Guide that accompanies this file.',\n",
       "  'title': 'Agriculture Resilience Act of 2023'}]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_cleaned_test_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "996ad83e-6b0c-4819-b44f-ed91d4d32c7e",
   "metadata": {},
   "outputs": [],
   "source": [
    "new_generated_summaries = []\n",
    "for i in range(len(new_cleaned_test_data)):\n",
    "    summary = summarizer(new_cleaned_test_data[i]['text'])\n",
    "    new_generated_summaries.append({\"generated\" : summary[0]['summary_text'], \"summary\" : new_cleaned_test_data[i]['summary']})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "4351fd53-9f09-4794-b34c-092732d27ff1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'Saving Access to Laboratory Services Act This bill modifies provisions relating to Medicare payment rates for clinical diagnostic laboratory services, including by requiring payments rates for certain widely available clinical diagnostic lab tests to be based on a statistical sampling of private sector rates.',\n",
       " 'summary': 'Saving Access to Laboratory Services Act This bill modifies provisions relating to Medicare payment rates for clinical diagnostic laboratory services, including by requiring payment rates for certain widely available clinical diagnostic laboratory tests to be based on a statistical sampling of private sector rates.'}"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_generated_summaries[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "f8f57bd4-5082-4c0a-a61b-523814f60db6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'generated': 'USDA must periodically review and revise the plan, as necessary, and annually report on its implementation. Additionally, the bill expands the scope of various USDA research, extension, and education programs; conservation programs; and livestock programs to incorporate climate change adaptation and mitigation. Expanded activities include efforts to improve soil health and preserve farmland and grassland. Further, the Bill changes programs that support renewable energy in rural areas to address carbon emissions in the agriculture sector.',\n",
       " 'summary': 'Agriculture Resilience Act of 2023 This bill establishes, expands, and revises multiple programs and activities of the Department of Agriculture (USDA) primarily to reduce carbon emissions from the agriculture sector. Specifically, USDA must finalize and implement a plan to achieve net-zero emissions from the sector by 2040. USDA must periodically review and revise the plan, as necessary, and annually report on its implementation. Additionally, the bill expands the scope of various USDA research, extension, and education programs; conservation programs; and livestock programs to incorporate climate change adaptation and mitigation. Expanded activities include efforts to improve soil health and preserve farmland and grassland. Further, the bill changes programs that support renewable energy in rural areas to address carbon emissions in the agriculture sector. Among these changes, the bill provides statutory authority for the AgSTAR program for reducing methane emissions from livestock waste and requires the program to be moved from the Environmental Protection Agency to USDA. The bill also addresses food waste, for example, by (1) standardizing the voluntary labels used by food producers to indicate the date by which food should be used or discarded, and (2) making composting activities eligible for support through USDA conservation programs. Moreover, the bill establishes grants to reduce and prevent food waste in landfills and in schools.'}"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_generated_summaries[1]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}