diff --git "a/notebooks/[ADVANCED] NLP_Primer_twitter_challenge.ipynb" "b/notebooks/[ADVANCED] NLP_Primer_twitter_challenge.ipynb" new file mode 100644--- /dev/null +++ "b/notebooks/[ADVANCED] NLP_Primer_twitter_challenge.ipynb" @@ -0,0 +1,4075 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "IucN-YUVlwQu" + }, + "source": [ + "# Introduction\n", + "\n", + "When the '#ZindiWeekendz Learning: To Vaccinate or Not to Vaccinate: It’s not a Question' originally ran as a hackathon, someone linked a notebook on Kaggle as a getting started resource: https://www.kaggle.com/abhishek/approaching-almost-any-nlp-problem-on-kaggle. It's some good info, but reading that and trying to place myself in the shoes of a beginner I felt like it was a) too much to take in and b) not even the best approach given current tools. So, I started a new discussion titled 'The Lazy NLP Route'. Here's what I said:\n", + "\n", + "*Thanks @overfitting_PLB for sharing the kaggle getting started with NLP notebook, but wow is that a lot of code. TF-IDF, word vectors, custom models, cross-validation, ensembles of different models.... I thought I'd share an alternate view.*\n", + "\n", + "*Deep learning has pretty much taken over NLP. Language models like those available through fastai or huggingface are able to capture nuances of text, and can be trained with very little effort. They handle the tokenization etc, and I find them super easy to use.*\n", + "\n", + "*I tried two different approaches, each ~10 lines of code, training time under 15 minutes. Both ~0.6 scores. Both have PLENTY of room for improvement since I did almost no optimising. I'm not going to share code for this one (maybe in the future) but here are some places to get started:*\n", + "\n", + "*1) Fastai text. The docs are decent: https://docs.fast.ai/text.html. I didn't do any language model tuning (there's a place to look for improvements!) but went straight to training a `text_classifier_learner(data_clas,AWD_LSTM,drop_mult=0.3, metrics=[rmse])` - give it a validation set and you get RMSE (like the Zindi score) as it trains!*\n", + "\n", + "*2) Huggingface transformers via the simpletransformers library. The github has docs including a regression example: https://github.com/ThilinaRajapakse/simpletransformers#minimal-start-for-regression. Hugginface do amazing work, but if you look for tutorials many of them have lots of code to copy and paste - I like the simpletransformers library as it simplifies a lot of that and gets out of the way. You specify some parameters, pick a model architecture (I chose DistilBERT) and basically hit go :)*\n", + "\n", + "*The reason I ran these models and am sharing this: a lot of smart people have tried very hard to make it easy to solve new challenges in the field of NLP. But there are so many options, and it's hard to know where to start. These are two ideas for you to research and play with. They're not hobbled beginner methods, they're the real deal. And it's possible to make good predictions with them. They've given me good results in the workplace and my hobby projects. So if you're not sure where to start, pick one and dig in, and see if you can get it working. You'll be playing with the cutting edge of NLP research, and hopefully, it'll let you get up there on the 'board without needing a masters degree in ML :) Good luck!*\n", + "\n", + "*PS: Disagree, and think you should start from the basics and work up? Let's chat! I'm hoping this will spark some interesting discussion about SOTA in NLP, how to learn, using first vs bottom up... Drop your view in the discussion here :)*\n", + "\n", + "So, now that this is open as a knowledge competition, I figured it's time to share the actual code! The winners blog and code repositories show that transformers won the day - score one for fancy new tools :) Let's dive in and see how we can use them ourselves.\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "_O6s11nLnVEv" + }, + "source": [ + "# 1) Quick LSTM with fastai\n", + "\n", + "Here's a minimal solution with fastai, using the AWD_LSTM language model to solve this task. TO run this, make sure you've uploaded the csv files from Zindi into Colab using the files pane on the left." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "iQ-8b6eLnrYk" + }, + "outputs": [], + "source": [ + "import numpy as np \n", + "import pandas as pd \n", + "from pathlib import Path\n", + "from fastai.text import *" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 142 + }, + "colab_type": "code", + "id": "Ymf7qs79nvQZ", + "outputId": "ed32dc22-f8c3-4fac-fd00-9d805042a900" + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
tweet_idsafe_textlabelagreement
0CL1KWCMYMe & The Big Homie meanboy3000 #MEANBOY #M...0.01.0
1E3303EMEI'm 100% thinking of devoting my career to pro...1.01.0
2M4IVFSMS#whatcausesautism VACCINES, DO NOT VACCINATE Y...-1.01.0
\n", + "
" + ], + "text/plain": [ + " tweet_id ... agreement\n", + "0 CL1KWCMY ... 1.0\n", + "1 E3303EME ... 1.0\n", + "2 M4IVFSMS ... 1.0\n", + "\n", + "[3 rows x 4 columns]" + ] + }, + "execution_count": 2, + "metadata": { + "tags": [] + }, + "output_type": "execute_result" + } + ], + "source": [ + "# Load the data\n", + "train = pd.read_csv('Train.csv').dropna(0) # Read in train, ignoring one row with missing data\n", + "test = pd.read_csv('Test.csv').fillna('') # Read in test\n", + "test['label']=0 # We'll fill this in with predictions later\n", + "train.head(3) # Take a peek at the data" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "y1TlPEUmoSgE" + }, + "source": [ + "Fastai uses something called a databunch to store the data. The docs show how to create one. Here, we add our test data with test_df=test, and split our training data into df_train and df_valid (to let us see scores on a validation set while it trains)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "colab_type": "code", + "id": "N8JEEdb_oL7s", + "outputId": "bfe885b1-1b16-492d-f35c-077754b6393c" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "(1000, 4) (8999, 4)\n" + ] + }, + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "# Build the databunch, and keep 1000 rows for validation\n", + "df_valid = train.sample(1000)\n", + "df_train = train.loc[~train.tweet_id.isin(df_valid.tweet_id.values)]\n", + "print(df_valid.shape, df_train.shape)\n", + "data_clas=TextClasDataBunch.from_df(path=Path(''),train_df=df_train, \n", + " valid_df=df_valid,\n", + " test_df=test,\n", + " label_cols='label',\n", + " text_cols='safe_text')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "iiFWYKWvonJA" + }, + "source": [ + "Now we have the data ready, we can create a model to train:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "colab_type": "code", + "id": "dE5u8ARMoqxK", + "outputId": "206b375c-9a2a-4166-edd1-d47c96609ed4" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Downloading https://s3.amazonaws.com/fast-ai-modelzoo/wt103-fwd.tgz\n" + ] + }, + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "# Learner\n", + "clas = text_classifier_learner(data_clas,AWD_LSTM,drop_mult=0.3, metrics=[rmse])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "X_3s-JopoxiU" + }, + "source": [ + "There are things we could do t pick learning rates etc, but this is a minimal example. Let's train our model! I'm running it for 20 epochs as a fairly arbitrary choice." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 669 + }, + "colab_type": "code", + "id": "u10h2Ol4o7R5", + "outputId": "9245b00d-b5cc-45d3-c09c-07f02cada964" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossroot_mean_squared_errortime
00.5642820.4651410.67984800:03
10.4623210.4362670.65781400:03
20.4322910.4277910.65120000:03
30.4216210.4275250.65111200:03
40.4240480.4361130.65798700:03
50.4243950.4327180.65511700:03
60.4236300.4295240.65254000:03
70.4219320.4306180.65344800:03
80.4173340.4280090.65137700:03
90.4202410.4317650.65454500:03
100.4130760.4251350.64914800:03
110.4101140.4266050.65015400:03
120.4086550.4249620.64913300:03
130.4134600.4214720.64609500:03
140.4086210.4218880.64654800:03
150.4110270.4217550.64655200:03
160.4082740.4195680.64479500:03
170.4118490.4193020.64452800:03
180.4124530.4201880.64528000:03
190.4169010.4198860.64490700:03
" + ], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "clas.fit_one_cycle(20)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "IhVVkm3ZpDN9" + }, + "source": [ + "You can see the RMSE decrease as it trains. This is the metric used in the competition, and a score of ~0.6 is pretty good looking at the leaderboard.We'll do better than 0.64 later, but for now let's save predictions:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 17 + }, + "colab_type": "code", + "id": "qxN-Vo09pAVA", + "outputId": "f0d3ac79-f09d-4458-b564-c87c156f5493" + }, + "outputs": [ + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "# Get predictions\n", + "preds, y = clas.get_preds(DatasetType.Test)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 204 + }, + "colab_type": "code", + "id": "YqJG0szrpa_B", + "outputId": "e2ad4b11-c685-4694-af30-090370d8c2bb" + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
tweet_idlabel
000BHHHP10.249226
100UNMD0E0.271966
201AXPTJF0.288489
301HOEQJW0.336241
401JUKMAO0.250573
\n", + "
" + ], + "text/plain": [ + " tweet_id label\n", + "0 00BHHHP1 0.249226\n", + "1 00UNMD0E 0.271966\n", + "2 01AXPTJF 0.288489\n", + "3 01HOEQJW 0.336241\n", + "4 01JUKMAO 0.250573" + ] + }, + "execution_count": 7, + "metadata": { + "tags": [] + }, + "output_type": "execute_result" + } + ], + "source": [ + "# Make a submission dataframe\n", + "sub = pd.DataFrame({\n", + " 'tweet_id':test['tweet_id'],\n", + " 'label':[p[0] for p in preds.numpy()]\n", + "})\n", + "sub.to_csv('first_try_fastai_20_epochs.csv', index=False)\n", + "sub.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "lpg0QEZdpjIM" + }, + "source": [ + "This scores 0.64 on the LB. Not bad, but we'll keep on improving. Bu tthis isn't bad for such a quick starting point!" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "qNwtv8P6qDTF" + }, + "source": [ + "# 1.2 - Fastai with some better tuning\n", + "\n", + "Building on the previous example, let's now follow the steps as taught in the fastai course for text. First, we'll re-train our language model on our data, then we'll train a classifier." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 236 + }, + "colab_type": "code", + "id": "liZkhfN3orq6", + "outputId": "3e808499-fff7-4fd7-d784-d605516ab4e3" + }, + "outputs": [ + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossaccuracytime
04.7481883.7063170.33035700:05
13.9509203.5733250.34672600:05
" + ], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossaccuracytime
03.4136203.3384160.38072900:07
13.2281223.2604420.39412200:07
23.0514553.2436840.39449400:07
" + ], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "# Creating a databunch for the language model\n", + "data_lm = TextLMDataBunch.from_df(path='', train_df=df_train, \n", + " valid_df=df_valid,\n", + " text_cols='safe_text')\n", + "# And the learner\n", + "learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.5)\n", + "\n", + "# Some very quick training - could do much more here\n", + "learn.fit_one_cycle(2, 1e-2)\n", + "learn.unfreeze()\n", + "learn.fit_one_cycle(3, 1e-3)\n", + "\n", + "# We save the encoder for later use\n", + "learn.save_encoder('ft_enc')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "CAZ75KBSqmF8" + }, + "source": [ + "Now we have a language model trained on tweets, we can use the encoder as part of our text classifier:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 408 + }, + "colab_type": "code", + "id": "fpChSjN0qgub", + "outputId": "615e750b-1018-45fe-b5cf-9e70e8575e96" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "
\n", + " \n", + " \n", + " 0.00% [0/1 00:00<00:00]\n", + "
\n", + " \n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossroot_mean_squared_errortime

\n", + "\n", + "

\n", + " \n", + " \n", + " 59.29% [83/140 00:02<00:01 0.6125]\n", + "
\n", + " " + ], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n", + "Min numerical gradient: 6.31E-07\n", + "Min loss divided by 10: 3.02E-02\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEGCAYAAABYV4NmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3hUZfbA8e9JJyGhhgBJgAChVwlIsSAqokuzIdjLWldd665uYV3d4q7154quZdG1LSoooqJYQFCK9CK9Q6ihJ4H08/tjbnAICUnI3MxMcj7PMw8zd947c14myZn33veeV1QVY4wxxtdC/B2AMcaYmskSjDHGGFdYgjHGGOMKSzDGGGNcYQnGGGOMK8L8HYCvNG7cWFu1auXvMIwxJqgsWrRon6rGu/HaNSbBtGrVioULF/o7DGOMCSoistWt17ZDZMYYY1xhCcYYY4wrLMEYY4xxhSUYY4wxrrAEY4wxxhWWYIwxxrjCEowxxhhXuJpgRGSIiKwVkQ0i8kgpzz8nIkud2zoROVTi+TgRSReRF92MMxh8+dMuvvxpF8fyCv0dijHGVIhrF1qKSCgwDrgQSAcWiMgUVV1V3EZV7/dqfw/Qs8TLPAHMcivGYHEgO4+73l1MkUKd8FDO6xDPkC7NGNwpgajwUH+HZ4wxpXJzBNMH2KCqm1Q1D5gAjDhF+zHA/4ofiEgvIAH4ysUYg8LMdXspUnh8RGcu75XIgi0Hufd/S7jt7UXYgnHGmEDlZoJJBLZ7PU53tp1ERFoCKcB053EI8AzwkIvxBY1vVu8lPjaSa89syV9GdmXeo+fz8EXtmbUug69X7fF3eMYYU6pAOck/GpioqsUnGO4Cpqpq+ql2EpHbRGShiCzMyMhwPUh/yC8sYtbaDAa1b0JIiAAQGiLcdk5rUpvU5S+frya3wM7LGGMCj5sJZgeQ7PU4ydlWmtF4HR4D+gF3i8gW4GngehF5suROqvqqqqapalp8vCvFQP1uwZYDZOYWMKhjkxO2h4eGMHZYJ7YdOMr4H7b49D0PZOfx2JSVtP/DF1z7+o/MWLOXoiI7FGeMqRw3qykvAFJFJAVPYhkNXF2ykYh0ABoAc4u3qeo1Xs/fCKSp6kmz0GqD6av3EhEawlltG5/03Nmp8VzQMYEXp6/n8jMSaRIXVaX3yskvZPzszbw8YyPZeQUM6dKURVsPctObC2gdH8NNA1K4sleSTSwwxlSIayMYVS0A7gamAauBD1R1pYg8LiLDvZqOBiaona0u1fQ1e+nXphExkaV/F/jDLzqSV1jEP6etrdL7bD9wlPOfmck/v1xLn5SGTLvvHF66phc//HYQ/ze6B3Ujw/jj5J8Y89o8MjJzq/RexpjaQWrK3/W0tDStaevBbMrIYtAzM3l8RGeu79eqzHZ//2I1r8zcxCe/GkD35Pqn9V4PvL+UqT/tYvyNvenf5uTRkqoydcVuHvxwKY1iIvnPjWl0aBp3Wu9ljAkcIrJIVdPceO1AOclvSjF9zV4Azmvf5JTt7hmUSnxsJGM/+YmCwqJKv8+mjCwmL93BdX1blppcAESEX3Rrxoe39ye/sIjLX5rD9DU2g80YUzZLMAHs29V7aZ8QS3LD6FO2qxsZxp+GdWJZ+mFembWp0u/z4owNRISFcNs5bcpt2zWpHp/cPYBWjWP45X8X8snSsuZtGGNqO0swAerwsXwWbDlw0uyxsgzt1pyh3Zrx/DfrWLXzSIXfZ/O+bCYv2cG1Z7YkPjayQvs0q1eHD+/oR/fk+jz+6Soyc/Ir/H7GmNrDEkyA+n59BgVFygUVTDAAT4zoQr06ETzwwVLyCip2qOzF6RsIDw3htnNbVyq+6IgwHhvWmf3Zebwys/KjJmNMzWcJJkBNX72XBtHh9EhuUOF9GsRE8ORlXVmzO5P/+3Zdue237Mtm8tIdXNu3JU1iKz/FuXtyfYZ3b87rP2xi1+Fjld7fGFOzWYIJQAWFRcxYu5fz2jch1Ll6v6Iu6JTAlb2SePm7jSzZdvCUbV+csYGwEOH2So5evD18UXuKiuCZr8pPaMaY2sUSTACasTaDg0fzuahL09Pa/4/DOtE0Lop7/reEbfuPltpm/Z5MPl6yg2vOPL3RS7HkhtHcOKAVkxanV+rcjzGm5rMEE4De/XErCXGRnN+h4udfvMVFhfPytb3Iyi3g8n/PYfWuE//wf78+gytfmUvdyDDuqMLopdivBrYlLiqcv3+xusqvZYypOSzBBJjtB44yc10Go3u3ICz09D+e7sn1+fD2foSFCKNemcuPm/ajqvx75kZuGD+fhNgoPvnVgCqXlwGoFx3Oveen8v36fcxcVzOLjhpjKs8STIB5b/42QkQY3Se5/MblSE2IZeKd/WkSG8l14+dz3X/m8+QXa7i4azM+uqs/rRrH+CBij+v6tqRlo2j++vmq07rYM1Coqk27NsZHLMEEkNyCQj5YsJ3zOzShWb06PnnNxPp1+PCO/nRsFsecjft49OIOvDimZ5m1zU5XRFgIv7ukI+v2ZPHuj9t8+trV5dDRPK77z3z6/306+7Os3poxVeVmNWVTSdNW7mF/dh7X9m3p09dtGBPBB7f3Zc/hXFo0OnVVgKoY3CmBAW0b8ezX6xjevTkNYiJcey9fW7s7k1vfWsiuw8fIL1Q+XrKDX55d9fNTxtRmNoIJIO/M20rLRtGlluavqsiwUFeTC3jqlY0d2pnMnHye+yZ4pi1/+dNuLn1pNsfyC5lwWz96tqjP+wu223LUxlSRJZgAsX5PJvM3H+DqPi2Or1wZjNo3jeXavi15Z95W1uwO/GnLb8/byh3vLCI1IZZP7z6LXi0bMLp3Muv3ZrF42yF/h2dMULMEEyDe/XEbEaEhXNEryd+hVNkDF7Yjrk44f56yKqBHASvSD/P4pysZ1KEJ79/Wl6b1PDPqhnZrTkxEKO8vCM5zScYECkswAeBYXiGTFqdzSdemNKpbsYKTgax+dAQPXNiOuZv2M23lbn+HU6qs3ALu+d9i4utG8uyo7ies0hkTGcaw7s35dNkum1FmTBVYggkAX6/eQ2ZOAaP7tPB3KD5zdZ8WtEuoyz+/XEtRkX9GMarKuBkbeHH6enLyC094buzkn9h24CjPj+5J/eiTJyOM6p3MsfxCPlu+q7rCNabGsQQTAD5fvpOEuEj6tGro71B8Jiw0hF+d15ZN+7KZsXavX2IYP3sLT01by9NfrWPI87OY5VwE+tHidD5asoN7z0+lT0rp/+c9k+vTLqEu7y/YXp0hG1OjWILxs8ycfGaszeCSrs2C+uR+aS7p2oxm9aJ4/fvN1f7e36zaw18+X8WQzk15+5Y+iAjXj5/PHW8v4o+Tf6JPSkPuGZRa5v4iwlW9W7B0+6GgmKxgTCCyBONn36zeQ15BEUO7Nfd3KD4XHhrCDf1bMXfTflbuPFxt7/vTjsPcO2EJXRPr8dxVPTg7NZ4vfn0291/Qjulr9xIWGsLzV/Uot1L1pT0TiQgNsVGMMafJ1QQjIkNEZK2IbBCRR0p5/jkRWerc1onIIWd7DxGZKyIrRWS5iFzlZpz+9NmyXTSvF0XP5Pr+DsUVY3q3oE54KP/5oXpGMbsP5/DL/y6kfp1wXr8+jToRnpP3UeGh/PqCVKY/eC5T7h5A8/rlV0poGBPB4M4JTFqUzovT1/PJ0h0s3naQg9l5bnfDmAr7ZOkOPlgYmF+CXLuSX0RCgXHAhUA6sEBEpqjqquI2qnq/V/t7gJ7Ow6PA9aq6XkSaA4tEZJqq1qgLEw4fy2fW+gxu7N+qxh0eK1YvOpxRaUm8N38bjwzpUOXimit3HuZgdj75RUUUFip5hUXsOpzD1v3ZbN1/lJU7j3Asr8BTg62U90pqULmLTe84tw1Lth3iaa/1bsJChJev7cWFnRKq1BdjfOGDhdvJyS9iVFrV6xf6mpulYvoAG1R1E4CITABGAKvKaD8G+BOAqh7/bVbVnSKyF4gHalSC+WrlbvILtUYeHvN204AU3pq3lbfmbuWhi9qX2S6/sIhl2w/RI7n+SZWkD2TnMfaTn8qc1RUbGUaLRtH0SWnAjf1T6Ngsziexd0msx+xHBnE0r4D0g8fYfuAoz369jocnLuPLX59z/NoZY/wlK6eAeqXMhAwEbiaYRMB73JYOnFlaQxFpCaQA00t5rg8QAWws5bnbgNsAWrQIvim+ny3fRXLDOnRLqufvUFzVqnEMF3RM4N0ft/Kr89oeP2xV0l8/X82bc7aQ1KAONw1I4areydSNDGPayt38/uMVHD6Wz/0XtKNfm0aEhQrhISGEhQoJcVE0iA5HxL1RYHREGO0SYmmXEEurxjEMfeEHHvxwKW/ffGaNHX2a4JCVW0BiA98Ux/W1QCl2ORqYqKonXKwgIs2At4EbVPWkGvCq+irwKkBaWlrgXjJeioPZeczesI9fnt3a1T+MgeKWs1L4etUePlqSzjVnnlzMc96m/bw5ZwuDOyVw6Gg+T3y2iue/Xke35HrM3rCfTs3iePuWM302MqmKNvF1+dOwTjzy0Qpe+34Tt5/bxt8hmVosO7eQmIhA+VN+Ijej2gF4HxRMcraVZjTwK+8NIhIHfA78XlXnuRKhH01buZuCImVot2b+DqVanJnSkC6Jcbw0YyPntW9ywkn27NwCHp64jFaNonl+dA+iI8JYtv0Qr/+wme/XZ3Dv+ancfV5bIsICZ9LjVb2Tmbkug6emraVfm0Z0S6qZkzRM4MvOLaBuVGAmGDd/YxcAqSKSIiIReJLIlJKNRKQD0ACY67UtAvgYeEtVJ7oYo998tnwXrRpF07m5/7+RVwcR4c/Du3DkWD5XvDyHDXuzjj/35BdrSD94jKeu7E60802se3J9/jWmJ0vHDuaBC9sFVHIBT3/+fllX4mMj+fWEpRyxkjLGD1SVrLwC6vp4fSdfce23VlULgLuBacBq4ANVXSkij4vIcK+mo4EJemJVxFHAOcCNXtOYe/g8yI0b4a67IC4OQkI8/951l2e7i/Zl5TJn4z6GdmteKw6PFevVsgETbu9LXqFy5b/nsGTbQWZv2Mfb87Zyy4AUegdZJYP60RE8O6oHW/dnc/4zM5m4KP2ksjgHs/N4f8E2ZqzdW2bJnN2Hc9hzJKc6QjY1zNG8QlTx+QKCviKBXO22MtLS0nThwoUV3+GLL+CKKyA/33MrFh7uuU2cCBdf7PtAgd9MXMaHi9L56r5zSE2IdeU9AtnW/dlc95/5ZGTmEhsVRt3IMKb++uwTCk4GkyXbDvLnT1exdPshuifV4w9DO3HkWD4TF6Xzzeo95Bd6fseSG9bh2jNbMiotmdBQ4csVu/l4yQ7mbd5P47qRfPvgucRFhfu5NyaY7D2SQ5+/fcsTI7tw3WkuVCgii1Q1zceheV67ViaYjRuhWzc4erTsNtHRsHw5tPHtCdwvVuzizncX86vz2vDwRR18+trBZO+RHG54YwFrdx/hwzv606tlA3+HVCVFRcrkpTv4x5dr2HPEs9xyo5gIRvZM5NKeiWzZn81bc7cyf/OB44f78gqKSGkcw3ntm/DGnM3c1D+FscM6+bMbJshsyshi0DMzef6qHozsmXhar+FmggnMcZXbnnnmxFFLafLz4bnn4MUXT3rqnXlb2ZeVy50D2xAZVvFv3bsOH+ORj1bQLake913QrrJR1yhN4qKYdGc/dh46RtsmwT+KCwkRLjsjiYs6N2XS4nSa1avDwPbxhDvX83RJrMfQbs1ZuzuT/833rDMzsmci3ZPqISLkFBTy37lbGNU7iQ5Na8d5OVN1WbkFgB0ic12lRjBxcZCZWbF2h0+soTVn4z6uef1HVKF9QizPXtWdzs3Lv46lqEi5bvyPLN56iM/vPYvW8XUrFqupFQ5m5zHome9IbRLL+7f3rVXn5szpm7NxH1e/9iPv3Xom/duc3lLrbo5gAmtqTnXJyiq/TSntDmbn8cD7y0hpHMO/rz2Dg0fzGDluNuNmbKCg8KTLdE7wnx82M3vDfsYO62TJxZykQUwEvx3SgflbDjB5aVmz+Y05UXau59LBWjeLLKDVreAfeK92qsqjH61gf3YuL4zuyZAuzZh23zkM7tyUp6at5eb/LixzeeDVu47w1LS1DO6UwOjegVcvyASGUWnJdE+uz9+mrrFpz6ZCsgP8EFntTDDXXuuZKXYq4eFw3XXHH76/YDtfrtzNQ4Pb0yXRc0isQUwE464+gz/8oiOz1mUwZdnOk15GVfnj5J+IjQrjycu72aEPU6aQEOGJEZ3Zl5XL09PWlvmFxZhimU6CibUEE0AefLBiCeZ+T7HnjRlZ/PnTVQxo24hbz259UtObB6TQuXkc//hiDcfyTlyad/LSHSzcepDfDGlPw5jALEhnAke3pPpc37clb83dyh3vLGJfVq6/QzIBzEYwgahNG891LtHRJyUaDQ/3bJ84Edq0obBIeeD9pUSGh/DMlT1KLWwYEiL8cWgndh7O4bXvNx3fnpVbwN+nrqFbUj2u7GWHxkzFjB3Wmd9d0oEZazMY/NwsvlhRegVpY7JzCxCB6DIKyPpb7Uww4LmIcvlyuO02iItDRciMiGb5xaM8252LLN+Zt5Vl6Yf58/DOpyzN3rd1Iy7u0pSXv9t4/Krsf01fz97MXP48vLNV3DUVFhoi3HZOGz6/5yySGtThzncX8+AHy8qsBGBqr6zcAmIiwgL20HvtTTDgGcm8+CIcPowUFXHLC9/wwDm/RFt7DoPtOZLDU9PWcnZqY4Z3L3/Nlkcv7khhkfLPL9eyMSOL8T9s5opeSfRsEdwXERr/SE2IZdKd/bn9nNZMWpzOrPUZ/g7JBJjs3MCtQwa1PcGUMKJHczZmZLNq1xEAHv9sFXmFRTwxokuFviG0aBTNzWelMGlxOne/t4SosFB+O6T2Xq1vqi48NIQHB7enSWwkb8ze4u9wTIDJyi0gJjIwD4+BJZgTXNKlGWEhwpSlO/lu7V4+X76Le85rS6vGMRV+jV+d14bGdSNYvesIv74glfjYSBcjNrVBRFgI1/Vtycx1GSdUoTYmK7fQRjDBokFMBOe2i2fKsp2M/WQlreNjuO3ck2eNnUpsVDj/uLwbl/ZM5Ib+rdwJ1NQ6V5/ZgoiwEN6cs9nfoZgAkp1bELAzyMASzEmG92jOrsM5bDtwlL+M7FKpWmPFzu+YwHNX9Theh8qYqmpUN5KRPZozadEODh3N83c4JkDYOZggc2GnBOKiwriiV9Jp1/Yxxg03DUjhWH4hExZs93coJkBk5gR2ggncyPwkOiKMbx8cSP1oW5fDBJaOzeLo17oRb83Zwi/PSiHMRsi1XnaeHSILOvGxkXZ4ywSkm89KYefhHKat3OPvUEwAsHMwxhifGdShCS0aRvPGbDvZX9vlFhSSX6jERlmCMcb4QGiIcGP/VizcepBXZ230dzjGj7JynDpkAVomBuwcjDFB59q+LVm09SB/m7qGw8fyeWhw+4AtFWLcU7wWTK09RCYiQ0RkrYhsEJFHSnn+ORFZ6tzWicghr+duEJH1zu0GN+M0JphEhIXwwpiejO6dzLgZGxn7yUqrU1YLFS+XXCtnkYlIKDAOuBBIBxaIyBRVXVXcRlXv92p/D9DTud8Q+BOQBiiwyNn3oFvxGhNMQkOEv1/WlXp1wnll1iaO5OTz9JXdbXJKLZKd5ySYWnoOpg+wQVU3qWoeMAEYcYr2Y4D/OfcvAr5W1QNOUvkaGOJirMYEHRHh0Us68psh7flk6U7uencxuQWF5e9oaoTj52ACeATjZoJJBLyvCEt3tp1ERFoCKcD0yuwrIreJyEIRWZiRYZVmTe1018C2/Hl4Z75etYfb315ETr4lmdogGA6RBcp4ejQwUVUr9Zuhqq+qapqqpsXHx7sUmjGB74b+rXjysq7MXJfBzW8u4Khz+MTUXIG+miW4m2B2AN7LOCY520ozmp8Pj1V2X2MMMLpPC565sjvzNu3nhvHzrWZZDXd8BBNROxPMAiBVRFJEJAJPEplSspGIdAAaAHO9Nk8DBotIAxFpAAx2thljTuGyM5J4YUxPlmw7xKBnZjJh/jabYVZDZR0fwQTudTCuJRhVLQDuxpMYVgMfqOpKEXlcRIZ7NR0NTFBV9dr3APAEniS1AHjc2WaMKcfQbs2ZcvdZtImP4ZGPVnDpS7NZtv1Q+TuaoJKdW0BUeEhA16QTr7/rQS0tLU0XLlzo7zCMCRiqyuSlO/jb1DXsy8rln5d348q05PJ3NEHh0Y9W8PWq3Sz8w4VVeh0RWaSqaT4K6wSBm/qMMVUiIlzaM4npD55Lz+T6/HPaWo7l2QyzmiLQC12CJRhjarzYqHB+O6QDGZm5vDNvq7/DMT4S6IuNgSUYY2qFM1s34qy2jfn3zI3Hp7ea4JZpIxhjTKB4YHA79mfn8d+5W/wdivEBG8EYYwLGGS0acF77eF6dtYnMnHx/h2OqyM7BGGMCyv0XtuPQ0XzG/7Dl+La1uzP5zcRljJuxwX+BmUrLyi0M+BFMYEdnjPGpbkn1ubBTAq//sIluSfV4e95Wpq/ZS4hAkUJi/TqM7FlqyUATYLJy86kbwBdZgo1gjKl17r+gHZk5Bdz05gKWbj/EAxe248ffXUCfVg155KPlrNx52N8hmnIUFBaRk18U8IfIAjs6Y4zPdWoexxMjOqPAlb2SqeMsufviNT0Z9q8fuOOdRXx691nUj44A4EB2Hv+avp7tB47x/OgeAX9YpjbIdq5nCvTPwkYwxtRC1/VrxfX9Wh1PLgBNYqN4+dpe7D6cw70TlnIsr5BXZm7k3Kdm8N85W5i+Zg/3vLeYgsIiP0Zu4OdKypZgjDFB44wWDXhseGdmrcugz1+/4e9frKF3q4ZMu+8c/jKyKzPWZjB2ykpqSompYJUVBKX6wQ6RGWNKuLpPCzbuzWZZ+iEevLAd/ds2BiA1IZbtB4/y8ncbSW4QzZ0D2/g50torGBYbA0swxpgSRISxwzqV+tzDg9uTfvAY//hyDYkN6jC8e/Nqjs5AcCw2BnaIzBhTCSEhwlNXdKN3qwY88P5Snv16HXkFdk6mutk5GGNMjRQVHsrrN/RmWPfmvPDteob96weWp9t6M9UpMyc4EkxgR2eMCUj16oTz3FU9GNqtGb/7eAUjx83m2r4tSYiLIje/kGP5hYSIcNOAFJrWi/J3uDVOdhCsZgmWYIwxVXB+xwS+atWQv09dzVtzPUsBiEBUWCj5hUV8tnwXb9/Sh9bxdf0cac1SfB1MoJ+DCezojDEBr16dcJ68vBt/GtaZkBCICA1BRFiRfpgb35jPFf+ey5s39aZbUn1/h1pjZOUWEBYiRIYF9lmOwI7OGBM06kSEEhkWiogA0DWpHhPv7E90RChjXp3HD+v3+TnCmiMrp4C6UWHH/68DlSUYY4xrUhrHMOnO/iQ3jOamN+ezYMsBf4dUI2TnFhATEfgHoFxNMCIyRETWisgGEXmkjDajRGSViKwUkfe8tv/T2bZaRF6QQE/VxphSJcRF8f7t/ahXJ5z/fL/Z3+HUCFlBsNgYuJhgRCQUGAdcDHQCxohIpxJtUoFHgQGq2hm4z9neHxgAdAO6AL2Bc92K1Rjjrnp1whnZI5Fv1+zhQHaev8MJetl5BQE/gwzcHcH0ATao6iZVzQMmACNKtLkVGKeqBwFUda+zXYEoIAKIBMKBPS7Gaoxx2eW9ksgvVKYs3eHvUIJeVm4hdaPC/R1GudxMMInAdq/H6c42b+2AdiIyW0TmicgQAFWdC8wAdjm3aaq6uuQbiMhtIrJQRBZmZGS40gljjG90bBZH5+ZxTFyc7u9Qgl5WTuAvNgb+P8kfBqQCA4ExwGsiUl9E2gIdgSQ8SWmQiJxdcmdVfVVV01Q1LT4+vhrDNsacjit6JfHTjiOs3nXE36EEtezcwppzkl9EYkQkxLnfTkSGi0h547MdQLLX4yRnm7d0YIqq5qvqZmAdnoRzKTBPVbNUNQv4AuhXkViNMYFrRI9EwkOFSYtsFFMV2bkFAX+RJVR8BDMLiBKRROAr4DrgzXL2WQCkikiKiEQAo4EpJdpMxjN6QUQa4zlktgnYBpwrImFOIjsXOOkQmTEmuDSMiWBQhyZMXrqDfFu47LSoKll5BcRG1ZwEI6p6FLgMeElVrwQ6n2oHVS0A7gam4UkOH6jqShF5XESGO82mAftFZBWecy4Pq+p+YCKwEVgBLAOWqeqnleybMSYAXdErmX1Zecxca+dNT8fRvEJUA79MDFS8VIyISD/gGuAWZ1u5Z5hUdSowtcS2sV73FXjAuXm3KQRur2BsxpggMrB9PI1iIpi4KJ0LOiX4O5ygEyxrwUDFRzD34ble5WNnFNIaz4jDGGMqJTw0hJE97ZqY0/XzapY1ZBaZqs5U1eGq+g/nZP8+Vb3X5diMMTXUFc41MZOX2DUxlZWd66mkXDeyhlwHIyLviUiciMQAPwGrRORhd0MzxtRUHZvF0T25Pm/P20pRkfo7nKCSmZsPBP5aMFDxQ2SdVPUIMBLPlOEUPDPJjDHmtNw8oBWb92Uzc52d7K+Mn0cwNeccTLgzXXgkznUreMq5GGPMabm4SzOaxEYyfrYVwKyMmniS/xVgCxADzBKRloBdimuMOW0RYSFc17cl36/fx4a9mf4OJ2gUn+SPrSkJRlVfUNVEVb1EPbYC57kcmzGmhrv6zBZEhIXwxuwt/g4laGTVtBGMiNQTkWeLC0uKyDN4RjPGGHPaGtWNZET35ny0eAeHj+b7O5ygkJ1bgAhER9Sck/zjgUxglHM7ArzhVlDGmNrjpgEpHMsvZMKCbf4OJShkOatZBsMajBVNMG1U9U/O2i6bVPXPQGs3AzPG1A6dmsfRt3VD3pq7lQKrT1YuT6HLwB+9QMUTzDEROav4gYgMAI65E5Ixpra5aUAKOw4d4+tVtq5gebJzC4NiijJUvBbZHcBbIlLPeXwQuMGdkIwxtc0FHRNIrF+H9+Zv4+KuzfwdTkDLzC0ImgRT0Vlky1S1O9AN6KaqPYFBrkZmjKk1QkOEkT2bM2fjfvZl5fo7nIAWLGvBQCVXtFTVI84V/VCiArIxxlTF8O6JFBYpU1fs8ncoAa3GJpgSAn8KgzEmaLRvGku7hLpMWbrT36EEtBmQX/UAABezSURBVMycgqC4yBKqlmCsVIwxxqeGd2/Owq0H2XHI5hCVJTuvhoxgRCRTRI6UcssEmldTjMaYWmJYd8+flc+W2SimNKpKVk4BdYNguWQoJ8GoaqyqxpVyi1XV4OihMSZotGwUQ/fk+kyxBFOq3IIiCoq0Zs0iM8aY6jK8e3NW7jzCxowsf4cScH5ezdISjDHGVNrQbs0QwU72lyLbEszPRGSIiKwVkQ0i8kgZbUaJyCoRWSki73ltbyEiX4nIauf5Vm7GaowJDAlxUfRNacSny3aianOJvGXmBE8lZXAxwYhIKDAOuBjoBIwRkU4l2qQCjwIDVLUzcJ/X028BT6lqR6APsNetWI0xgWVY9+Zs2pfNyp227JS34hFMbE04yV9FfYANTnHMPGACMKJEm1uBcap6EEBV9wI4iShMVb92tmep6lEXYzXGBJCLuzQlLETsZH8JwbQWDLibYBKB7V6P051t3toB7URktojME5EhXtsPichHIrJERJ5yRkQnEJHbiteoyciwdb2NqSkaxEQwsH08Hy/ZQV6BVVguZif5KycMSAUGAmOA10SkvrP9bOAhoDeepQFuLLmzqr6qqmmqmhYfH19dMRtjqsE1Z7YkIzOXr1bt9ncoAcMSzM92AMlej5Ocbd7SgSmqmq+qm4F1eBJOOrDUObxWAEwGznAxVmNMgDmnXTzJDevwzryt/g4lYByfRWbnYFgApIpIiohEAKOBKSXaTMYzekFEGuM5NLbJ2be+iBQPSwYBq1yM1RgTYEJDhKv7tGTepgOs35Pp73ACQpYziyw6vGYtOFZpzsjjbmAasBr4QFVXisjjIjLcaTYN2C8iq4AZwMOqul9VC/EcHvtWRFbgKaz5mluxGmMC06i0JCJCQ3j3R1tOGSDLWWwsJCQ4ag27Os5S1anA1BLbxnrdVzxl/08q/e/MIOvmZnzGmMDWqG4kl3RtyqRF6Tx8UfugmT3llqzc/KBZLhn8f5LfGGNO6bp+LcnMLbApywTXcslgCcYYE+DOaNGADk1jeXvu1lp/ZX8wLZcMlmCMMQFORLiuX0tW7TrC4m2H/B2OX2XnBk+pfrAEY4wJAiN7JFI3MqzWT1nOyikgJsISjDHG+ExMZBgjezZn6opdHMnJ93c4fpNlIxhjjPG9K3olk1tQxOfLd/k7FL/JsnMwxhjje92T6tEmPoZJi9L9HYpfqKrnHIwlGGOM8S0R4fJeSSzcepAt+7L9HU61K14uOZiuBbIEY4wJGpf2TEQEPlpc+0YxWUG2FgxYgjHGBJFm9epwVtvGTFq8g6Ki2nVNTHEdMptFZowxLrn8jCR2HDrGj5sP+DuUapUVZJWUwRKMMSbIXNS5KXUjw5hUyw6TBdtaMGAJxhgTZOpEhPKLrs34YsUujuYV+DucapNtCcYYY9x3ea8ksvMK+fKn2rPaZfEIxmaRGWOMi3q3akCLhtFMrEXXxNgsMmOMqQYiwqi0JOZs3M/ibQf9HU61KJ5FZofIjDHGZTcNSCE+NpInPltVK8r4Z+cWIALREbbgmDHGuComMoyHB7dnybZDtWIxsszcAupGhCESHMslgyUYY0wQu7xXEp2axfGPL9aQk1/o73BclZ1bEFQn+MESjDEmiIWGCH8c2omdh3N4/ftN/g7HVcFWqh9cTjAiMkRE1orIBhF5pIw2o0RklYisFJH3SjwXJyLpIvKim3EaY4JXvzaNGNwpgZe+28jeIzn+Dsc1WbmFNoIpJiKhwDjgYqATMEZEOpVokwo8CgxQ1c7AfSVe5glgllsxGmNqht9d0pH8wiKe/mqtv0NxTVZOPrGWYI7rA2xQ1U2qmgdMAEaUaHMrME5VDwKo6t7iJ0SkF5AAfOVijMaYGqBV4xhu6NeKDxels2rnEX+H44rs3EJiIoNnBhm4m2ASge1ej9Odbd7aAe1EZLaIzBORIQAiEgI8Azx0qjcQkdtEZKGILMzIyPBh6MaYYHPPoFTiosJ58ss1/g7FFZ7VLMP9HUal+PskfxiQCgwExgCviUh94C5gqqqe8jJdVX1VVdNUNS0+Pt71YI0xgatedDj3DGrLrHUZfL++5n3h9CQYG8EU2wEkez1OcrZ5SwemqGq+qm4G1uFJOP2Au0VkC/A0cL2IPOlirMaYGuC6fi1JalCHv09dU6PWi1FVm0VWwgIgVURSRCQCGA1MKdFmMp7RCyLSGM8hs02qeo2qtlDVVngOk72lqqXOQjPGmGKRYaE8fFF7Vu06wuSlJb/PBq+c/CIKg2y5ZHAxwahqAXA3MA1YDXygqitF5HERGe40mwbsF5FVwAzgYVXd71ZMxpiab1i35nRNrMfT09bWmIsvjxe6tATzM1WdqqrtVLWNqv7V2TZWVac491VVH1DVTqraVVUnlPIab6rq3W7GaYypOUJChEcv6cDOwzm8OWeLv8PxiWAs1Q/+P8lvjDE+179NY85rH8+4GRs4mJ3n73CqLBgXGwNLMMaYGuqRizuSlVvAS99t8HcoVZYZhKX6wRKMMaaGat80lsvPSOK/c7ay49Axf4dTJcdHMDaLzBhjAsP9F7YDgee+XufvUKrEzsEYY0yASaxfhxv6tWTS4nTW7A7eEjI2i8wYYwLQXQPbUjcyjKe+DN5CmDaCMcaYANQgJoI7B7bh2zV7mb/5gL/DOS3BuFwyWIIxxtQCN/VPISEukie/WM3hY/kcyfHcjuYV+Du0CsnMCb7lksFTbNIYY2q0OhGh3HdBOx79aAXd//zzCiChIcI/Lu/GFb2S/Bhd+bKDsA4ZWIIxxtQSo9KSiY4IJSMz9/i2Kct28uQXq7m4S9OAPr+RlVsQ0PGVJfgiNsaY0xAaIozoceKSVL1aNuDSl+Yw/ofN3HN+qp8iK5+nVH/w/bm2czDGmFqrZ4sGXNQ5gVdnbeJAAJeUsQRjjDFB6KHB7cnOK+DlAC4pk20Jxhhjgk9qglNSZu5WdgZoSZmsnOA8yW8JxhhT6913YTtQ+L9v1vs7lFLZITJjjAlSifXrcF2/lny4aDsb9mb5O5wTHF8u2RKMMcYEp1+d15Y64aG8MnOjv0M5QU5+EUUafGViwBKMMcYA0DAmgqHdmvP5il3Hy+MHgszcfCD4SvWDJRhjjDluVO8kjuYV8vmKXf4O5bjs3EIA6kYGVx0ysARjjDHHndGiAa3jY5i4MN3foRyXdXw1y3A/R1J5riYYERkiImtFZIOIPFJGm1EiskpEVorIe862HiIy19m2XESucjNOY4wBEBGu6JXE/C0H2Lwv29/hAN6l+m0Ec5yIhALjgIuBTsAYEelUok0q8CgwQFU7A/c5Tx0Frne2DQGeF5H6bsVqjDHFLj8jiRCBiYu2+zsUwHuxMRvBeOsDbFDVTaqaB0wARpRocyswTlUPAqjqXuffdaq63rm/E9gLxLsYqzHGAJAQF8W57eKZtGgHhUXq73COTziwEcyJEgHvrwDpzjZv7YB2IjJbROaJyJCSLyIifYAI4KS5gyJym4gsFJGFGRkZPgzdGFObXZmWzO4jOfywYZ+/QyHTSTA2i6zywoBUYCAwBnjN+1CYiDQD3gZuUtWikjur6quqmqaqafHxNsAxxvjG+R2bUD86nA8W+v8wWfEIxi60PNEOINnrcZKzzVs6MEVV81V1M7AOT8JBROKAz4Hfq+o8F+M0xpgTRIaFMrJHIl+v3MOho/6tspyVU0CIQJ1wO0TmbQGQKiIpIhIBjAamlGgzGc/oBRFpjOeQ2San/cfAW6o60cUYjTGmVFemJZFXWMSHfp6yXLzYWLAtlwwuJhhVLQDuBqYBq4EPVHWliDwuIsOdZtOA/SKyCpgBPKyq+4FRwDnAjSKy1Ln1cCtWY4wpqXPzepyZ0pC/Tl3NXz9fRW5BoV/iyMotIDYID48BiKr/Z0n4Qlpami5cuNDfYRhjapBjeYX8bepq3p63lQ5NY/m/0T1p3zS2WmO4851FbNibxdcPnOvK64vIIlVNc+O1/X2S3xhjAladiFCeGNmF8TemsS8rl2Ev/sC/vl3P4aP51RZDVm5wrgUDlmCMMaZcgzok8OV95zCwXTzPfL2Ofk9+y2NTVrJ1v/tX+wdrqX6wBGOMMRXSuG4kr16fxtR7z2ZIl6a8++NWBj79HY9MWk6RixdkZuVYgjHGmFqhU/M4nh3Vgx9+O4jr+7ZkwoLt/Gv6BtfeL9uZRRaMgjNqY4zxs4S4KB4b3pnMnAKe/3YdXZPiGNQhwafvUVBYxKFj+TaCMcaY2kZE+NtlXenYNI77Jixli48rMH+/YR9H8wrp27qRT1+3uliCMcaYKogKD+WV63oREiLc8c4ijub5bjXMjxbvoH50OOd1CM5SWJZgjDGmipIbRvPC6J6s25PJwxN9c9L/SE4+X63czbBuzYkMC74yMWAJxhhjfOKcdvH8ZkgHPl++i7FTfqKqF7F/sWIXuQVFXHZGySL0wSM4zxwZY0wAuv2c1hzMzuOVWZuICgvl97/oeNo1xCYt3kHrxjH0SA7etRYtwRhjjI+ICI9c3IHcgiJe/2EzUeGhPHRRewBUlTW7M9l24CiDOjQhPLTsA0jbDxxl/uYDPDS4XVAWuSxmCcYYY3xIRBg7tBM5+YW8OGMD+7PzOJpXwOwN+9mXlQvA2amNefnaXmVOP/54iWdlk5E9g/fwGFiCMcYYnwsJEf56aVfyCor43/xtNK4bwYC2jRnQtjHH8gp5/LNVXPXKXN64qTdNYqNO2FdV+WhxOn1bNySpQbSfeuAblmCMMcYFoSHCM6O685shHWgSG0lIyM+Hulo0iuaudxZz2Utz+O/NfWgTX/f4c4u3HWLL/qPcdV5bf4TtUzaLzBhjXCIiNK0XdUJyATivfRPev70vOfmFXP7yHB6bspIpy3aSfvAoHy1OJyo8hIu7NPVT1L5jIxhjjPGDbkn1mXRnf/4w+ScmLNjGm3O2HH9uRI/mxEaF+y84H7EEY4wxftKyUQxv33Im+YVFrN2dyeJtB1m96wg3DUjxd2g+YQnGGGP8LDw0hC6J9eiSWM/fofiUnYMxxhjjCkswxhhjXOFqghGRISKyVkQ2iMgjZbQZJSKrRGSliLzntf0GEVnv3G5wM05jjDG+59o5GBEJBcYBFwLpwAIRmaKqq7zapAKPAgNU9aCINHG2NwT+BKQBCixy9j3oVrzGGGN8y80RTB9gg6puUtU8YAIwokSbW4FxxYlDVfc62y8CvlbVA85zXwNDXIzVGGOMj7mZYBKB7V6P051t3toB7URktojME5EhldgXEblNRBaKyMKMjAwfhm6MMaaq/H2SPwxIBQYCY4DXRKTCtalV9VVVTVPVtPj44FzxzRhjaio3E8wOINnrcZKzzVs6MEVV81V1M7AOT8KpyL7GGGMCmFR11bUyX1gkDE/COB9PclgAXK2qK73aDAHGqOoNItIYWAL0wDmxD5zhNF0M9FLVA6d4vwxga4nN9YDDldxW3v3GwL6y4ihHae9dmTYV6U919aW8WMtrU9m+lHxcfN97m302FYu1vDb22fj3b8Cp2rnRlxhVdecQkKq6dgMuwZNkNgK/d7Y9Dgx37gvwLLAKWAGM9tr3ZmCDc7vpNN//1cpuK+8+sLAK/x8nvXdl2lSkP9XVl6r2p7J9OUUfvLfZZ2OfTUB/NhXpiy8/G7d/zsq7uVoqRlWnAlNLbBvrdV+BB5xbyX3HA+OrGMKnp7GtIvd9GU9l2lSkP9XVl4q+TlltKtuXko8/LaPN6bLP5tTb7bOpvr8Bp2oXSH0pl2uHyGoqEVmoqmn+jsMXalJfoGb1pyb1BWpWf6wvFefvWWTB6FV/B+BDNakvULP6U5P6AjWrP9aXCrIRjDHGGFfYCMYYY4wrLMEYY4xxRa1OMCIyXkT2ishPp7FvLxFZ4VSKfkFExOu5e0RkjVMh+p++jbrMeHzeFxF5TER2iMhS53aJ7yMvMyZXPhvn+QdFRJ1rr1zn0mfzhIgsdz6Xr0Skue8jLzUeN/rylPP7slxEPq5MNY+qcqk/Vzq/+0Ui4vpkgKr0oYzXK7WSfXm/V6Vycw50oN+Ac/BczPnTaew7H+iL51qeL4CLne3nAd8Akc7jJkHcl8eAh2rKZ+M8lwxMw3NRbuNg7QsQ59XmXuDfQdyXwUCYc/8fwD+C+ecM6Ai0B74D0gK1D058rUpsawhscv5t4NxvcKr+nupWq0cwqjoLOKE6gIi0EZEvRWSRiHwvIh1K7icizfD8gs9Tz//8W8BI5+k7gSdVNdd5j70l93eDS33xGxf78xzwGzzVIqqFG31R1SNeTWOopv641JevVLXAaToPT2moauFSf1ar6trqiN95v9PqQxlKrWR/un8nanWCKcOrwD2q2gt4CHiplDaJeOqoFfOu9twOOFtEfhSRmSLS29VoT62qfQG42zl0MV5EGrgXaoVUqT8iMgLYoarL3A60Aqr82YjIX0VkO3ANMBb/8cXPWbGb8Xw79idf9sdfKtKH0pRVyf60+uvqlfzBRkTqAv2BD70OL0ZW8mXC8Awv+wK9gQ9EpLWT9auNj/ryMvAEnm/HTwDP4PkDUO2q2h8RiQZ+h+dwjF/56LNBVX8P/F5EHgXuxrNIX7XyVV+c1/o9UAC865voTisGn/XHX07VBxG5Cfi1s60tMFVE8oDNqnqpr2OxBHOiEOCQqvbw3iie1TkXOQ+n4PnD6z2M9672nA585CSU+SJShKegXHUvWFPlvqjqHq/9XgM+czPgclS1P22AFGCZ80uXBCwWkT6qutvl2Evyxc+Zt3fxlGSq9gSDj/oiIjcCQ4Hzq/vLWAm+/mz8odQ+AKjqG8AbACLyHXCjqm7xarIDz/IpxZLwnKvZwen01+0TUIF+A1rhdXIMmANc6dwXoHsZ+5U84XWJs/0O4HHnfjs8w00J0r4082pzPzAhmD+bEm22UE0n+V36bFK92twDTAzivgzBU/A2vjp/vtz+OaOaTvKfbh8o+yT/Zjwn+Bs49xtWpL+lxuWPDzRQbsD/gF1APp6Rxy14vuV+CSxzfujHlrFvGvATnkrRL/JzVYQI4B3nucXAoCDuy9t4qlwvx/OtrVl19MWt/pRos4Xqm0Xmxmczydm+HE/hwsQg7ssGPF/Eljq3apkR52J/LnVeKxfYA0wLxD5QSoJxtpdayb4iv1clb1YqxhhjjCtsFpkxxhhXWIIxxhjjCkswxhhjXGEJxhhjjCsswRhjjHGFJRhTo4lIVjW/3xwfvc5AETksnmrJa0Tk6QrsM1JEOvni/Y3xBUswxlSCiJyy+oWq9vfh232vnquxewJDRWRAOe1HApZgTMCwBGNqnbIqzYrIMKdI6RIR+UZEEpztj4nI2yIyG3jbeTxeRL4TkU0icq/Xa2c5/w50np/ojEDeLV4/Q0QucbYtctbVOGUJHlU9hucCxOKinbeKyAIRWSYik0QkWkT6A8OBp5xRT5sqVNQ1xicswZjaqKxKsz8AfVW1JzABT1n/Yp2AC1R1jPO4A57S5n2AP4lIeCnv0xO4z9m3NTBARKKAV/CspdELiC8vWKeKdSowy9n0kar2VtXuwGrgFlWdg6fawsOq2kNVN56in8ZUCyt2aWqVcqrlJgHvO2tfROCpw1RsijOSKPa5etb8yRWRvUACJ5YzB5ivqunO+y7FUy8qC9ikqsWv/T/gtjLCPVtEluFJLs/rz0U5u4jIX4D6QF08C6hVpp/GVAtLMKa2KbPSLPAv4FlVnSIiA/Gs6Fksu0TbXK/7hZT+u1SRNqfyvaoOFZEUYJ6IfKCqS4E3gZGqusypQjywlH1P1U9jqoUdIjO1inpWgtwsIlcCiEd35+l6/FyC/IbS9veBtUBrEWnlPL6qvB2c0c6TwG+dTbHALuew3DVeTTOd58rrpzHVwhKMqemiRSTd6/YAnj/KtziHn1YCI5y2j+E5pLQI2OdGMM5htruAL533yQQOV2DXfwPnOInpj8CPwGxgjVebCcDDziSFNpTdT2OqhVVTNqaaiUhdVc1yZpWNA9ar6nP+jssYX7MRjDHV71bnpP9KPIflXvFzPMa4wkYwxhhjXGEjGGOMMa6wBGOMMcYVlmCMMca4whKMMcYYV1iCMcYY44r/B30uyn3SCAPrAAAAAElFTkSuQmCC", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light", + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "# Creating the learner\n", + "learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.5, metrics=[rmse])\n", + "# Loading the encoder we just saved\n", + "learn.load_encoder('ft_enc')\n", + "# Using lr_find to pick a learning rate:\n", + "learn.lr_find()\n", + "learn.recorder.plot(suggestion=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "ciDPSPJZrAap" + }, + "source": [ + "We could do this in stages, picking a learning rate at each stage, gradually unfreezing and training our model. But here I'll just do a rough first pass with some learning rates that are pretty much just guesses:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 516 + }, + "colab_type": "code", + "id": "wz0T8wtwrMoN", + "outputId": "6aa19b95-a15a-4a5a-a9de-ad1c882d6983" + }, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossroot_mean_squared_errortime
00.4089220.3882170.61851900:03
10.3729630.3766490.61000200:03
" + ], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossroot_mean_squared_errortime
00.3641280.3781040.61088800:04
10.3418240.3624470.59666900:04
20.3084110.3550830.59030900:04
" + ], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
epochtrain_lossvalid_lossroot_mean_squared_errortime
00.2898660.3527240.58799700:08
10.2794830.3546780.58967000:08
20.2638480.3562440.59041700:08
30.2325950.3488970.58464300:08
40.2095470.3537520.58806400:08
50.1903200.3499320.58465600:08
60.1742330.3563480.58963600:08
70.1656920.3604380.59311700:08
" + ], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + } + ], + "source": [ + "# Lots of room to optimize here\n", + "learn.fit_one_cycle(2, 1e-2)\n", + "learn.freeze_to(-2)\n", + "learn.fit_one_cycle(3, slice(5e-3/2., 5e-3))\n", + "learn.unfreeze()\n", + "learn.fit_one_cycle(8, slice(2e-3/100, 2e-3))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 204 + }, + "colab_type": "code", + "id": "xgKskOj0rdJ-", + "outputId": "f39a16d6-855d-4c7a-bc07-3778b488ce37" + }, + "outputs": [ + { + "data": { + "text/html": [], + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
tweet_idlabel
000BHHHP10.352108
100UNMD0E0.577899
201AXPTJF0.398042
301HOEQJW1.351928
401JUKMAO0.114119
\n", + "
" + ], + "text/plain": [ + " tweet_id label\n", + "0 00BHHHP1 0.352108\n", + "1 00UNMD0E 0.577899\n", + "2 01AXPTJF 0.398042\n", + "3 01HOEQJW 1.351928\n", + "4 01JUKMAO 0.114119" + ] + }, + "execution_count": 12, + "metadata": { + "tags": [] + }, + "output_type": "execute_result" + } + ], + "source": [ + "# Save and see how we do\n", + "preds, y = learn.get_preds(DatasetType.Test)\n", + "sub = pd.DataFrame({\n", + " 'tweet_id':test['tweet_id'],\n", + " 'label':[p[0] for p in preds.numpy()]\n", + "})\n", + "sub.to_csv('fastai_2nd_try_lm.csv', index=False)\n", + "sub.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "txpwWKc3rke1" + }, + "source": [ + "Now we're talking! By tuning the language model first, we get a learner more suited to our task and we end up scoring 0.588. This already puts us top 30 out of 250+ entrants, and this model is a single LSTM that can make predictions VERY quickly compared to the larger transformers used by the winning entrants. Note we spent almost no time training, guessed some numbers for lr etc, and basically just threw this together. I think that with a bit more time spent this could get a competitive model going. BUT transformer models are the rage, and so let's move on to trying some of those to see how much better we can get." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "MOFFs5M8sixg" + }, + "source": [ + "# 2) Transformers Assemble!\n", + "\n", + "You can start here if you want - this is independant from Section 1.\n", + "\n", + "Background on transformers... [I guess you can google it]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "c7xl_8FvsojM" + }, + "outputs": [], + "source": [ + "# Install the simpletransformers library:\n", + "# !pip install simpletransformers -q" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 142 + }, + "colab_type": "code", + "id": "VUb8ASkjtABi", + "outputId": "804bdd16-107e-4e54-ed27-2a9927753bcd" + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
tweet_idsafe_textlabelagreement
0CL1KWCMYMe &amp; The Big Homie meanboy3000 #MEANBOY #M...0.01.0
1E3303EMEI'm 100% thinking of devoting my career to pro...1.01.0
2M4IVFSMS#whatcausesautism VACCINES, DO NOT VACCINATE Y...-1.01.0
\n", + "
" + ], + "text/plain": [ + " tweet_id ... agreement\n", + "0 CL1KWCMY ... 1.0\n", + "1 E3303EME ... 1.0\n", + "2 M4IVFSMS ... 1.0\n", + "\n", + "[3 rows x 4 columns]" + ] + }, + "execution_count": 1, + "metadata": { + "tags": [] + }, + "output_type": "execute_result" + } + ], + "source": [ + "# Imports and load the data\n", + "import pandas as pd \n", + "from simpletransformers.classification import ClassificationModel\n", + "\n", + "train = pd.read_csv('Train.csv').dropna(0) # Read in train, ignoring one row with missing data\n", + "test = pd.read_csv('Test.csv').fillna('') # Read in test\n", + "test['label']=0 # We'll fill this in with predictions later\n", + "\n", + "# Get some local validation going\n", + "df_valid = train.sample(1000)\n", + "df_train = train.loc[~train.tweet_id.isin(df_valid.tweet_id.values)]\n", + "\n", + "train.head(3) # Remind ourselves what the data looks like" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "XJ2FkfCctcQ1" + }, + "source": [ + "Simpletransformers expects a dataframe with a text column and a label column. We tell it what model we want (there is a wide selection) and specify things like the number of epochs and the learning rate with a dictionary of arguments. This example comes pretty much vebatim from their readme on github..." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 601, + "referenced_widgets": [ + "7bfc92373971404c82c893f6110ee02f", + "06b1e0e6273044e7923923a9835f0bdb", + "8af370330541427db3bab98008001207", + "8dfd25a5820747559299ec4ab8ec7e48", + "6801510baeef42189b06c3190c88b22e", + "9eb71605fb1a46c987e830a6373f8931", + "1937a47f310c4e92a6c2cd931fe6dc9c", + "a6bf9b1468104506a9153cdb5d043a22", + "4d7e804c6097492dae849e9f6f17dfea", + "928db950860f4d7eb1244976e07340ec", + "c21e200a0eaa42559ecd60c9f43c6c41", + "a98245ca84b948ec9d93a1f56d2d93cc", + "2e817807da2e4d93ac8dc05ad4481c81", + "60e82ae75dff4eec978ef8821fa7ed59", + "a402f09ca15f48478ef3ebef9d8deede", + "84b959b97c494e119eb44dd729d494fc", + "a599f37980174e1fac1cf52979c245c0", + "99d0a0cb7263435f950f396778bcd7dc", + "7156685b440f4b9e8ebf08b7aadbd833", + "95ca19de8c9544ccb60664f695466c9f", + "eee873822e4940a0b57f10622c31436a", + "e9d48e11bae24b0185dab499900bcd0d", + "60ad1420aa434290aba6b1b1666cedce", + "bba22b3ab5304851aca8fc6e369de611", + "89d634f4c697419192bbca8ba1c7a00c", + "a7597d4690c247df8d405c5ac949d5da", + "bfced04a7c6c4192a03d77d4fa455b57", + "38602ece6def4541ae784224f7502e2c", + "88ab8cf638c3415baba06d3bdbf404df", + "347846c981364e81977e8e8507c07833", + "854581df260344b1bb8ccc05586e9882", + "7c6ce8c73c4b4d23bebc46f9afb55816", + "4dbbdf27eaff473e8e2b51e6325ea40f", + "8f32c914655242f5ba8d36a163dc87a7", + "ce8f2d8a8b67463d85993ca0aece6baa", + "ac446200ebee4256b013faaeb2b59fa9", + "b31517bc0bc247e7ac8a42e6f6a5381d", + "ff4132f18677435289b13e2db78bf44e", + "e38110db81274fc1a6b3513c8e3ff095", + "757dcc97d2404e39a3bcb258ea2f9277", + "8f26cf7b54af474b90feb0a033227bb2", + "b3d74d636c5446e48a996388040ff428", + "fd6de9c69886427ba37c9b1f1b248efa", + "8e3a8aa1a6cd4f1da417723697f17ba5", + "021cba019ed643a0918f06f4e4b05070", + "a159cc22f15c47a38754873ee0e4d9ca", + "a2669d9b06504677900f062e1874f51b", + "6b0728e3a5474e859ae4b9f58f0c5f0c", + "0749fef36bab4a8c8a2274874ee7d2e2", + "3846517229694c89834eacbf7dc611cd", + "7f3e59b13f2b45e2b758cddc2fa5fd96", + "4e825c21b2e44b6399fa7b4f77b02b96", + "280479ca5c4d4ff79f7bc50f82b022f4", + "68f56100fd4f4ac6ac094af47a5e6799", + "b8c2d92d57254af7af30d0e5ec7797c2", + "9025128f32f049b39ee238049971475e" + ] + }, + "colab_type": "code", + "id": "C3ka5BTssvRp", + "outputId": "a2acc75f-0ccb-4d9f-cd45-5ea89ed610f7" + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertForSequenceClassification: ['vocab_transform.weight', 'vocab_transform.bias', 'vocab_layer_norm.weight', 'vocab_layer_norm.bias', 'vocab_projector.weight', 'vocab_projector.bias']\n", + "- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).\n", + "- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", + "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['pre_classifier.weight', 'pre_classifier.bias', 'classifier.weight', 'classifier.bias']\n", + "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n", + "/usr/local/lib/python3.6/dist-packages/simpletransformers/classification/classification_model.py:275: UserWarning: Dataframe headers not specified. Falling back to using column 0 as text and column 1 as labels.\n", + " \"Dataframe headers not specified. Falling back to using column 0 as text and column 1 as labels.\"\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "7bfc92373971404c82c893f6110ee02f", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, max=8999.0), HTML(value='')))" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "4d7e804c6097492dae849e9f6f17dfea", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, description='Epoch', max=3.0, style=ProgressStyle(description_width='i…" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "a599f37980174e1fac1cf52979c245c0", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, description='Running Epoch 0 of 3', max=1125.0, style=ProgressStyle(de…" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.6/dist-packages/torch/optim/lr_scheduler.py:231: UserWarning: To get the last learning rate computed by the scheduler, please use `get_last_lr()`.\n", + " warnings.warn(\"To get the last learning rate computed by the scheduler, \"\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.6/dist-packages/torch/optim/lr_scheduler.py:200: UserWarning: Please also save or load the state of the optimzer when saving or loading the scheduler.\n", + " warnings.warn(SAVE_STATE_WARNING, UserWarning)\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "89d634f4c697419192bbca8ba1c7a00c", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, description='Running Epoch 1 of 3', max=1125.0, style=ProgressStyle(de…" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "4dbbdf27eaff473e8e2b51e6325ea40f", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, description='Running Epoch 2 of 3', max=1125.0, style=ProgressStyle(de…" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.6/dist-packages/simpletransformers/classification/classification_model.py:745: UserWarning: Dataframe headers not specified. Falling back to using column 0 as text and column 1 as labels.\n", + " \"Dataframe headers not specified. Falling back to using column 0 as text and column 1 as labels.\"\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "8f26cf7b54af474b90feb0a033227bb2", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "0749fef36bab4a8c8a2274874ee7d2e2", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, description='Running Evaluation', max=125.0, style=ProgressStyle(descr…" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "# Model arguments:\n", + "args = {\"reprocess_input_data\": True, \"overwrite_output_dir\": True, \n", + " 'fp16':False,\n", + " \"num_train_epochs\": 3,\n", + " \"learning_rate\": 1e-4,\n", + " \"max_seq_length\": 128, \n", + " 'regression': True} # Regression is this simple! \n", + "\n", + "# Just the text and labels\n", + "df_train = df_train[['safe_text', 'label']]\n", + "df_valid = df_valid[['safe_text', 'label']]\n", + "\n", + "# Create a ClassificationModel\n", + "model = ClassificationModel(\n", + " \"distilbert\", \"distilbert-base-uncased\",num_labels=1,args=args\n", + ")\n", + "\n", + "# Train the model\n", + "model.train_model(df_train)\n", + "\n", + "# Evaluate the model\n", + "result, model_outputs, wrong_predictions = model.eval_model(df_valid)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 34 + }, + "colab_type": "code", + "id": "QgxAA_rToj8_", + "outputId": "91d56796-3ca1-447b-eb7a-9907a429d7c2" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "0.5860418579448166" + ] + }, + "execution_count": 3, + "metadata": { + "tags": [] + }, + "output_type": "execute_result" + } + ], + "source": [ + "# RMSE scoring on our validation set:\n", + "from sklearn.metrics import mean_squared_error as skmse\n", + "skmse(df_valid['label'], model_outputs)**0.5" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 115, + "referenced_widgets": [ + "b9a456bb440a47d580bb57dde7e2a671", + "5087a1d42d5c4c4188014be3b3aaa822", + "8dff72db6a3a48ae811ae8c714070041", + "e1fea716e5da4cf2861ccd1179a3bd77", + "11281e73b4ee4822b748ef9f0484342a", + "7ef816fb26e44cd9832422a6afcdfdaf", + "b34f58920f74497e94a6f93f2c067a7f", + "5983754c3eff48979d40be82d165a507", + "c556108d7f654a6498b358f733775483", + "7e0d65d4f98741519a03ce63cfc91a77", + "56c41d26a1754a17b9757269658015cb", + "de36c2e593e24abba3374ae79df1b872", + "0bf46d6513a5495496042e39e27dfd51", + "fd13451cc9054116886d995daeb3e571", + "d15cc24c4be9496381f9125313d8b5c2", + "7c76f6d506c44cedbc191048bb97cda5" + ] + }, + "colab_type": "code", + "id": "hudTydcuv9A2", + "outputId": "b0dc8882-68e1-4c3b-f480-48f9b4894b3e" + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "b9a456bb440a47d580bb57dde7e2a671", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, max=5177.0), HTML(value='')))" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "c556108d7f654a6498b358f733775483", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "HBox(children=(FloatProgress(value=0.0, max=648.0), HTML(value='')))" + ] + }, + "metadata": { + "tags": [] + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n" + ] + } + ], + "source": [ + "# Saving Predictions\n", + "sub = pd.DataFrame({\n", + " 'tweet_id':test['tweet_id'],\n", + " 'label':model.predict(test['safe_text'].values)[0]\n", + "})\n", + "sub.to_csv('transfomer_1.csv', index=False)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "s5Bts95Ixb7K" + }, + "source": [ + "This scores ~0.6 on the LB. Again, we've picked an arbitrary lr and num_epocs based on our desire to try something quickly. Some ways to improve:\n", + "\n", + "\n", + "* Try a different model. Roberta seems to be the favourite\n", + "* Train a bit more.\n", + "* Ensemble! Why use just one model...\n", + "\n", + "Want to use fastai's goodness with transformers? You can mash the two together! Check out https://github.com/morganmcg1/fasthugs or seach around for examples. I think this will be the way to go to really nail this type of contest.\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "vOsYIvyjwpHU" + }, + "source": [ + "# 3) Ensemble Like The Cool Kids\n", + "\n", + "An ensemble built from a few different models can do better than any of its constituents. Let's take our three submissions and combine, paying more attention to the better ones, and submit that to see if it improves our situation..." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "3zmNtbcOexNr" + }, + "outputs": [], + "source": [ + "s1 = pd.read_csv('/content/first_try_fastai_20_epochs.csv')\n", + "s2 = pd.read_csv('/content/fastai_2nd_try_lm.csv')\n", + "s3 = pd.read_csv('/content/transfomer_1.csv')\n", + "sub = pd.DataFrame({\n", + " 'tweet_id':test['tweet_id'],\n", + " 'label':s1['label']*0.15 + s2['label']*0.35 + s3['label']*0.5\n", + "})\n", + "sub.to_csv('fancy_ensemble.csv', index=False)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "EVfYel8Mx-a2" + }, + "source": [ + "### 0.56! Just for combining three different sets of predictions!!! \n", + "\n", + "You can see why this is a popular approach. Imagine if we actually put effort into our models, and trained 5 or 10 different ones in different ways.... " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "CHbojwdGybFx" + }, + "source": [ + "# 4) Conclusion\n", + "\n", + "So, there are a few solutions to the challenge. Unsuprisingly, an ensemble of deep learning models does very well. But I'm always conflicted by this - in the real world, no-one wants to deplot 15 models if one simple one will do almost as well. A single well-tuned LSTM does decently - and while the competition framework encourages hoards of fancy transformers I really think that for most real problems that approach is overkill. \n", + "\n", + "Musings aside, I hope this is helpful :) Please let me know if you use this, and reach out if you have any questions. I am @johnowhitaker on Zindi, twitter, gmail, yahoo.... \n", + "\n", + "Happy Hacking :)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": {}, + "colab_type": "code", + "id": "P4pk01ZSgsXL" + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "accelerator": "GPU", + "colab": { + "collapsed_sections": [], + "name": "NLP Primer twitter challenge.ipynb", + "provenance": [], + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3.8.9 64-bit", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.9.6" + }, + "vscode": { + "interpreter": { + "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6" + } + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "021cba019ed643a0918f06f4e4b05070": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "06b1e0e6273044e7923923a9835f0bdb": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "0749fef36bab4a8c8a2274874ee7d2e2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_7f3e59b13f2b45e2b758cddc2fa5fd96", + "IPY_MODEL_4e825c21b2e44b6399fa7b4f77b02b96" + ], + "layout": "IPY_MODEL_3846517229694c89834eacbf7dc611cd" + } + }, + "0bf46d6513a5495496042e39e27dfd51": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "11281e73b4ee4822b748ef9f0484342a": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "1937a47f310c4e92a6c2cd931fe6dc9c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "280479ca5c4d4ff79f7bc50f82b022f4": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "2e817807da2e4d93ac8dc05ad4481c81": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "347846c981364e81977e8e8507c07833": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "3846517229694c89834eacbf7dc611cd": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "38602ece6def4541ae784224f7502e2c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7c6ce8c73c4b4d23bebc46f9afb55816", + "placeholder": "​", + "style": "IPY_MODEL_854581df260344b1bb8ccc05586e9882", + "value": " 1125/1125 [03:02<00:00, 6.18it/s]" + } + }, + "4d7e804c6097492dae849e9f6f17dfea": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_c21e200a0eaa42559ecd60c9f43c6c41", + "IPY_MODEL_a98245ca84b948ec9d93a1f56d2d93cc" + ], + "layout": "IPY_MODEL_928db950860f4d7eb1244976e07340ec" + } + }, + "4dbbdf27eaff473e8e2b51e6325ea40f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_ce8f2d8a8b67463d85993ca0aece6baa", + "IPY_MODEL_ac446200ebee4256b013faaeb2b59fa9" + ], + "layout": "IPY_MODEL_8f32c914655242f5ba8d36a163dc87a7" + } + }, + "4e825c21b2e44b6399fa7b4f77b02b96": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_9025128f32f049b39ee238049971475e", + "placeholder": "​", + "style": "IPY_MODEL_b8c2d92d57254af7af30d0e5ec7797c2", + "value": " 125/125 [00:04<00:00, 29.82it/s]" + } + }, + "5087a1d42d5c4c4188014be3b3aaa822": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "56c41d26a1754a17b9757269658015cb": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "100%", + "description_tooltip": null, + "layout": "IPY_MODEL_fd13451cc9054116886d995daeb3e571", + "max": 648, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_0bf46d6513a5495496042e39e27dfd51", + "value": 648 + } + }, + "5983754c3eff48979d40be82d165a507": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "60ad1420aa434290aba6b1b1666cedce": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "60e82ae75dff4eec978ef8821fa7ed59": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6801510baeef42189b06c3190c88b22e": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "68f56100fd4f4ac6ac094af47a5e6799": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6b0728e3a5474e859ae4b9f58f0c5f0c": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7156685b440f4b9e8ebf08b7aadbd833": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "Epochs 0/3. Running Loss: 0.8864: 100%", + "description_tooltip": null, + "layout": "IPY_MODEL_e9d48e11bae24b0185dab499900bcd0d", + "max": 1125, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_eee873822e4940a0b57f10622c31436a", + "value": 1125 + } + }, + "757dcc97d2404e39a3bcb258ea2f9277": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7bfc92373971404c82c893f6110ee02f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_8af370330541427db3bab98008001207", + "IPY_MODEL_8dfd25a5820747559299ec4ab8ec7e48" + ], + "layout": "IPY_MODEL_06b1e0e6273044e7923923a9835f0bdb" + } + }, + "7c6ce8c73c4b4d23bebc46f9afb55816": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7c76f6d506c44cedbc191048bb97cda5": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7e0d65d4f98741519a03ce63cfc91a77": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7ef816fb26e44cd9832422a6afcdfdaf": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "7f3e59b13f2b45e2b758cddc2fa5fd96": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "Running Evaluation: 100%", + "description_tooltip": null, + "layout": "IPY_MODEL_68f56100fd4f4ac6ac094af47a5e6799", + "max": 125, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_280479ca5c4d4ff79f7bc50f82b022f4", + "value": 125 + } + }, + "84b959b97c494e119eb44dd729d494fc": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "854581df260344b1bb8ccc05586e9882": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "88ab8cf638c3415baba06d3bdbf404df": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "89d634f4c697419192bbca8ba1c7a00c": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_bfced04a7c6c4192a03d77d4fa455b57", + "IPY_MODEL_38602ece6def4541ae784224f7502e2c" + ], + "layout": "IPY_MODEL_a7597d4690c247df8d405c5ac949d5da" + } + }, + "8af370330541427db3bab98008001207": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "100%", + "description_tooltip": null, + "layout": "IPY_MODEL_9eb71605fb1a46c987e830a6373f8931", + "max": 8999, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_6801510baeef42189b06c3190c88b22e", + "value": 8999 + } + }, + "8dfd25a5820747559299ec4ab8ec7e48": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_a6bf9b1468104506a9153cdb5d043a22", + "placeholder": "​", + "style": "IPY_MODEL_1937a47f310c4e92a6c2cd931fe6dc9c", + "value": " 8999/8999 [04:33<00:00, 32.85it/s]" + } + }, + "8dff72db6a3a48ae811ae8c714070041": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "100%", + "description_tooltip": null, + "layout": "IPY_MODEL_7ef816fb26e44cd9832422a6afcdfdaf", + "max": 5177, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_11281e73b4ee4822b748ef9f0484342a", + "value": 5177 + } + }, + "8e3a8aa1a6cd4f1da417723697f17ba5": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_6b0728e3a5474e859ae4b9f58f0c5f0c", + "placeholder": "​", + "style": "IPY_MODEL_a2669d9b06504677900f062e1874f51b", + "value": " 1000/1000 [00:05<00:00, 171.15it/s]" + } + }, + "8f26cf7b54af474b90feb0a033227bb2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_fd6de9c69886427ba37c9b1f1b248efa", + "IPY_MODEL_8e3a8aa1a6cd4f1da417723697f17ba5" + ], + "layout": "IPY_MODEL_b3d74d636c5446e48a996388040ff428" + } + }, + "8f32c914655242f5ba8d36a163dc87a7": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9025128f32f049b39ee238049971475e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "928db950860f4d7eb1244976e07340ec": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "95ca19de8c9544ccb60664f695466c9f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_bba22b3ab5304851aca8fc6e369de611", + "placeholder": "​", + "style": "IPY_MODEL_60ad1420aa434290aba6b1b1666cedce", + "value": " 1125/1125 [04:29<00:00, 4.17it/s]" + } + }, + "99d0a0cb7263435f950f396778bcd7dc": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "9eb71605fb1a46c987e830a6373f8931": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a159cc22f15c47a38754873ee0e4d9ca": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a2669d9b06504677900f062e1874f51b": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a402f09ca15f48478ef3ebef9d8deede": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "a599f37980174e1fac1cf52979c245c0": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_7156685b440f4b9e8ebf08b7aadbd833", + "IPY_MODEL_95ca19de8c9544ccb60664f695466c9f" + ], + "layout": "IPY_MODEL_99d0a0cb7263435f950f396778bcd7dc" + } + }, + "a6bf9b1468104506a9153cdb5d043a22": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a7597d4690c247df8d405c5ac949d5da": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "a98245ca84b948ec9d93a1f56d2d93cc": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_84b959b97c494e119eb44dd729d494fc", + "placeholder": "​", + "style": "IPY_MODEL_a402f09ca15f48478ef3ebef9d8deede", + "value": " 3/3 [04:29<00:00, 89.83s/it]" + } + }, + "ac446200ebee4256b013faaeb2b59fa9": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_757dcc97d2404e39a3bcb258ea2f9277", + "placeholder": "​", + "style": "IPY_MODEL_e38110db81274fc1a6b3513c8e3ff095", + "value": " 1125/1125 [01:33<00:00, 12.07it/s]" + } + }, + "b31517bc0bc247e7ac8a42e6f6a5381d": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "b34f58920f74497e94a6f93f2c067a7f": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "b3d74d636c5446e48a996388040ff428": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "b8c2d92d57254af7af30d0e5ec7797c2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "b9a456bb440a47d580bb57dde7e2a671": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_8dff72db6a3a48ae811ae8c714070041", + "IPY_MODEL_e1fea716e5da4cf2861ccd1179a3bd77" + ], + "layout": "IPY_MODEL_5087a1d42d5c4c4188014be3b3aaa822" + } + }, + "bba22b3ab5304851aca8fc6e369de611": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "bfced04a7c6c4192a03d77d4fa455b57": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "Epochs 1/3. Running Loss: 0.7346: 100%", + "description_tooltip": null, + "layout": "IPY_MODEL_347846c981364e81977e8e8507c07833", + "max": 1125, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_88ab8cf638c3415baba06d3bdbf404df", + "value": 1125 + } + }, + "c21e200a0eaa42559ecd60c9f43c6c41": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "Epoch 3 of 3: 100%", + "description_tooltip": null, + "layout": "IPY_MODEL_60e82ae75dff4eec978ef8821fa7ed59", + "max": 3, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_2e817807da2e4d93ac8dc05ad4481c81", + "value": 3 + } + }, + "c556108d7f654a6498b358f733775483": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_56c41d26a1754a17b9757269658015cb", + "IPY_MODEL_de36c2e593e24abba3374ae79df1b872" + ], + "layout": "IPY_MODEL_7e0d65d4f98741519a03ce63cfc91a77" + } + }, + "ce8f2d8a8b67463d85993ca0aece6baa": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "Epochs 2/3. Running Loss: 0.0219: 100%", + "description_tooltip": null, + "layout": "IPY_MODEL_ff4132f18677435289b13e2db78bf44e", + "max": 1125, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_b31517bc0bc247e7ac8a42e6f6a5381d", + "value": 1125 + } + }, + "d15cc24c4be9496381f9125313d8b5c2": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "de36c2e593e24abba3374ae79df1b872": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_7c76f6d506c44cedbc191048bb97cda5", + "placeholder": "​", + "style": "IPY_MODEL_d15cc24c4be9496381f9125313d8b5c2", + "value": " 648/648 [00:11<00:00, 57.16it/s]" + } + }, + "e1fea716e5da4cf2861ccd1179a3bd77": { + "model_module": "@jupyter-widgets/controls", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_5983754c3eff48979d40be82d165a507", + "placeholder": "​", + "style": "IPY_MODEL_b34f58920f74497e94a6f93f2c067a7f", + "value": " 5177/5177 [00:01<00:00, 2811.77it/s]" + } + }, + "e38110db81274fc1a6b3513c8e3ff095": { + "model_module": "@jupyter-widgets/controls", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "e9d48e11bae24b0185dab499900bcd0d": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "eee873822e4940a0b57f10622c31436a": { + "model_module": "@jupyter-widgets/controls", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "initial" + } + }, + "fd13451cc9054116886d995daeb3e571": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "fd6de9c69886427ba37c9b1f1b248efa": { + "model_module": "@jupyter-widgets/controls", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "100%", + "description_tooltip": null, + "layout": "IPY_MODEL_a159cc22f15c47a38754873ee0e4d9ca", + "max": 1000, + "min": 0, + "orientation": "horizontal", + "style": "IPY_MODEL_021cba019ed643a0918f06f4e4b05070", + "value": 1000 + } + }, + "ff4132f18677435289b13e2db78bf44e": { + "model_module": "@jupyter-widgets/base", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + } + } + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}