{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gq9-Z9DSkT14"
      },
      "source": [
        "In this notebook, I show how to fine-tune a NLLB-200 machine translation model for a new language.\n",
        "\n",
        "The new language will be [Tyvan](https://en.wikipedia.org/wiki/Tuvan_language), and I will use a Tyvan-Russian parallel corpus as the training data.\n",
        "\n",
        "I am running this notebook on Google Colab with a T4 GPU that has 15Gb of memory. If you run it elsewhere, you may want to adjust the batch size, so that there are no OOM errors, but the GPU is well utilized."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_iBrOtwcjnml"
      },
      "source": [
        "# 0. Preliminaries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CZgBsg-Xjpu8"
      },
      "source": [
        "I run this notebook in Google Colab (which is ephemeral), and to read the dataset and to write the resulting model I use Google Drive, which I mount in the cell below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "XuO8k6JIwK5c"
      },
      "outputs": [],
      "source": [
        "# from google.colab import drive\n",
        "# import os\n",
        "# if not os.path.exists('/gd'):\n",
        "#     drive.mount('/gd')"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Kt2LZHZ41oTC",
        "outputId": "488ef1dd-bd31-4519-bed5-11f29ed0b746"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "os.chdir('/content/drive/MyDrive/MT_Research')"
      ],
      "metadata": {
        "id": "euW7z2h_2EQy"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dc8NcXYHj2Zj"
      },
      "source": [
        "Installing dependencies:\n",
        "* `transformers`, as a neural network framework\n",
        "* `sentencepiece`, a backend for my tokenizer (the algorithm for converting a text into symbols from the model's vocabulary)\n",
        "* `sacremoses`, a package required for text preprocessing with which NLLB models were pretrained.\n",
        "* `sacrebleu`, a package for evaluating translation models"
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "rlfoDgf-16XW"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# import locale\n",
        "# def gpe(x=None):\n",
        "#     return \"UTF-8\"\n",
        "# locale.getpreferredencoding = gpe"
      ],
      "metadata": {
        "id": "qPjx54id5ko8"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xu8BrYo292Nx",
        "outputId": "60fe97e6-9f0b-41d7-c9f9-9152c140b54c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m119.9/119.9 kB\u001b[0m \u001b[31m7.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.0/58.0 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.6/7.6 MB\u001b[0m \u001b[31m60.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m547.8/547.8 kB\u001b[0m \u001b[31m26.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m897.5/897.5 kB\u001b[0m \u001b[31m43.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m106.7/106.7 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.3/116.3 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m39.9/39.9 MB\u001b[0m \u001b[31m22.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m64.9/64.9 kB\u001b[0m \u001b[31m4.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m58.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m14.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "cudf-cu12 24.4.1 requires pyarrow<15.0.0a0,>=14.0.1, but you have pyarrow 17.0.0 which is incompatible.\n",
            "google-colab 1.0.0 requires requests==2.31.0, but you have requests 2.32.3 which is incompatible.\n",
            "ibis-framework 8.0.0 requires pyarrow<16,>=2, but you have pyarrow 17.0.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0m"
          ]
        }
      ],
      "source": [
        "!pip install sentencepiece transformers==4.33 datasets sacremoses sacrebleu  -q"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OqdSSIVLlCir"
      },
      "source": [
        "# 1. Exploring the data\n",
        "\n",
        "In this section, I try to understand what is the training data that I have, and how suitable it is for fine-tuning a NLLB model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "CTyDFaZf984A"
      },
      "outputs": [],
      "source": [
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "A6zKA3Fd-M39",
        "outputId": "2161868a-bcd6-4ae4-b966-d48eebcd75c9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(15021, 2)\n",
            "Index(['English', 'Yoruba'], dtype='object')\n"
          ]
        }
      ],
      "source": [
        "\n",
        "train = pd.read_csv('train.csv')\n",
        "test= pd.read_csv('test.csv',)\n",
        "print(train.shape)\n",
        "print(train.columns)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pd.options.display.max_colwidth = 100"
      ],
      "metadata": {
        "id": "VrJAf8LlfU8S"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(test.shape)\n",
        "print(test.columns)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ul5Q3UON4ym4",
        "outputId": "52bbcbc6-3bcc-4881-8e87-d4829013bf2b"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(1669, 2)\n",
            "Index(['English', 'Yoruba'], dtype='object')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "J6ZCvaV1lkkd"
      },
      "outputs": [],
      "source": [
        "# trans_df.sample(10)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "train.isnull().sum()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XKQJ1VPEQ8Kt",
        "outputId": "24ef5ba5-7ff1-44e7-af55-b82ff55b6f39"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "English    0\n",
              "Yoruba     0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "3sNJergv-ap2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d431d280-c41b-4581-d8ed-bbf15d953459"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "English    0\n",
              "Yoruba     0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "test.isnull().sum()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "nKH5EGKY-c0q"
      },
      "outputs": [],
      "source": [
        "# df_train = trans_df[trans_df.split=='train'].copy() # 49000 items\n",
        "# df_dev = trans_df[trans_df.split=='dev'].copy()     # 500 items\n",
        "# df_test = trans_df[trans_df.split=='test'].copy()   # 500 items"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Data Cleaning"
      ],
      "metadata": {
        "id": "nFXYtDweKmlr"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K6qHP-DAA4YD"
      },
      "source": [
        "# 2. How well does the data fit into a NLLB tokenizer?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "2xL261VQtyLl"
      },
      "outputs": [],
      "source": [
        "from transformers import NllbTokenizer\n",
        "from tqdm.auto import tqdm, trange"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "05GfWpzKtvcz",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 272,
          "referenced_widgets": [
            "a3cce029e48b451ab7e3b3cd6de9fbb5",
            "df100e6cd14344d9850db922a5e13450",
            "48c6fab6b4c6428abcc5208231fbe054",
            "caa478c43f7c4034b483a2e2697265fb",
            "e99da02112534d7c9a1eac3bb414ece1",
            "ff797d41acda4a5f930c379d86514042",
            "5a0058193e6e4ad19169f63a65f3bf51",
            "26469d725ff040279e89d6553ca51218",
            "ff9672ff7b154d1f811984b723c497d1",
            "fa16d40023fe4587986cfee853426a48",
            "665cc2bf3da6485aac757b3db5e0472c",
            "a092f5d3de9b4de183c6c24bbb94fd61",
            "0b3e1204ef4049e492b98f6a04f23800",
            "40bdb135046f4cdea1bfca440fec082b",
            "1fdff1ba87df48338d0d5eb3b47ec3a0",
            "9deec01ab0ce44d890a3195b8ae2a3ab",
            "45ffe8bd2c21400187966e6fa7fb1aa9",
            "ce35a79a4256487f9c8129cb88696fc1",
            "7b74fe6ab70a4dfd8ca86fc41d4aa11c",
            "0b3ac9ae5cec4aefbd73834d3e5c6e7e",
            "c999c707f6174d0f8d1d7c4488dc06eb",
            "2fad97512f17469aa5c6ed607e72247b",
            "23f7497f85ee46d5ba558b61c883b431",
            "2fea688288aa4b0e9ce4b9a3d26d2d6a",
            "917536ea52a84e1ea2e708a5786bc79d",
            "debb6444da7c4410b2a137fb2f2db62f",
            "b465d4f10c10450baa1ed660487b3929",
            "7ced34a412a74e0ea53e07e6624d3b85",
            "0492f2a503284dbdb105b8c443dc53b0",
            "fd5e94baf2e445e89e6ea77f14e22647",
            "f5674c11b43e4eec9964626fb3ba60e8",
            "e1af2035af3c4c2dba8ae5da974978e9",
            "08f92e34af284860a3d72c6c24899aec"
          ]
        },
        "outputId": "e37a9422-4f71-4504-ccea-92e7d40729c8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:89: UserWarning: \n",
            "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
            "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
            "You will be able to reuse this secret in all of your notebooks.\n",
            "Please note that authentication is recommended but still optional to access public models or datasets.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "sentencepiece.bpe.model:   0%|          | 0.00/4.85M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a3cce029e48b451ab7e3b3cd6de9fbb5"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "special_tokens_map.json:   0%|          | 0.00/3.55k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a092f5d3de9b4de183c6c24bbb94fd61"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/564 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "23f7497f85ee46d5ba558b61c883b431"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "tokenizer = NllbTokenizer.from_pretrained('facebook/nllb-200-distilled-600M')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "NQywlyv7t9VH"
      },
      "outputs": [],
      "source": [
        "import re\n",
        "\n",
        "def word_tokenize(text):\n",
        "    # a very naive word tokenizer for languages with English-like orthography\n",
        "    return re.findall('(\\w+|[^\\w\\s])', text)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "QzD0htfzuAPu"
      },
      "outputs": [],
      "source": [
        "smpl = train.sample(5000, random_state=1)\n",
        "\n",
        "smpl['eng_toks'] = smpl.English.apply(tokenizer.tokenize)\n",
        "smpl['yor_toks'] = smpl.Yoruba.apply(tokenizer.tokenize)\n",
        "\n",
        "smpl['eng_words'] = smpl.English.apply(word_tokenize)\n",
        "smpl['yor_words'] = smpl.Yoruba.apply(word_tokenize)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 466
        },
        "id": "TrDHIgCwuHeN",
        "outputId": "2f9a2a40-7ba5-460b-8edb-86a37e75e95d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                                                                    Yoruba  \\\n",
              "14800  Ó dára láti “ya” owó láti inu o kò-owó rẹ, ṣugbọn ní gbàti o bá ṣe eyi, ri daju pe o gbá  èlé lo...   \n",
              "4960   Ìwé àkọsílẹ̀ ìṣura ni: àṣùwọ̀n owó tí a ní sílẹ̀ àti owó tí a fi léde tí a fi sínu owó tí ó wà n...   \n",
              "7556   Moniepoint\\nTí a mọ tẹlẹ̀ sí TeamApt, Moniepoint tún jẹ ilé-ìfowópamọ́ ìnáwó kékeré tí ó ní ìwé ...   \n",
              "10467                              Ìjọba le fún wọn ní àkókò nítorí èyí lè gbà tó ọdún mẹ́ta láti ṣe yọrí.   \n",
              "313    Iṣẹ́ rere náà wá ní ẹ̀yìn ọdún iye owó ńlá níbi tí àwọn okòwò àti àwọn oníbàárà wọn ti ní láti k...   \n",
              "\n",
              "                                                                                                 yor_words  \\\n",
              "14800  [Ó, dára, láti, “, ya, ”, owó, láti, inu, o, kò, -, owó, rẹ, ,, ṣugbọn, ní, gbàti, o, bá, ṣe, ey...   \n",
              "4960   [Ìwé, àkọsílè, ̣, ìṣura, ni, :, àṣùwò, ̣, n, owó, tí, a, ní, sílè, ̣, àti, owó, tí, a, fi, léde,...   \n",
              "7556   [Moniepoint, Tí, a, mọ, tẹlẹ, ̀, sí, TeamApt, ,, Moniepoint, tún, jẹ, ilé, -, ìfowópamọ, ́, ìnáw...   \n",
              "10467       [Ìjọba, le, fún, wọn, ní, àkókò, nítorí, èyí, lè, gbà, tó, ọdún, mẹ, ́, ta, láti, ṣe, yọrí, .]   \n",
              "313    [Iṣẹ, ́, rere, náà, wá, ní, ẹ, ̀, yìn, ọdún, iye, owó, ńlá, níbi, tí, àwọn, okòwò, àti, àwọn, on...   \n",
              "\n",
              "                                                                                                  yor_toks  \\\n",
              "14800  [▁Ó, ▁dára, ▁láti, ▁, “, ya, ”, ▁owó, ▁láti, ▁inu, ▁o, ▁kò, -, o, wó, ▁rẹ, ,, ▁ṣugbọn, ▁ní, ▁gbà...   \n",
              "4960   [▁Ìwé, ▁àkọ, sí, lè, ̣, ▁ìṣ, ura, ▁ni, :, ▁à, ṣù, wò, ̣, n, ▁owó, ▁tí, ▁a, ▁ní, ▁sílè, ̣, ▁àti, ...   \n",
              "7556   [▁Mon, ie, point, ▁Tí, ▁a, ▁mọ, ▁tẹlẹ, ̀, ▁sí, ▁Team, A, pt, ,, ▁Mon, ie, point, ▁tún, ▁jẹ, ▁ilé...   \n",
              "10467  [▁Ìjọba, ▁le, ▁fún, ▁wọn, ▁ní, ▁àkókò, ▁nítorí, ▁èyí, ▁lè, ▁gbà, ▁tó, ▁ọdún, ▁mẹ, ́, ta, ▁láti, ...   \n",
              "313    [▁I, ṣẹ, ́, ▁rere, ▁náà, ▁wá, ▁ní, ▁ẹ, ̀, yìn, ▁ọdún, ▁iye, ▁owó, ▁ńlá, ▁níbi, ▁tí, ▁àwọn, ▁ok, ...   \n",
              "\n",
              "                                                                                                   English  \\\n",
              "14800             It is ok to “borrow” cash from your business, but when you do, charge yourself interest.   \n",
              "4960                             The journal entry is:   Capital/Drawings Account   Dr   To Cash/Bank   15   \n",
              "7556   Moniepoint\\nFormerly known as TeamApt, Moniepoint is also a CBN-licensed microfinance bank deliv...   \n",
              "10467     The government can give them a timeline because this could take up to three years to accomplish.   \n",
              "313    The positive performance came on the back of an inflationary year where businesses and their con...   \n",
              "\n",
              "                                                                                                 eng_words  \\\n",
              "14800  [It, is, ok, to, “, borrow, ”, cash, from, your, business, ,, but, when, you, do, ,, charge, you...   \n",
              "4960                [The, journal, entry, is, :, Capital, /, Drawings, Account, Dr, To, Cash, /, Bank, 15]   \n",
              "7556   [Moniepoint, Formerly, known, as, TeamApt, ,, Moniepoint, is, also, a, CBN, -, licensed, microfi...   \n",
              "10467  [The, government, can, give, them, a, timeline, because, this, could, take, up, to, three, years...   \n",
              "313    [The, positive, performance, came, on, the, back, of, an, inflationary, year, where, businesses,...   \n",
              "\n",
              "                                                                                                  eng_toks  \n",
              "14800  [▁It, ▁is, ▁ok, ▁to, ▁, “, bor, row, ”, ▁cash, ▁from, ▁your, ▁business, ,, ▁but, ▁when, ▁you, ▁d...  \n",
              "4960   [▁The, ▁journal, ▁entry, ▁is, :, ▁Capital, /, D, raw, ings, ▁Account, ▁Dr, ▁To, ▁Cash, /, Bank, ...  \n",
              "7556   [▁Mon, ie, point, ▁For, mer, ly, ▁known, ▁as, ▁Team, A, pt, ,, ▁Mon, ie, point, ▁is, ▁also, ▁a, ...  \n",
              "10467  [▁The, ▁government, ▁can, ▁give, ▁them, ▁a, ▁tim, eline, ▁because, ▁this, ▁could, ▁take, ▁up, ▁t...  \n",
              "313    [▁The, ▁positive, ▁performance, ▁came, ▁on, ▁the, ▁back, ▁of, ▁an, ▁inf, lation, ary, ▁year, ▁wh...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-34ea596c-cc9f-47cf-adbc-f2dddb517558\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Yoruba</th>\n",
              "      <th>yor_words</th>\n",
              "      <th>yor_toks</th>\n",
              "      <th>English</th>\n",
              "      <th>eng_words</th>\n",
              "      <th>eng_toks</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>14800</th>\n",
              "      <td>Ó dára láti “ya” owó láti inu o kò-owó rẹ, ṣugbọn ní gbàti o bá ṣe eyi, ri daju pe o gbá  èlé lo...</td>\n",
              "      <td>[Ó, dára, láti, “, ya, ”, owó, láti, inu, o, kò, -, owó, rẹ, ,, ṣugbọn, ní, gbàti, o, bá, ṣe, ey...</td>\n",
              "      <td>[▁Ó, ▁dára, ▁láti, ▁, “, ya, ”, ▁owó, ▁láti, ▁inu, ▁o, ▁kò, -, o, wó, ▁rẹ, ,, ▁ṣugbọn, ▁ní, ▁gbà...</td>\n",
              "      <td>It is ok to “borrow” cash from your business, but when you do, charge yourself interest.</td>\n",
              "      <td>[It, is, ok, to, “, borrow, ”, cash, from, your, business, ,, but, when, you, do, ,, charge, you...</td>\n",
              "      <td>[▁It, ▁is, ▁ok, ▁to, ▁, “, bor, row, ”, ▁cash, ▁from, ▁your, ▁business, ,, ▁but, ▁when, ▁you, ▁d...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4960</th>\n",
              "      <td>Ìwé àkọsílẹ̀ ìṣura ni: àṣùwọ̀n owó tí a ní sílẹ̀ àti owó tí a fi léde tí a fi sínu owó tí ó wà n...</td>\n",
              "      <td>[Ìwé, àkọsílè, ̣, ìṣura, ni, :, àṣùwò, ̣, n, owó, tí, a, ní, sílè, ̣, àti, owó, tí, a, fi, léde,...</td>\n",
              "      <td>[▁Ìwé, ▁àkọ, sí, lè, ̣, ▁ìṣ, ura, ▁ni, :, ▁à, ṣù, wò, ̣, n, ▁owó, ▁tí, ▁a, ▁ní, ▁sílè, ̣, ▁àti, ...</td>\n",
              "      <td>The journal entry is:   Capital/Drawings Account   Dr   To Cash/Bank   15</td>\n",
              "      <td>[The, journal, entry, is, :, Capital, /, Drawings, Account, Dr, To, Cash, /, Bank, 15]</td>\n",
              "      <td>[▁The, ▁journal, ▁entry, ▁is, :, ▁Capital, /, D, raw, ings, ▁Account, ▁Dr, ▁To, ▁Cash, /, Bank, ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7556</th>\n",
              "      <td>Moniepoint\\nTí a mọ tẹlẹ̀ sí TeamApt, Moniepoint tún jẹ ilé-ìfowópamọ́ ìnáwó kékeré tí ó ní ìwé ...</td>\n",
              "      <td>[Moniepoint, Tí, a, mọ, tẹlẹ, ̀, sí, TeamApt, ,, Moniepoint, tún, jẹ, ilé, -, ìfowópamọ, ́, ìnáw...</td>\n",
              "      <td>[▁Mon, ie, point, ▁Tí, ▁a, ▁mọ, ▁tẹlẹ, ̀, ▁sí, ▁Team, A, pt, ,, ▁Mon, ie, point, ▁tún, ▁jẹ, ▁ilé...</td>\n",
              "      <td>Moniepoint\\nFormerly known as TeamApt, Moniepoint is also a CBN-licensed microfinance bank deliv...</td>\n",
              "      <td>[Moniepoint, Formerly, known, as, TeamApt, ,, Moniepoint, is, also, a, CBN, -, licensed, microfi...</td>\n",
              "      <td>[▁Mon, ie, point, ▁For, mer, ly, ▁known, ▁as, ▁Team, A, pt, ,, ▁Mon, ie, point, ▁is, ▁also, ▁a, ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10467</th>\n",
              "      <td>Ìjọba le fún wọn ní àkókò nítorí èyí lè gbà tó ọdún mẹ́ta láti ṣe yọrí.</td>\n",
              "      <td>[Ìjọba, le, fún, wọn, ní, àkókò, nítorí, èyí, lè, gbà, tó, ọdún, mẹ, ́, ta, láti, ṣe, yọrí, .]</td>\n",
              "      <td>[▁Ìjọba, ▁le, ▁fún, ▁wọn, ▁ní, ▁àkókò, ▁nítorí, ▁èyí, ▁lè, ▁gbà, ▁tó, ▁ọdún, ▁mẹ, ́, ta, ▁láti, ...</td>\n",
              "      <td>The government can give them a timeline because this could take up to three years to accomplish.</td>\n",
              "      <td>[The, government, can, give, them, a, timeline, because, this, could, take, up, to, three, years...</td>\n",
              "      <td>[▁The, ▁government, ▁can, ▁give, ▁them, ▁a, ▁tim, eline, ▁because, ▁this, ▁could, ▁take, ▁up, ▁t...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>313</th>\n",
              "      <td>Iṣẹ́ rere náà wá ní ẹ̀yìn ọdún iye owó ńlá níbi tí àwọn okòwò àti àwọn oníbàárà wọn ti ní láti k...</td>\n",
              "      <td>[Iṣẹ, ́, rere, náà, wá, ní, ẹ, ̀, yìn, ọdún, iye, owó, ńlá, níbi, tí, àwọn, okòwò, àti, àwọn, on...</td>\n",
              "      <td>[▁I, ṣẹ, ́, ▁rere, ▁náà, ▁wá, ▁ní, ▁ẹ, ̀, yìn, ▁ọdún, ▁iye, ▁owó, ▁ńlá, ▁níbi, ▁tí, ▁àwọn, ▁ok, ...</td>\n",
              "      <td>The positive performance came on the back of an inflationary year where businesses and their con...</td>\n",
              "      <td>[The, positive, performance, came, on, the, back, of, an, inflationary, year, where, businesses,...</td>\n",
              "      <td>[▁The, ▁positive, ▁performance, ▁came, ▁on, ▁the, ▁back, ▁of, ▁an, ▁inf, lation, ary, ▁year, ▁wh...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-34ea596c-cc9f-47cf-adbc-f2dddb517558')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-34ea596c-cc9f-47cf-adbc-f2dddb517558 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-34ea596c-cc9f-47cf-adbc-f2dddb517558');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-4edf5816-42d3-44cf-8b6c-28bc20e39765\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4edf5816-42d3-44cf-8b6c-28bc20e39765')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-4edf5816-42d3-44cf-8b6c-28bc20e39765 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"smpl\",\n  \"rows\": 5,\n  \"fields\": [\n    {\n      \"column\": \"Yoruba\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"\\u00ccw\\u00e9 \\u00e0k\\u1ecds\\u00edl\\u00e8\\u0323 \\u00ec\\u1e63ura ni: \\u00e0\\u1e63\\u00f9w\\u00f2\\u0323n ow\\u00f3 t\\u00ed a n\\u00ed s\\u00edl\\u00e8\\u0323 \\u00e0ti ow\\u00f3 t\\u00ed a fi l\\u00e9de t\\u00ed a fi s\\u00ednu ow\\u00f3 t\\u00ed \\u00f3 w\\u00e0 n\\u00edl\\u00e9 \\u00ecfow\\u00f3pam\\u00f3\\u0323\",\n          \"I\\u1e63\\u1eb9\\u0301 rere n\\u00e1\\u00e0 w\\u00e1 n\\u00ed \\u1eb9\\u0300y\\u00ecn \\u1ecdd\\u00fan iye ow\\u00f3 \\u0144l\\u00e1 n\\u00edbi t\\u00ed \\u00e0w\\u1ecdn ok\\u00f2w\\u00f2 \\u00e0ti \\u00e0w\\u1ecdn on\\u00edb\\u00e0\\u00e1r\\u00e0 w\\u1ecdn ti n\\u00ed l\\u00e1ti koj\\u00fa iye ow\\u00f3 \\u1ecdj\\u00e0 \\u00e0ti i\\u1e63\\u1eb9\\u0301 t\\u00f3 \\u0144 p\\u1ecd\\u0300 s\\u00ed i. \",\n          \"Moniepoint\\nT\\u00ed a m\\u1ecd t\\u1eb9l\\u1eb9\\u0300 s\\u00ed TeamApt, Moniepoint t\\u00fan j\\u1eb9 il\\u00e9-\\u00ecfow\\u00f3pam\\u1ecd\\u0301 \\u00ecn\\u00e1w\\u00f3 k\\u00e9ker\\u00e9 t\\u00ed \\u00f3 n\\u00ed \\u00ecw\\u00e9 \\u00e0\\u1e63\\u1eb9 CBN t\\u00ed \\u00f3 p\\u00e8s\\u00e8 \\u00e0w\\u1ecdn i\\u1e63\\u1eb9 ow\\u00f3 n\\u00ed on\\u00ed-n\\u1ecd\\u0301m\\u0301b\\u00e0\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"yor_words\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"yor_toks\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"English\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"The journal entry is:   Capital/Drawings Account   Dr   To Cash/Bank   15\",\n          \"The positive performance came on the back of an inflationary year where businesses and their consumers have had to deal with the rising cost of goods and services.\",\n          \"Moniepoint\\nFormerly known as TeamApt, Moniepoint is also a CBN-licensed microfinance bank delivering financial services digitally.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"eng_words\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"eng_toks\",\n      \"properties\": {\n        \"dtype\": \"object\",\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "smpl.sample(5)[['Yoruba', 'yor_words', 'yor_toks', 'English', 'eng_words', 'eng_toks']]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "EbgRYDlTuC9z",
        "outputId": "9b1f8e90-80f0-469e-ccbc-f24e31a61c66"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          eng_toks    yor_toks    eng_words    yor_words\n",
              "count  5000.000000  5000.00000  5000.000000  5000.000000\n",
              "mean     26.342200    47.60500    22.071400    38.320000\n",
              "std       9.735443    22.63276     8.095913    24.581986\n",
              "min       1.000000     3.00000     1.000000     1.000000\n",
              "25%      20.000000    33.00000    17.000000    24.000000\n",
              "50%      26.000000    44.00000    21.000000    33.000000\n",
              "75%      32.000000    58.00000    27.000000    45.000000\n",
              "max     153.000000   297.00000   123.000000   413.000000"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-80eb54f6-6587-4d4d-b3af-e18669e6cde5\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>eng_toks</th>\n",
              "      <th>yor_toks</th>\n",
              "      <th>eng_words</th>\n",
              "      <th>yor_words</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>5000.000000</td>\n",
              "      <td>5000.00000</td>\n",
              "      <td>5000.000000</td>\n",
              "      <td>5000.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>26.342200</td>\n",
              "      <td>47.60500</td>\n",
              "      <td>22.071400</td>\n",
              "      <td>38.320000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>9.735443</td>\n",
              "      <td>22.63276</td>\n",
              "      <td>8.095913</td>\n",
              "      <td>24.581986</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>3.00000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>20.000000</td>\n",
              "      <td>33.00000</td>\n",
              "      <td>17.000000</td>\n",
              "      <td>24.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>26.000000</td>\n",
              "      <td>44.00000</td>\n",
              "      <td>21.000000</td>\n",
              "      <td>33.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>32.000000</td>\n",
              "      <td>58.00000</td>\n",
              "      <td>27.000000</td>\n",
              "      <td>45.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>153.000000</td>\n",
              "      <td>297.00000</td>\n",
              "      <td>123.000000</td>\n",
              "      <td>413.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-80eb54f6-6587-4d4d-b3af-e18669e6cde5')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-80eb54f6-6587-4d4d-b3af-e18669e6cde5 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-80eb54f6-6587-4d4d-b3af-e18669e6cde5');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-c6e1caf7-97c8-4823-bb2a-bc8e26e4ad6e\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c6e1caf7-97c8-4823-bb2a-bc8e26e4ad6e')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-c6e1caf7-97c8-4823-bb2a-bc8e26e4ad6e button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_4237ee97-e2cb-45a7-86d9-213c03d18049\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('stats')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_4237ee97-e2cb-45a7-86d9-213c03d18049 button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('stats');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "stats",
              "summary": "{\n  \"name\": \"stats\",\n  \"rows\": 8,\n  \"fields\": [\n    {\n      \"column\": \"eng_toks\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1754.8796634960947,\n        \"min\": 1.0,\n        \"max\": 5000.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          26.3422,\n          26.0,\n          5000.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"yor_toks\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1744.7443582140572,\n        \"min\": 3.0,\n        \"max\": 5000.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          47.605,\n          44.0,\n          5000.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"eng_words\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1757.1152600657247,\n        \"min\": 1.0,\n        \"max\": 5000.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          22.0714,\n          21.0,\n          5000.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"yor_words\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 1743.7977550957332,\n        \"min\": 1.0,\n        \"max\": 5000.0,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          38.32,\n          33.0,\n          5000.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "stats = smpl[['eng_toks', 'yor_toks', 'eng_words', 'yor_words']].applymap(len).describe()\n",
        "stats"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WUJQQzYDuEc5",
        "outputId": "ee702df0-e960-48d0-f869-95cdde55f9fe"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "1.1934992796107178\n",
            "1.2423016701461378\n"
          ]
        }
      ],
      "source": [
        "print(stats.eng_toks['mean'] / stats.eng_words['mean'])\n",
        "print(stats.yor_toks['mean'] / stats.yor_words['mean'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iUXEaJlbuqJf",
        "outputId": "6961ff9c-190f-44a6-e2d4-d525c1dc7549"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<unk> 3\n"
          ]
        }
      ],
      "source": [
        "print(tokenizer.unk_token, tokenizer.unk_token_id)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RdLDZ9L6uGin"
      },
      "source": [
        "Good news: both for Russian and Tyvan, the NLLB tokenizer seems to produce around 2 tokens per word (more precisely, 2.3 and 1.8), which means that the translation quality of fine-tuned NLLB may be decent even without vocabulary extension."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "27BIJ7HGvKs-"
      },
      "source": [
        "One more check: how often does the <unk> token happen in the tokenizer output for Tyvan? If this is too often, we need to fix it somehow"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "38a8a1febe7a4c15be012c046d943b7b",
            "363bc9e667054eb1b3697e627c87373f",
            "04ab7ee6804a4faab6eb88a91915ea88",
            "4506edf6e191456b833683bdcf99ebd5",
            "5ef9c560df0a46eb995d1ecba63ac8c5",
            "ec638390ce3a4a509ad29a8d8c71b63e",
            "301bff8171df4572bc8ec9f969110a69",
            "75ca7cf36acc4ef4ab29217dae170984",
            "e849b6a3c8bb4b73ab742c8db6ad2323",
            "5dee707988dc4a1b885c785f1c5fdaab",
            "aa8d19e5f70d4a5a98ac828a4c5eb55f"
          ]
        },
        "id": "nAEe9lYNu6kv",
        "outputId": "0d6cda77-1cec-45ed-c445-61df348965ee"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/15021 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "38a8a1febe7a4c15be012c046d943b7b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "389\n"
          ]
        }
      ],
      "source": [
        "texts_with_unk = [text for text in tqdm(train.Yoruba) if tokenizer.unk_token_id in tokenizer(text).input_ids]\n",
        "print(len(texts_with_unk))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qt5d16TnvSJt",
        "outputId": "f33a7f37-625d-4b89-92ab-49d49472fa6e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['Ewu – àyè láti ni ìrírí ìpàdánù owo',\n",
              " 'Ní ọdún 2022, FIRS kéde pé òun rí tírílíọ̀nù mẹ́wàá lé ní mílíọ̀nù kan náírà (₦10.1 million) gẹ́gẹ́ bí owó tó ń wọlé, èyí tó lọ sókè jù nínú ìtàn orílẹ̀ èdè náà.',\n",
              " 'A ti “ṣe ipele ” idokòowó rẹ ki idokòowó akọkọ rẹ le máà ní  èrè ní  gbogbo ọdún fún ọdun marun to nbọ.',\n",
              " 'Akojọpọ owo ti ilé iṣé náà pa wọlé lórí ìpín ìdókòwò kan gbéra sọ pẹlu ìdá máàrún nínú ìdá ọgọrun (5%) sí bílíọ̀nù méjì-le-ní-ogún o le igbọnjẹ mẹta-dín-ní-ọgbọn owó náírà (₦22.27 billion), nígbà tí owó tí ilé iṣẹ pà wọlé lórí ìpín ìdókòwò kan gbéra sókè pẹlú ìdá mẹfa nínú ìdá ọgọrun (6%) sí bílíọ̀nù mẹta-le-lógún o le igbọnjẹ mẹta-le-ní-aadọrun owó náírà (₦23.87 billion)',\n",
              " 'Sùgbón àwon àlàyé díè síi yóò jáde lóri irú àwọn itosi ati ònà tí yíò gbée gbà,” o so']"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ],
      "source": [
        "import random\n",
        "s = random.sample(texts_with_unk, 5)\n",
        "s"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jZhSDbKWvZ0b"
      },
      "source": [
        "Apparently, most of the texts with 3634 unknown tokens just have some punctuation unfamiliar to the NLLB tokenizer.\n",
        "\n",
        "This is because the NLLB model was pretrained on normalized texts. If we reproduce the normalization, most of the problems would be fixed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "UBXPm8Ckvtf3"
      },
      "outputs": [],
      "source": [
        "# this code is adapted from  the Stopes repo of the NLLB team\n",
        "# https://github.com/facebookresearch/stopes/blob/main/stopes/pipelines/monolingual/monolingual_line_processor.py#L214\n",
        "\n",
        "import re\n",
        "import sys\n",
        "import typing as tp\n",
        "import unicodedata\n",
        "from sacremoses import MosesPunctNormalizer\n",
        "\n",
        "\n",
        "mpn = MosesPunctNormalizer(lang=\"en\")\n",
        "mpn.substitutions = [\n",
        "    (re.compile(r), sub) for r, sub in mpn.substitutions\n",
        "]\n",
        "\n",
        "\n",
        "def get_non_printing_char_replacer(replace_by: str = \" \") -> tp.Callable[[str], str]:\n",
        "    non_printable_map = {\n",
        "        ord(c): replace_by\n",
        "        for c in (chr(i) for i in range(sys.maxunicode + 1))\n",
        "        # same as \\p{C} in perl\n",
        "        # see https://www.unicode.org/reports/tr44/#General_Category_Values\n",
        "        if unicodedata.category(c) in {\"C\", \"Cc\", \"Cf\", \"Cs\", \"Co\", \"Cn\"}\n",
        "    }\n",
        "\n",
        "    def replace_non_printing_char(line) -> str:\n",
        "        return line.translate(non_printable_map)\n",
        "\n",
        "    return replace_non_printing_char\n",
        "\n",
        "replace_nonprint = get_non_printing_char_replacer(\" \")\n",
        "\n",
        "def preproc(text):\n",
        "    clean = mpn.normalize(text)\n",
        "    clean = replace_nonprint(clean)\n",
        "    # replace 𝓕𝔯𝔞𝔫𝔠𝔢𝔰𝔠𝔞 by Francesca\n",
        "    clean = unicodedata.normalize(\"NFKC\", clean)\n",
        "    return clean"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 66,
          "referenced_widgets": [
            "b358392d266c451694a78d1c9303b4fc",
            "160ccde1b33044c59b0cd27f81e3a358",
            "40fe8b64f2414c8eaffc672494f8e115",
            "1a265a8a39e5445183f1d73147715aa8",
            "e139fa59521b49f6908081473c20b9f8",
            "6e5db85fe0ca462da2a32b7bd32c7ec2",
            "f3f7440ec22c4c8483d4e7744395980b",
            "53e3a3e3444c41d4a3f5f1941bc0c19f",
            "1589724f361a42b0990b83d9e8d12aa5",
            "219fe916d2094b56a81087231bcd7604",
            "85d483f550da4b8481106489f44a794d"
          ]
        },
        "id": "3MJp75LAv6Wo",
        "outputId": "9beb1aef-3291-4087-d00b-9ae63b7170d1"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/389 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b358392d266c451694a78d1c9303b4fc"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "133\n"
          ]
        }
      ],
      "source": [
        "texts_with_unk_normed = [text for text in tqdm(texts_with_unk) if tokenizer.unk_token_id in tokenizer(preproc(text)).input_ids]\n",
        "print(len(texts_with_unk_normed))"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import random\n",
        "s = random.sample(texts_with_unk_normed, 5)\n",
        "s"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yTH7fWJy_jiM",
        "outputId": "6cb8205b-1885-4517-f666-6951aa1b294c"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['Ní ìfọ́síwẹ́wẹ́ síwájú síi, ó fihàn pé wọ́n rí tírílíọ̀nù mẹ́rin lé ní àádọta bílíọ̀nù náírà (₦4. 07 trillion) láti ara epo, nígbà tí àwọn tí kì í ṣe epo jẹ́ tírílíọ̀nù mẹ́fà dín díè náírà (₦5. 96 trillion).',\n",
              " 'Ní àgbékalẹ̀ olósooṣù, èyí n lewo pẹlu ìdá kan ati belenja mọkanla nínú ìdá ọgọrun láti náírà méjì-din-lewa o le ní ẹgbẹrin àti belenja (₦808.87) owó jala kan, ti wọn kò sílè ní oṣù belu (November) ọdún 2022.',\n",
              " 'Láárin ìṣòro ìdíyelé owó orí-ọjà, NBS sọ wípé ìdíyelé owó orí epo to jẹ ẹrin-din-lewa-le-nigba náírà (₦206) owó jálá kan ní oṣù ọpẹ (December), 2022.',\n",
              " 'Rs ṃèwá fún mítà kan nị ̣òké kan, ̣ẹ̀éḍégbạ̀ètàdínlógún Kéréjù: Ìdínkù ìṣòwò ṃèwá ̣ẹ̀éḍégḅẹ́ètàdínlógún , eedegbaeedogun ibi òwò aṣ̣ọ  ̣egḅèrún ̣èla Jeans @',\n",
              " 'Àgbéyè wo ẹtọ ìṣúna ilé ìṣe na fi hàn wípé merin lè lógorìn pelu esun méje ìpín Ógùrún (84.7%) òṣìṣẹ́ ilé iṣẹ́ na ni òun je àwọn òṣìṣẹ́ tí òun gba owó ńlá náírà marun lọ́nà ẹgbẹ̀rún lọ́nà ẹgbẹ̀rún (₦5 million) àti sókè. ']"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h5qipLorwBG-"
      },
      "source": [
        "Indeed, after normalizing texts, none of them contain unknown tokens. We will use this as one more piece of evidence that we don't have to update the tokenizer vocabulary to use it with Tyvan."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 3 (optional). Expanding the vocabulary"
      ],
      "metadata": {
        "id": "vPDFg392tyFB"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4hUhun80t5u9"
      },
      "source": [
        "# 4. Adding a new language tag to the tokenizer and model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "MhG4XWTP-g3w"
      },
      "outputs": [],
      "source": [
        "from transformers import AutoModelForSeq2SeqLM\n",
        "from transformers import NllbTokenizer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GGh6UDG_-m1K",
        "outputId": "1ef09499-b97e-4aaf-ccfc-24a37674fc38"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "256204\n",
            "['zul_Latn', '<mask>']\n"
          ]
        }
      ],
      "source": [
        "tokenizer = NllbTokenizer.from_pretrained('facebook/nllb-200-distilled-600M')\n",
        "print(len(tokenizer))\n",
        "print(tokenizer.convert_ids_to_tokens([256202, 256203]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "d02fbR_L-nCh"
      },
      "outputs": [],
      "source": [
        "def fix_tokenizer(tokenizer, new_lang='yor_imo'):\n",
        "    \"\"\"\n",
        "    Add a new language token to the tokenizer vocabulary\n",
        "    (this should be done each time after its initialization)\n",
        "    \"\"\"\n",
        "    old_len = len(tokenizer) - int(new_lang in tokenizer.added_tokens_encoder)\n",
        "    tokenizer.lang_code_to_id[new_lang] = old_len-1\n",
        "    tokenizer.id_to_lang_code[old_len-1] = new_lang\n",
        "    # always move \"mask\" to the last position\n",
        "    tokenizer.fairseq_tokens_to_ids[\"<mask>\"] = len(tokenizer.sp_model) + len(tokenizer.lang_code_to_id) + tokenizer.fairseq_offset\n",
        "\n",
        "    tokenizer.fairseq_tokens_to_ids.update(tokenizer.lang_code_to_id)\n",
        "    tokenizer.fairseq_ids_to_tokens = {v: k for k, v in tokenizer.fairseq_tokens_to_ids.items()}\n",
        "    if new_lang not in tokenizer._additional_special_tokens:\n",
        "        tokenizer._additional_special_tokens.append(new_lang)\n",
        "    # clear the added token encoder; otherwise a new token may end up there by mistake\n",
        "    tokenizer.added_tokens_encoder = {}\n",
        "    tokenizer.added_tokens_decoder = {}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "jZ7YPnHQ-pDT"
      },
      "outputs": [],
      "source": [
        "fix_tokenizer(tokenizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ppwnJUrj-rLu",
        "outputId": "37ae072b-b4e8-4e36-bb41-eb3c1c581c2c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['zul_Latn', 'yor_imo', '<mask>']\n",
            "[256202, 256203, 256204]\n"
          ]
        }
      ],
      "source": [
        "print(tokenizer.convert_ids_to_tokens([256202, 256203, 256204])) # ['zul_Latn', 'tyv_Cyrl', '<mask>']\n",
        "print(tokenizer.convert_tokens_to_ids(['zul_Latn', 'yor_imo', '<mask>'])) # [256202, 256203, 256204]\n",
        "# this is consistent now, wow!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ktO8outV-xws",
        "outputId": "c17e1501-1fd8-454f-8e9a-db7552253aa8"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "256203 256198\n"
          ]
        }
      ],
      "source": [
        "added_token_id = tokenizer.convert_tokens_to_ids('yor_imo')\n",
        "similar_lang_id = tokenizer.convert_tokens_to_ids('yor_Latn')\n",
        "print(added_token_id, similar_lang_id)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 272,
          "referenced_widgets": [
            "e95932f91bb6487585b1dac0e17af7b8",
            "b556be65181f436f995eb21e7f458b00",
            "114aaf3174eb4f838e50c19558383b04",
            "6bdba58bd25444f6b8a61e4958459662",
            "2c71903d089f478eac895388be3f789e",
            "1b69592b3eee4327a4aa87b9b1351eee",
            "14aa13da61c740b5ad53eaf1780f083b",
            "f66db39bad81474aaf634825fd88ee36",
            "eb86a7722b894e978400e65198a1c7f7",
            "e7dfbbced006476dae5c160f1d363a85",
            "3a3374913d8c440794587bb7a2859154",
            "18eb8063745a4566abee6f69eb7bc2b9",
            "752a364aaaff4cd3a54fb256bb8b5f65",
            "b1b5b3c9ed3e4aa08db37615af52a291",
            "518b3c406c2346d7ad279bcdc2243d70",
            "fb75663870b346ac930e68431c8810d8",
            "bba9a22a4b6c49a7b725d58fe8bf4414",
            "5407739c7a044787a388cbee927136d0",
            "c5333d0501d748d7bd99ca3f3358751d",
            "0223df94c49445b0abc6fc7986f8c749",
            "5941bc78a34c4cf5a9d22a811380c9c7",
            "2a50c218d62b4e279f4bd470c8ceb772",
            "5f6ba05a2caf43658644638b293e5dd4",
            "6e0c3d6f87774c528ba452c8a07ba3a1",
            "d7e06ccdd1bd4315a16392b34ea8bf52",
            "8487a78750d841cd9c402ba016be5b09",
            "7838363f24944d1184758a7745cdb056",
            "4dace0ae80cb47ce9879a94028826e35",
            "8fc2ec641a7d446ca9bac60c82d557dd",
            "87c2086cf6b14c198da5fe35b59b6dcc",
            "bee75c5460ea459e960e4c56af75c35d",
            "1b8f53ff04434cb59a4b4ab3adecfd38",
            "4b5a8dd1723d41a2b74e37eff62a7395"
          ]
        },
        "id": "tLlwR3_R-tDL",
        "outputId": "40c60819-7a78-4c04-9025-1240a52b4a3f"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "config.json:   0%|          | 0.00/846 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e95932f91bb6487585b1dac0e17af7b8"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n",
            "  torch.utils._pytree._register_pytree_node(\n",
            "/usr/local/lib/python3.10/dist-packages/transformers/utils/generic.py:311: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.\n",
            "  torch.utils._pytree._register_pytree_node(\n",
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "pytorch_model.bin:   0%|          | 0.00/2.46G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "18eb8063745a4566abee6f69eb7bc2b9"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/189 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "5f6ba05a2caf43658644638b293e5dd4"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "You are resizing the embedding layer without providing a `pad_to_multiple_of` parameter. This means that the new embedding dimension will be 256205. This might induce some performance reduction as *Tensor Cores* will not be available. For more details about this, or help on choosing the correct value for resizing, refer to this guide: https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html#requirements-tc\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Embedding(256205, 1024)"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "model = AutoModelForSeq2SeqLM.from_pretrained('facebook/nllb-200-distilled-600M')\n",
        "model.resize_token_embeddings(len(tokenizer))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "lV-fIcWZ-3WJ"
      },
      "outputs": [],
      "source": [
        "# moving the embedding for \"mask\" to its new position\n",
        "model.model.shared.weight.data[added_token_id+1] = model.model.shared.weight.data[added_token_id]\n",
        "# initializing new language token with a token of a similar language\n",
        "model.model.shared.weight.data[added_token_id] = model.model.shared.weight.data[similar_lang_id]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ssJCguZ-3oH"
      },
      "source": [
        "# 5. Preparing the training loop"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "OjuuYbpG-7nS"
      },
      "outputs": [],
      "source": [
        "import gc\n",
        "import random\n",
        "import numpy as np\n",
        "import torch\n",
        "from tqdm.auto import tqdm, trange\n",
        "from transformers.optimization import Adafactor\n",
        "from transformers import get_constant_schedule_with_warmup\n",
        "\n",
        "def cleanup():\n",
        "    \"\"\"Try to free GPU memory\"\"\"\n",
        "    gc.collect()\n",
        "    torch.cuda.empty_cache()\n",
        "\n",
        "cleanup()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "id": "olSkAk2p-9IE"
      },
      "outputs": [],
      "source": [
        "model.cuda();"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "id": "ScoroAeY-_-J"
      },
      "outputs": [],
      "source": [
        "optimizer = Adafactor(\n",
        "    [p for p in model.parameters() if p.requires_grad],\n",
        "    scale_parameter=False,\n",
        "    relative_step=False,\n",
        "    lr=1e-4,\n",
        "    clip_threshold=1.0,\n",
        "    weight_decay=1e-3,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "id": "t9cxb-64_Bco"
      },
      "outputs": [],
      "source": [
        "batch_size = 16  # 32 already doesn't fit well to 15GB of GPU memory\n",
        "max_length = 128\n",
        "warmup_steps = 1_000\n",
        "training_steps = 57000"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "id": "1tbPSr7w_Hnp"
      },
      "outputs": [],
      "source": [
        "losses = []\n",
        "scheduler = get_constant_schedule_with_warmup(optimizer, num_warmup_steps=warmup_steps)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "H15rBohL_MaC",
        "outputId": "9b28ca39-a9bf-4596-d71b-8483a81cf75c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(['Domestic transactions on NGX hits N170 billion in January, out-performs foreign transactions by 584%.'], ['Kárà-kátà ni abẹ́lé lori NGX gbẹnu lè 170 biliọnu náírà ní oṣù kínní, tí o tayọ àwọn kárà-kátà ókéré pẹlú ìdá ẹrinlelẹẹdẹgbẹta àti mẹrin.'], 'eng_Latn', 'yor_imo')\n"
          ]
        }
      ],
      "source": [
        "# LANGS = [('ru', 'rus_Cyrl'), ('tyv', 'tyv_Cyrl')]\n",
        "\n",
        "# def get_batch_pairs(batch_size, data=train):\n",
        "#     (l1, long1), (l2, long2) = random.sample(LANGS, 2)\n",
        "#     xx, yy = [], []\n",
        "#     for _ in range(batch_size):\n",
        "#         item = data.iloc[random.randint(0, len(data)-1)]\n",
        "#         xx.append(preproc(item[l1]))\n",
        "#         yy.append(preproc(item[l2]))\n",
        "#     return xx, yy, long1, long2\n",
        "\n",
        "# print(get_batch_pairs(1))\n",
        "# # (['чеди'], ['семь'], 'tyv_Cyrl', 'rus_Cyrl')\n",
        "\n",
        "LANGS = [('English', 'eng_Latn'), ('Yoruba', 'yor_imo')]\n",
        "\n",
        "def get_batch_pairs(batch_size, data=train):\n",
        "    (l1, long1), (l2, long2) = ('English', 'eng_Latn'), ('Yoruba', 'yor_imo')\n",
        "    xx, yy = [], []\n",
        "    for _ in range(batch_size):\n",
        "        item = data.iloc[random.randint(0, len(data)-1)]\n",
        "        xx.append(preproc(item[l1]))  # Preprocessing English text\n",
        "        yy.append(preproc(item[l2]))  # Preprocessing Yoruba-imo text\n",
        "    return xx, yy, long1, long2\n",
        "\n",
        "print(get_batch_pairs(1))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "2ACqa2if_N5U"
      },
      "outputs": [],
      "source": [
        "MODEL_SAVE_PATH = '/content/drive/MyDrive/MT_Research/Yoruba/models/nllb-eng-yor-v1'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V1BV9mcZwmLd"
      },
      "source": [
        "# 6. The training loop"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 101,
          "referenced_widgets": [
            "26b724eadf264317bff2da2b3cc7fa59",
            "18cebb22a2df4e458694eea2d28a451f",
            "599293ac93044a4dba47d5b98ee38b57",
            "227b1591dbde44c3afa0e45ab95be0d7",
            "abdca147cab842788870a2849696e071",
            "6a436d06080743fba25772d2df3ce5c5",
            "48b626d1d8f349e3982c3793ca7f006e",
            "080f970af40d498ab21ecc31365b6f7c",
            "71a00bde017844919bfb981d179a2eb2",
            "6217008254a542128962af75eee3b979",
            "2eaed57764954dbe9772864307668094"
          ]
        },
        "id": "ahPBT-vt_c91",
        "outputId": "34cfca7a-4ea0-4ac4-c63a-8a2b43400320"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "26b724eadf264317bff2da2b3cc7fa59",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/57000 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0 2.8191304206848145\n",
            "1000 2.1505459456443785\n",
            "2000 1.6128845066428183\n"
          ]
        }
      ],
      "source": [
        "model.train()\n",
        "x, y, loss = None, None, None\n",
        "cleanup()\n",
        "\n",
        "tq = trange(len(losses), training_steps)\n",
        "for i in tq:\n",
        "    xx, yy, lang1, lang2 = get_batch_pairs(batch_size)\n",
        "    try:\n",
        "        tokenizer.src_lang = lang1\n",
        "        x = tokenizer(xx, return_tensors='pt', padding=True, truncation=True, max_length=max_length).to(model.device)\n",
        "        tokenizer.src_lang = lang2\n",
        "        y = tokenizer(yy, return_tensors='pt', padding=True, truncation=True, max_length=max_length).to(model.device)\n",
        "        y.input_ids[y.input_ids == tokenizer.pad_token_id] = -100\n",
        "\n",
        "        loss = model(**x, labels=y.input_ids).loss\n",
        "        loss.backward()\n",
        "        losses.append(loss.item())\n",
        "\n",
        "        optimizer.step()\n",
        "        optimizer.zero_grad(set_to_none=True)\n",
        "        scheduler.step()\n",
        "\n",
        "    except RuntimeError as e:\n",
        "        optimizer.zero_grad(set_to_none=True)\n",
        "        x, y, loss = None, None, None\n",
        "        cleanup()\n",
        "        print('error', max(len(s) for s in xx + yy), e)\n",
        "        continue\n",
        "\n",
        "    if i % 1000 == 0:\n",
        "        print(i, np.mean(losses[-1000:]))\n",
        "\n",
        "    if i % 1000 == 0 and i > 0:\n",
        "        model.save_pretrained(MODEL_SAVE_PATH)\n",
        "        tokenizer.save_pretrained(MODEL_SAVE_PATH)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 430
        },
        "id": "xXXT9pcd_9Au",
        "outputId": "58658ffc-f3d0-4a85-8884-cdca6ba08e17"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhYAAAGdCAYAAABO2DpVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAxLUlEQVR4nO3deXxU5b3H8e9kmwTIhC0hIGGTTXZkM6wiyOrutWqpRfRWsXhdWq3gTl0Sa8utWqVqW2ndqFpRr4CIrIKsYd93CEsAgWQSIEOSee4fyMAQAllOcpIzn/frldfrzDnPzPmF84rz9TnneR6XMcYIAADAAmF2FwAAAJyDYAEAACxDsAAAAJYhWAAAAMsQLAAAgGUIFgAAwDIECwAAYBmCBQAAsExERZ/Q7/dr//79io2NlcvlqujTAwCAUjDGKDs7Ww0aNFBYWNH9EhUeLPbv36+kpKSKPi0AALBAenq6GjZsWOTxCg8WsbGxkk4X5vF4Kvr0AACgFLxer5KSkgLf40Wp8GBx5vaHx+MhWAAAUMVc6jEGHt4EAACWIVgAAADLECwAAIBlCBYAAMAyBAsAAGAZggUAALAMwQIAAFiGYAEAACxDsAAAAJYhWAAAAMsQLAAAgGUIFgAAwDIVvghZeZnw7WZl+/J1f9/LlRgXbXc5AACEJMf0WExelq73Fu7S0eOn7C4FAICQ5ZhgAQAA7EewAAAAliFYAAAAyxAsAACAZQgWAADAMgQLAABgGYIFAACwDMECAABYhmABAAAsQ7AAAACWIVgAAADLECwAAIBlCBYAAMAyBAsAAGAZggUAALAMwQIAAFjGccHCyNhdAgAAIavEwWLfvn36xS9+oTp16igmJkbt27fX8uXLy6M2AABQxUSUpPGxY8fUq1cv9e/fX9OnT1d8fLy2bt2qWrVqlVd9AACgCilRsHjllVeUlJSk9957L7CvadOmlhcFAACqphLdCvnqq6/UtWtX3XbbbUpISFDnzp317rvvXvQ9Pp9PXq836AcAADhTiYLFjh07NHHiRLVo0UIzZszQAw88oIceekj//Oc/i3xPSkqK4uLiAj9JSUllLvpCXK5y+VgAAFACLmNMsYdRREVFqWvXrvrhhx8C+x566CEtW7ZMixYtuuB7fD6ffD5f4LXX61VSUpKysrLk8XjKUHqwHi9/p4Nen6Y+1FttG8RZ9rkAAOD093dcXNwlv79L1GNRv359tWnTJmjfFVdcoT179hT5HrfbLY/HE/QDAACcqUTBolevXtq8eXPQvi1btqhx48aWFgUAAKqmEgWLRx99VIsXL9bLL7+sbdu26aOPPtI777yjMWPGlFd9AACgCilRsOjWrZumTJmijz/+WO3atdMLL7ygP//5zxoxYkR51QcAAKqQEs1jIUnXXXedrrvuuvKoBQAAVHGOWysEAADYh2ABAAAsQ7AAAACWIVgAAADLECwAAIBlCBYAAMAyBAsAAGAZxwWL4i+pBgAArOa4YAEAAOxDsAAAAJYhWAAAAMsQLAAAgGUIFgAAwDIECwAAYBmCBQAAsIxjgoVLLrtLAAAg5DkmWAAAAPsRLAAAgGUIFgAAwDIECwAAYBmCBQAAsAzBAgAAWIZgAQAALEOwAAAAliFYAAAAyxAsAACAZQgWAADAMgQLAABgGYIFAACwDMECAABYhmABAAAsQ7AAAACWIVgAAADLECwAAIBlCBYAAMAyBAsAAGAZxwSLDG+uJMmXX2BzJQAAhC7HBIszJs7dbncJAACELMcFi9V7s+wuAQCAkOW4YOGyuwAAAEKY84IFyQIAANs4LliEkSwAALCN44IFsQIAAPs4L1jQYwEAgG0cFywAAIB9CBYAAMAyBAsAAGAZggUAALAMwQIAAFimRMHi+eefl8vlCvpp3bp1edUGAACqmIiSvqFt27b67rvvzn5ARIk/AgAAOFSJU0FERIQSExPLoxYAAFDFlfgZi61bt6pBgwZq1qyZRowYoT179ly0vc/nk9frDfoBAADOVKJg0aNHD02aNEnffPONJk6cqJ07d6pPnz7Kzs4u8j0pKSmKi4sL/CQlJZW5aAAAUDm5jDGmtG/OzMxU48aNNWHCBN17770XbOPz+eTz+QKvvV6vkpKSlJWVJY/HU9pTF9Jk7FRJ0mU1Y7Rw7DWWfS4AADj9/R0XF3fJ7+8yPXlZs2ZNtWzZUtu2bSuyjdvtltvtLstpAABAFVGmeSxycnK0fft21a9f36p6AABAFVaiYPHYY49p3rx52rVrl3744QfdfPPNCg8P15133lle9QEAgCqkRLdC9u7dqzvvvFNHjhxRfHy8evfurcWLFys+Pr686gMAAFVIiYLF5MmTy6sOAADgAKwVAgAALEOwAAAAliFYAAAAyxAsAACAZQgWAADAMgQLAABgGYIFAACwDMECAABYhmABAAAsQ7AAAACWIVgAAADLOC5Y7Ms8qeO+fLvLAAAgJDkuWEjSx0v32F0CAAAhyZHBIq/A2F0CAAAhyZHBwohgAQCAHRwZLE7l++0uAQCAkOTIYDFn0yG7SwAAICQ5Mlj4uRMCAIAtHBksPDERdpcAAEBIcmSwGNQm0e4SAAAISY4MFmFhLrtLAAAgJDkyWBArAACwhyODBc9uAgBgD0cGCz/DQgAAsIUzg4UhWAAAYAdHBosCeiwAALCFI4MFPRYAANjDkcFi/X6v3SUAABCSHBksvly13+4SAAAISY4MFgAAwB4ECwAAYBnHBIumdavbXQIAACHPMcHii1/3srsEAABCnmOCRVy1SN3fr1ngNXNZAABQ8RwTLCTJHREe2M4r8NtYCQAAoclRwaLAfzZM5NNjAQBAhXNUsDg3S+TTYwEAQIVzVLA41ymCBQAAFc5RwcITHRnYnrnhoI2VAAAQmhwVLEb2bBzYfmrKOhsrAQAgNDkqWFSLirC7BAAAQpqjggUAALAXwQIAAFiGYAEAACzj2GAxtF2i3SUAABByHBcs7u97er2QzBN5NlcCAEDocVyweHv+DknSoh1HbK4EAIDQ47hgAQAA7EOwAAAAliFYAAAAy5QpWKSmpsrlcumRRx6xqJyy+/vIrnaXAABAyCp1sFi2bJnefvttdejQwcp6yqxODbck6bKaMTZXAgBA6ClVsMjJydGIESP07rvvqlatWlbXVCbuiNO/0r7Mk8rx5dtcDQAAoaVUwWLMmDEaPny4Bg4ceMm2Pp9PXq836Kc85RX4A9tPT1lbrucCAADBSrwc6OTJk7VixQotW7asWO1TUlI0fvz4EhdWWhFhZ7PSgazcCjsvAAAoYY9Fenq6Hn74YX344YeKjo4u1nvGjRunrKyswE96enqpCi2uK+rHBrYb16lWrucCAADBStRjkZaWpkOHDunKK68M7CsoKND8+fP1l7/8RT6fT+Hh4UHvcbvdcrvd1lRbDC6XK7Ddv1VChZ0XAACUMFgMGDBAa9cGP7cwatQotW7dWk888UShUGGXHk1ra8nOoyowxu5SAAAIKSUKFrGxsWrXrl3QvurVq6tOnTqF9tsp6qeRIec+yAkAAMqfI2fejAo//WudyidYAABQkUo8KuR8c+fOtaAMa53psSBYAABQsRzZYxF5pseigGcsAACoSI4MFvRYAABgD0cGizMDTo8zpTcAABXKkcHi07S9kqS/zNlmcyUAAIQWRwYLAABgD0cGi/v6NpMk3dipgc2VAAAQWhwZLDzRp0fRZp3Ms7kSAABCiyODxecr90mS5m4+bHMlAACEFkcGix2Hj9tdAgAAIcmRwaJPi7qB7b3HTthYCQAAocWRweK/ujQMbPd+ZY6NlQAAEFocGSwGt020uwQAAEKSI4NFdGS43SUAABCSHBksztWsbnW7SwAAIGQ4Pljs+PG4juT47C4DAICQ4PhgIUmPfrLa7hIAAAgJIREsFmxloiwAACqCY4NFkzrVAtt+Y2MhAACEEMcGi8cGt7K7BAAAQo5jg8W1berZXQIAACHHscHCHcFcFgAAVDTHBgtJuqxmTGA7N6/AxkoAAAgNjg4WU8b0DGx/tGSPjZUAABAaHB0s4mu4A9vVorg1AgBAeXN0sHC5XIHt+Fj3RVoCAAArODpYSFK3JrUkSb58v82VAADgfI4PFst2HZMkfZa21+ZKAABwPscHizNmbzpkdwkAADie44PFVc1q210CAAAhw/HBYvGOo3aXAABAyHB8sDhXAauRAQBQrhwfLL77Td/A9p6jJ2ysBAAA53N8sGieEBvYXrcvy8ZKAABwPscHi3P9z8cr7S4BAABHC6lgAQAAyhfBAgAAWIZgAQAALEOwAAAAlgmJYNG5Uc3A9p4jDDkFAKC8hESwaFXv7JDT3PwCGysBAMDZQiJYjBt6RWA7OzfPxkoAAHC2kAgWcdUiA9sT5263sRIAAJwtJILFuY6doMcCAIDyEjLBolnd6pKkGzs1sLkSAACcK2SCRWx0hCTpjzM221wJAADOFTLBYvXe0wuQeXPz9dLUDTZXAwCAM4VMsDjXu9/vtLsEAAAcKSSDBQAAKB8hEyz+PrKr3SUAAOB4JQoWEydOVIcOHeTxeOTxeJScnKzp06eXV22WGnBFPbtLAADA8UoULBo2bKjU1FSlpaVp+fLluuaaa3TjjTdq/fr15VWfpV79rw52lwAAgKNFlKTx9ddfH/T6pZde0sSJE7V48WK1bdvW0sLKQ7cmtSVJ1aPCba4EAABnKlGwOFdBQYE+/fRTHT9+XMnJyUW28/l88vl8gdder7e0pyyzqIjTHTTHTxUor8CvyPCQecQEAIAKUeJv1rVr16pGjRpyu90aPXq0pkyZojZt2hTZPiUlRXFxcYGfpKSkMhVslR2Hj9tdAgAAjlPiYNGqVSutWrVKS5Ys0QMPPKCRI0dqw4aiJ5waN26csrKyAj/p6ellKrgs6nmiA9t5BX7b6gAAwKlKfCskKipKzZs3lyR16dJFy5Yt02uvvaa33377gu3dbrfcbnfZqrRIeJgrsL1uX5baXRZnYzUAADhPmR8y8Pv9Qc9QVBVjP19rdwkAADhOiXosxo0bp6FDh6pRo0bKzs7WRx99pLlz52rGjBnlVR8AAKhCShQsDh06pF/+8pc6cOCA4uLi1KFDB82YMUPXXnttedVnubo1ovRjzilJ0rZD2WqeEGtzRQAAOIfLGGMq8oRer1dxcXHKysqSx+OpyFNLkj5dnq7HP1sjSYqODNOmF4ZWeA0AAFQ1xf3+DrmJHPq3Tghs5+YxMgQAACuFXLCoW6NyjFABAMCJQi5YSNLvbzw7/fipfHotAACwSkgGi4HnrHR6//vLbawEAABnCclgkXjODJxzNh+2sRIAAJwlJINF2DkzcAIAAOuEZLCQpC6NawW2K3jELQAAjhWyweLhAS0C23uOnrCxEgAAnCNkg0WfFnUD24eyq95aJwAAVEYhGyxcrrPPWdz210U6evyUjdUAAOAMIRssznfjmwvsLgEAgCovpINFTGR4YDv96EkbKwEAwBlCOlisfX6Q3SUAAOAoIR0sIsLDNKpXE7vLAADAMUI6WEjSjZ0uC2x/uGS3jZUAAFD1hXywqF0tKrA9/v822FgJAABVX8gHi0Z1qgW2WekUAICyCflgcb5jzGcBAECpESwk/XfvpoHtzi/MVDpTfAMAUCoEC0n397s86HWfP8yxqRIAAKo2goWk+Fi3Hh3Y0u4yAACo8ggWP/HERNhdAgAAVR7B4icjk5sEve7w/AzlFzBKBACAkiBY/CQszKXpD/cJvPbm5mvY69/bWBEAAFUPweIcV9T3BL3ecjDHpkoAAKiaCBYAAMAyBItLOJDFcuoAABQXweI8O1OGBQ09TU6ZbWM1AABULQSL87hcLj08sEXQvj1HmIkTAIDiIFgUw+gP0uwuAQCAKoFgUYSdKcMC2xsOeG2sBACAqoNgUQSXyxX0OseXb1MlAABUHQSLi5j5aN/A9j2TltlYCQAAVQPB4iJa1IsNbC/deZQpvgEAuASCRQk0f2q6/H5jdxkAAFRaBItLuL9fs6DX3244aFMlAABUfgSLSxg7pHXQ69EfpOnkqQKbqgEAoHIjWFyCy+XSjpeHBe274tlvbKoGAIDKjWBRDGFhrkL7vLl5NlQCAEDlRrAoptH9Lg96vXj7EZsqAQCg8iJYFNPYoa01qleTwOv73k/TvcxtAQBAEIJFCTx3fdug17M2HdLhbJ9N1QAAUPkQLEroX/d0D3rd7aXvbKoEAIDKh2BRQn1bxttdAgAAlRbBohR2pQ5XfKw78PqxT1cHHd+feVK5ecx1AQAIPQSLUnrtjk6B7c/S9mrH4RzN3XxITcZOVc/U2Wr9DHNdAABCT4TdBVRVPS+vG/T6mj/NK9RmxZ5jurJRrYoqCQAA29FjUQbrxw++6PFb3vpBM1lbBAAQQggWZVDdfekOn1/9azlriwAAQgbBoowWjbvmkm1YWwQAECpK9IxFSkqKPv/8c23atEkxMTHq2bOnXnnlFbVq1aq86qv06sfFaFfq8MDrbYeyVT8uRne+u1hr9mbZWBkAABWvRD0W8+bN05gxY7R48WLNnDlTeXl5GjRokI4fP15e9VU5zRNiVd0docn3XRW0n0XLAAChwGWMMaV98+HDh5WQkKB58+apb9++xXqP1+tVXFycsrKy5PF4SnvqKiG/wK/mT02XJE0a1U1Xt0qwuSIAAEqnuN/fZXrGIivrdFd/7dq1i2zj8/nk9XqDfkJFRPjZf94Pl+xRXoFfe46csLEiAADKV6l7LPx+v2644QZlZmZqwYIFRbZ7/vnnNX78+EL7Q6HHQpKajJ1aaN+1berpZ12T1KdFXUVHhttQFQAAJVPcHotSB4sHHnhA06dP14IFC9SwYcMi2/l8Pvl8Z1cA9Xq9SkpKCulgca5zH/wEAKCyKtdbIQ8++KC+/vprzZkz56KhQpLcbrc8Hk/QTyh5564uFz3OrREAgJOUKFgYY/Tggw9qypQpmj17tpo2bVpedTnGoLaJFz3e99U5yi/wV1A1AACUrxIFizFjxuiDDz7QRx99pNjYWGVkZCgjI0MnT54sr/oc4R93d73o8eZPTVce4QIA4AAlesbC5XJdcP97772nu+++u1ifEUrDTc+1fn+WEmKjlTJ9oz5fsa/Q8ejIMG38/ZAi/40BALBTuT+8WVqhGizOlZGVq32ZJ3TrxEVFttn4+yGKiWLECACgcqiQeSxQOolx0erSuLZe/a8ORba50Poifr+R31+hORAAgBIp0VohsNZtXZPUMammBv3v/CLb5Bf49X9r9iv96ElNmLlFEkNUAQCVF7dCKoHME6fU6fczL3js9q5J+vfy9KB9jwxsoUcGtqyI0gAAkMStkCqlZrUoPXNdG0lSrDu4E+n8UCFJf/5ua4XUBQBASREsKol7ezfVrtThmvP41cVqX8EdTQAAFAvBopKpW8OtxwZd+jbH/363lXABAKh0CBaV0IPXtNCmF4YU2v+n2zoGtl+ftVVNx03TqfyzE2s1f3Kamo6bSuAAANiGYFFJRUeGa8YjfdW9aW1d26aehrVP1A2dGhRq9+sPV0iSluw4ony/kTFS03HTNPh/5xMwAAAVjlEhVcylVks937zHr1a/V+fqlVvb6/ZujcqpKgCA0zEqxKFWPXttidr3e3WuJOmJ/6wth2oAAAhGsKhialaL0q7U4frbLy++sNmFvDN/uyTpha836MGPVlhdGgAA3Aqpqowxuu6NBVq/31umz2EWTwBAcRT3+5spvasol8ulqQ/1kRT83MW/77tKufl+eaIjdPNbPxT78/61aJfW7ctSyi0dFB7GCqsAgNLhVogDLH1yQGC7R7M66tcyXp0b1dKu1OH67jd9L/reJmOn6p352/Xsl+v1yfK9mrKy8JLuAAAUF7dCQsTUNQc0ppjPVbxya3v9a9FuPXNdG13VrE45VwYAqAoYFYIgwzvU16PFXLjsif+s1fr9Xt3xzmJtyjj9DMfhbJ/+vWyPCli2HQBwEfRYhJjs3DxFhodp77GTGjhhXrHesyt1eNBzHDd2aqAnhrRWg5oxhdoOnDBPBX6j737Tj2c1AMBB6LHABcVGRyo6MlzNE2po+8vDivWeR/+9Kuj1l6v2q2fqbP1l9tlVVvdlnlSTsVO17VCOdv54XJc/Oc3KsgEAVQQ9FiHuSI5PqdM36YWb2qn1M99Y+tmv3dFJw9rXV2Q4+RUAqjp6LFAsdWq49eptHRUdGa4+LepKkpKb1dHWl4aW+bMfnrxKLZ6aLj/PZQBAyGAeCwS8f28P5RX4i+xh2PziEB3O9qn3K3NK9LnNnpym1c8Nki+vQAmeaCtKBQBUUvRYIMi5oaJvy/igY+6IcDWsVS1o3ozzff0/vS+4v+P4b9X95Vna9eNxawoFAFRKPGOBIuUX+DV5WbqMMbqje6Og0LFkxxEt2XlUf523XSdOFQT270odrhV7jumWS8z6yVTiAFC1FPf7m2CBMvP7jR79ZJXuuqqxujapHdi/Kj1TN7258ILvqedxa8mTA5V54pTGfLRCC7cd0eT7rmJCLgCopHh4ExUmLMyl1+7oHBQqJKlTUs0i33PQ61P60RPq9PuZWrjtiCTpjncWl2eZAIAKQI8FKsTiHUdKFRzSnh6oN2Zv06QfdkmSfn9jW63dm6XxN7ZVtajiPXt8ZlRKGBN2AUCpcSsEldKXq/bp4cmrLPmsT+5P1sm8Ai3c9qN+1rWhmifEBh1ftP2IargjdP1fFkiStrw4VFERdNIBQGkQLFBpnTs9uJU+/O8e6tX89FwcP+b41PXF7wq16d6ktibd063YvR0AgNN4xgKV1pYXgyff2vHyMH33m35l/twRf1uiJmOn6kiOT/9YsPOCbZbuOqo2z87Q377fIUkq8BudOJUf1ObkqQIdzvaVuR4ACEX0WMA2i3ccUZfGtQLDWI0xOpTt05GcU3pn/nZ9sWp/hdWy7KmBio9161S+Xy2fni5Jeu76NhrVq2mF1QAAlRm3QlDlPfTxSn21+nS42JU6XDm+fL01Z5seuPpyrd/v1fuLdmvq2gOWnKtZfHW9NeJKvT5rq6atzQg6tvWloax3AiDkESxQ5W0/nKOb31yo+/tdrjH9mxfZ7lLPbMx57GptPZitT9P2auaGg2Wua+mTA5iaHEDIIVjAEYwxcrkuPkw0bfdR3Tpx0QWPbX5xiNwR4YHXKdM26u35O8pc167U4fLlF6jV02dXhP3ttS3Vv3WCWifGKoIeDgAOQ7BASMnx5ataZLjCwlwXXUhNktKPnlCfP8zRgif66+HJq5S2+1iJz/fSze301JR1RR73REdozfODdSrfHxji+uqMTdpyMEfv/rJric8HAHYjWADFYIzR1X+cK1+eX1kn83Qyr0Dv3NVFg9om6r//uVzfbSz7rZPHB7fSvb2bqvUzp3s3Pri3h3r/tEQ9AFQVBAvAIi2fnq5T+X6Nv6GtnvtqvSWf+d1v+ql5Qg1Jki+/QBFhYTLGBN1CKc5tIACoKMX9/maWIOASzp13Y9eR43pv4a4yf+bACfMUHRmm3Dx/oWO7UocHPZAaH+vWsqcGlvmcAFAR6LEASuiuvy/R91t/lCT934O9ZWR0w18W6qpmtVU/LkZTVu6z/Jx3dm+kj5fukSS9eFM7Xd+xgTqO/1aStOTJAarHKBUA5YxbIUA5yi/wKzffrxru051+uXkFio48O/pkU4ZX905arqTaMVq842i517PymWtVq3qUNuz3qp7HrTo13Mor8Cvc5Spy8bXs3Dyt2JOpPs3rskAbgEsiWACV0EFvru57P02r0zPL9TznPg/y119cqbYN4jRh5hbVj4vWW3O3F2p/d88mev6GtjLGaO2+LLVrEEfYABCEYAFUcidO5avNszPUpr5HN3VuoBW7M/XN+rOzfvZpUVfXdaivJ/6ztkLqaVynmnYfORF4PXZoa43ud3ngtS+/QD1TZuuFm9ppWPv6gf08ZAqEBoIFUEX5/Saot+DFrzfobz8tqvaPu7vqnknLK6yW8Te0VUKsW3Exkfr535YE9k/5dU91blRLK/Yc0y1v/SBJ+usvuqhX8zqKjY4MtJu54aC+Wr1fzepW16PXtqywugFYj2ABOEhuXoEkKToyXBNmbtHrs7ZKkqY+1FvDX19gS03nj14pjsn3XaVuTWor/KfgZIzR+v1eXffGAj3Yv7keG9yqPEoFYAGCBeBgf5yxWfl+oyeGtArchrh14g9K231Mk0Z107ETpxRfI1rdm9YOrNaa6IlWhjdXkjQyubGeu76twsJcys7NU/vnv63Q+tOeHqgtB3N057uLg/bvSh1eoXUAKD6CBQBJwc9AFPU8xLk9D+cObbXLZ6OT1bVJbVtrABCMYAGg2Iwx2nvspBrWigkEjwvd5nhkYAv9+butFVpbr+Z1lHpLB8XHuoOG9J6vuA+R5hX49eLXG/Srvs3UsFY1K0sFHI1gAaBMCvxGq9KPyZfvV1KtampQM0bhYS5d86e52nH4eKDdr/o01VPD20g6vdT9B4t3q1l8DQ1rl6g6Ndw65M2VkdTj5Vllrum73/TTyj3HdF2HBvIbo/2ZJ9WwVjVd8ezZVWZTb2mvqWsP6L27uwWmSM/NK9A/Fu7UPb3OrtkicesFKAmCBYByceJUvj5cvEdX1Peodf1Y1a3hLtH7f9j2Y2CEyZC2ifrzHZ3ky/cHZhK1i10hI7/ArzX7stT+sriLrsoL2I1gAaBKWbrzqNbty9LXa/ZrxZ7MCj//ymeuVecXZko6PVV7+4Zxpf6sts9+o+OnCjRxxJUaes6cHxdy7i2nnSnDmBMElVa5BYv58+fr1VdfVVpamg4cOKApU6bopptusrwwAKFrdXqmEjxuvTxtk/5v9X5ba7m/XzN9sy5Dl8fX0K/6NFPy5XUCx278ywKt3pulL8f0Ul6BX/9ZsVcfL00Pev8/7+mumRsyFBEWpudvaBt0LG33Ud06cVHQvrFDW2vlnmOasf6glj41QAmxrAODyqHcgsX06dO1cOFCdenSRbfccgvBAkCFuNScGUufGqBqURF66OOVmr3p0EXbRoa7FF/Drf1ZuaWq5U+3ddT3Ww/ri1UlDz2bXhiiYydOKTlldrHan3+LJutEntYfyFL3n+YDoYcDFaVCboW4XC6CBYAKkZtXoCkr9ykh1q2vVu/Xn2/vJJfLVWg0iN9vtOGAV6M/SNPeYyclSQ9d01yjr75coz9YoR+2/ahNLwxRRHiYjDFqOm6aXb9Ssax+bpA6jv9Wd13VWO8v3n3BNpNGdVPy5XXkjjg9auZMCFvxzLWqXT2qwmqFs1WaYOHz+eTz+YIKS0pKIlgAKHcfLdmjq1vFq0HNmIu2O7c35O27uuj+99PKu7RyUad6lMLCXDqcffa/uUPaJgbWoDn3GY4zv/OWF4cqKuLsQ6O3/fUHLdt1TGlPD1SdEj6YC2erNMHi+eef1/jx4wvtJ1gAqCyMMerx8iyN7NlEY/o3147DOUrwRMslqbo7Ql+s3KcPl+zWsl3HLvlZZ768dxzO0TV/mles8+9KHa6sE3nq+PuKGRnTuVFNrTznAdmtLw3V+v1e3fTmwqB2218eFph+/VzHjp9Sj5RZmjjiSg24ol55l4tKotIEC3osADjF5KV7NPbz4NVmOzSM05q9WZKk73/XX0m1z0669f7i3Trhy1fflvG6or5HL03doHe/3xk4/sx1bfSLqxoFbmEs2XFEt78TPM35+Zon1NC2QzlW/UqW6NgwTl+M6RV0S+pMj0jqLe11R/dGdpUGC1WaYFHawgCgqjiS4yv2bYPcvAJ9mrZX/VvFX3Dmzw8W79b3Ww/rtTs6a9eR4/KezFf60RP67aerA8NX92WeVK/U2bq/XzO9PW+H1b9OqW38/RDFRIWrwG90+ZNnn105fxitMUatn/lGCR63rr0iUc9cd0WhWzTzH++vRnWYGbUyIVgAQAjZcjBbg/53voa1T9TTw9uoZ+rpUSe3d01S/9YJGv1B4edGGsRFX3BkTHRkmHLz/KWq43dDWmlQm0QNnHD2NtAtnS/ThNs7BR60vXbCPG09r9clIdatiDBXUD3nBhJjjJ77ar3+tWi3ft6jkV6+ub0kacHWH/WLvy9R68RYbcrI1qheTfTc9cHDemGNcgsWOTk52rZtmySpc+fOmjBhgvr376/atWurUaNLd3cRLACg/G0/nCO/36hFvdig/bl5Bfpy1T7d2OkyRUeGFxrG+9+9m+rp69oor8Cv336yWl8VMY/IP+7uqjrV3brxvOcyrPTrqy/X9HUZSrmlve447xbRa3d00rVt6qnNszMKve+Gjg30+p2dJUnr92epad3q2p+Zq/SjJ9S/dUJQ29y8ArkjwgqNLBr7+Rpd16GB+raML4ffrGoqt2Axd+5c9e/fv9D+kSNHatKkSZYVBgAof1sPZuuFqRv15s87KzfPr7o1ooK+ZA96czVx7nb9+urLleCJVkZWrup53IE2uXkFQeuvVBbTHuqjYa9/X2j/3T2bKCYqXBPnbg/aP/exq/XytI3yxETqs7S9gf1FTfVujFGB3wTWowkFTOkNAKgQh7Jz1f2lwovMvfnzKzXmoxU2VGSdrx7spQ4NawbtO+jNDVpU73dDWum+Ps1001sLlZGVq/jYaE39n94Ku8CImtIwxijfb2xfS4ZgAQCoMMt2HdVtfz07Pfk/7+mufi3ji5wxddKobrq6VcIFj385pleRt1geHtBCr83aak3RpVCa8y99aoBGvbdMk0Z1V3ysW/syT6pBXHSg1+fY8VPal3lSEeEuRUeE6+o/zg28d8fLw9Tspwdh/z6yq63DewkWAIAKt/vIcdWuHqXY6MjAvqU7j2pzhlcxURE67svXyJ5Ngt5T4DcKc0mfLE/XtW0SVbt6lPx+o1GTlinMJXliIlXPE63HBrVSVERY0KiTv4/sqmtaJ1T6WzNnnDuD6raXhsrlcgWNoLmUXanD9e9le/TEf04Pe76tS0Pd0b2Rbp34gyRp0bhrVLt6VGAIs5UIFgAAxzLGyBhd8HbD0p1H9bO3T/ee9GsZr3lbDkuSRvVqovcW7tLgtvX06m0dFeuOkMvl0qyNB3XvP5cHfcb68YPV9rnCD4ZWFUU9G1IWBAsAQMg7sx5Mw1oxWvDENRdt6/eboKBy9PgpXfnCzCLbr352kL7dkKHHP1sT2NcioUahobR2WPrkACV4rF0Zt7jf3xGWnhUAgErE5XIV+//ez+/9qF09KvDeAr/R4h1H9P6i3fpmfYa+GNNLcdUidVvXJM3bclhfrzmgf993lXo0qyNvbp4WbP1RjWpX03VvLNC4oa2VMn1TierelTo88PzJHd2SNHlZeone78sv3TwkVqDHAgCAMjDGyJubr7iYyCLbzFifcdHF7ba+NFT5BUavfLNJjwxsoZrVglel3XjAq6GvBQ+f7d6ktt74eWclxLqDVuld8uQA1bO4t0LiVggAAJVOXoFfxkhbD2WrTnW3VqVnqmnd6mqVGHvpN/9kw36v6sZGKSH2bHg4cSpfd/19qZ4cdoW6NK5VHqUTLAAAgHWK+/0dOlOGAQCAckewAAAAliFYAAAAyxAsAACAZQgWAADAMgQLAABgGYIFAACwDMECAABYhmABAAAsQ7AAAACWIVgAAADLECwAAIBlCBYAAMAyERV9wjOLqXq93oo+NQAAKKUz39uXWhS9woNFdna2JCkpKamiTw0AAMooOztbcXFxRR53mUtFD4v5/X7t379fsbGxcrlcln2u1+tVUlKS0tPTL7pOPCoPrlnVwzWrerhmVU9lvWbGGGVnZ6tBgwYKCyv6SYoK77EICwtTw4YNy+3zPR5PpboQuDSuWdXDNat6uGZVT2W8ZhfrqTiDhzcBAIBlCBYAAMAyjgkWbrdbzz33nNxut92loJi4ZlUP16zq4ZpVPVX9mlX4w5sAAMC5HNNjAQAA7EewAAAAliFYAAAAyxAsAACAZRwTLN588001adJE0dHR6tGjh5YuXWp3SY40f/58XX/99WrQoIFcLpe++OKLoOPGGD377LOqX7++YmJiNHDgQG3dujWozdGjRzVixAh5PB7VrFlT9957r3JycoLarFmzRn369FF0dLSSkpL0hz/8oVAtn376qVq3bq3o6Gi1b99e06ZNs/z3repSUlLUrVs3xcbGKiEhQTfddJM2b94c1CY3N1djxoxRnTp1VKNGDd166606ePBgUJs9e/Zo+PDhqlatmhISEvT4448rPz8/qM3cuXN15ZVXyu12q3nz5po0aVKhevg7vbSJEyeqQ4cOgcmRkpOTNX369MBxrlfll5qaKpfLpUceeSSwL6Sum3GAyZMnm6ioKPOPf/zDrF+/3vzqV78yNWvWNAcPHrS7NMeZNm2aeeqpp8znn39uJJkpU6YEHU9NTTVxcXHmiy++MKtXrzY33HCDadq0qTl58mSgzZAhQ0zHjh3N4sWLzffff2+aN29u7rzzzsDxrKwsU69ePTNixAizbt068/HHH5uYmBjz9ttvB9osXLjQhIeHmz/84Q9mw4YN5umnnzaRkZFm7dq15f5vUJUMHjzYvPfee2bdunVm1apVZtiwYaZRo0YmJycn0Gb06NEmKSnJzJo1yyxfvtxcddVVpmfPnoHj+fn5pl27dmbgwIFm5cqVZtq0aaZu3bpm3LhxgTY7duww1apVM7/5zW/Mhg0bzBtvvGHCw8PNN998E2jD32nxfPXVV2bq1Klmy5YtZvPmzebJJ580kZGRZt26dcYYrldlt3TpUtOkSRPToUMH8/DDDwf2h9J1c0Sw6N69uxkzZkzgdUFBgWnQoIFJSUmxsSrnOz9Y+P1+k5iYaF599dXAvszMTON2u83HH39sjDFmw4YNRpJZtmxZoM306dONy+Uy+/btM8YY89Zbb5latWoZn88XaPPEE0+YVq1aBV7/7Gc/M8OHDw+qp0ePHub++++39Hd0mkOHDhlJZt68ecaY09cnMjLSfPrpp4E2GzduNJLMokWLjDGnw2RYWJjJyMgItJk4caLxeDyBa/S73/3OtG3bNuhct99+uxk8eHDgNX+npVerVi3zt7/9jetVyWVnZ5sWLVqYmTNnmn79+gWCRahdtyp/K+TUqVNKS0vTwIEDA/vCwsI0cOBALVq0yMbKQs/OnTuVkZERdC3i4uLUo0ePwLVYtGiRatasqa5duwbaDBw4UGFhYVqyZEmgTd++fRUVFRVoM3jwYG3evFnHjh0LtDn3PGfacM0vLisrS5JUu3ZtSVJaWpry8vKC/i1bt26tRo0aBV2z9u3bq169eoE2gwcPltfr1fr16wNtLnY9+DstnYKCAk2ePFnHjx9XcnIy16uSGzNmjIYPH17o3zbUrluFL0JmtR9//FEFBQVBF0OS6tWrp02bNtlUVWjKyMiQpAteizPHMjIylJCQEHQ8IiJCtWvXDmrTtGnTQp9x5litWrWUkZFx0fOgML/fr0ceeUS9evVSu3btJJ3+94yKilLNmjWD2p5/zS70b33m2MXaeL1enTx5UseOHePvtATWrl2r5ORk5ebmqkaNGpoyZYratGmjVatWcb0qqcmTJ2vFihVatmxZoWOh9ndW5YMFgOIZM2aM1q1bpwULFthdCi6hVatWWrVqlbKysvTZZ59p5MiRmjdvnt1loQjp6el6+OGHNXPmTEVHR9tdju2q/K2QunXrKjw8vNDTtQcPHlRiYqJNVYWmM//eF7sWiYmJOnToUNDx/Px8HT16NKjNhT7j3HMU1YZrfmEPPvigvv76a82ZM0cNGzYM7E9MTNSpU6eUmZkZ1P78a1ba6+HxeBQTE8PfaQlFRUWpefPm6tKli1JSUtSxY0e99tprXK9KKi0tTYcOHdKVV16piIgIRUREaN68eXr99dcVERGhevXqhdR1q/LBIioqSl26dNGsWbMC+/x+v2bNmqXk5GQbKws9TZs2VWJiYtC18Hq9WrJkSeBaJCcnKzMzU2lpaYE2s2fPlt/vV48ePQJt5s+fr7y8vECbmTNnqlWrVqpVq1agzbnnOdOGax7MGKMHH3xQU6ZM0ezZswvdYurSpYsiIyOD/i03b96sPXv2BF2ztWvXBgXCmTNnyuPxqE2bNoE2F7se/J2Wjd/vl8/n43pVUgMGDNDatWu1atWqwE/Xrl01YsSIwHZIXbcKe0y0HE2ePNm43W4zadIks2HDBnPfffeZmjVrBj1dC2tkZ2eblStXmpUrVxpJZsKECWblypVm9+7dxpjTw01r1qxpvvzyS7NmzRpz4403XnC4aefOnc2SJUvMggULTIsWLYKGm2ZmZpp69eqZu+66y6xbt85MnjzZVKtWrdBw04iICPPHP/7RbNy40Tz33HMMN72ABx54wMTFxZm5c+eaAwcOBH5OnDgRaDN69GjTqFEjM3v2bLN8+XKTnJxskpOTA8fPDIMbNGiQWbVqlfnmm29MfHz8BYfBPf7442bjxo3mzTffvOAwOP5OL23s2LFm3rx5ZufOnWbNmjVm7NixxuVymW+//dYYw/WqKs4dFWJMaF03RwQLY4x54403TKNGjUxUVJTp3r27Wbx4sd0lOdKcOXOMpEI/I0eONMacHnL6zDPPmHr16hm3220GDBhgNm/eHPQZR44cMXfeeaepUaOG8Xg8ZtSoUSY7OzuozerVq03v3r2N2+02l112mUlNTS1UyyeffGJatmxpoqKiTNu2bc3UqVPL7feuqi50rSSZ9957L9Dm5MmT5te//rWpVauWqVatmrn55pvNgQMHgj5n165dZujQoSYmJsbUrVvX/Pa3vzV5eXlBbebMmWM6depkoqKiTLNmzYLOcQZ/p5d2zz33mMaNG5uoqCgTHx9vBgwYEAgVxnC9qorzg0UoXTeWTQcAAJap8s9YAACAyoNgAQAALEOwAAAAliFYAAAAyxAsAACAZQgWAADAMgQLAABgGYIFAACwDMECAABYhmABAAAsQ7AAAACWIVgAAADL/D+s32CoxRXQ6wAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "pd.Series(losses).ewm(100).mean().plot();"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6MGVf4Vc_fS4"
      },
      "outputs": [],
      "source": [
        "def translate(text, src_lang='rus_Cyrl', tgt_lang='eng_Latn', a=16, b=1.5, max_input_length=1024, **kwargs):\n",
        "    tokenizer.src_lang = src_lang\n",
        "    tokenizer.tgt_lang = tgt_lang\n",
        "    inputs = tokenizer(text, return_tensors='pt', padding=True, truncation=True, max_length=max_input_length)\n",
        "    result = model.generate(\n",
        "        **inputs.to(model.device),\n",
        "        forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),\n",
        "        max_new_tokens=int(a + b * inputs.input_ids.shape[1]),\n",
        "        **kwargs\n",
        "    )\n",
        "    #print(inputs.input_ids.shape[1], result.shape[1])\n",
        "    return tokenizer.batch_decode(result, skip_special_tokens=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c69XqtpbAgjN",
        "outputId": "2b963659-10e1-4cfc-fe20-ef136aef75e8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "['бир-ле черден соок үрүп тур']\n",
            "['откуда-то несёт холодом']\n",
            "['откуда-то дует холод']\n"
          ]
        }
      ],
      "source": [
        "xx, yy, lang1, lang2 = get_batch_pairs(1, data=df_dev)\n",
        "print(xx)\n",
        "print(yy)\n",
        "model.eval()\n",
        "print(translate(xx[0], lang1, lang2, no_repeat_ngram_size=3, num_beams=5))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aCZR50GxAiPJ",
        "outputId": "4815110a-b8eb-4bc5-9453-977cb14d146d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "total 2.3G\n",
            "1.0K -rw------- 1 root root  896 Sep 30 07:40 config.json\n",
            " 512 -rw------- 1 root root  184 Sep 30 07:40 generation_config.json\n",
            "2.3G -rw------- 1 root root 2.3G Sep 30 07:41 pytorch_model.bin\n",
            "4.7M -rw------- 1 root root 4.7M Sep 30 07:41 sentencepiece.bpe.model\n",
            "3.5K -rw------- 1 root root 3.5K Sep 30 07:41 special_tokens_map.json\n",
            "1.0K -rw------- 1 root root  570 Sep 30 07:41 tokenizer_config.json\n"
          ]
        }
      ],
      "source": [
        "!ls -alsh $MODEL_SAVE_PATH"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 6. Using the model"
      ],
      "metadata": {
        "id": "0qubmjZNAxJB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "from transformers import NllbTokenizer, AutoModelForSeq2SeqLM, AutoConfig\n",
        "from tqdm.auto import tqdm, trange"
      ],
      "metadata": {
        "id": "PKGZ8zuN2mV6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "trans_df = pd.read_csv('/gd/MyDrive/datasets/nlp/tyvan/for_translator.csv')\n",
        "trans_df.dropna(subset=['ru', 'tyv'], inplace=True)\n",
        "df_train, df_devtest = train_test_split(trans_df, test_size=1000, random_state=1)\n",
        "df_dev, df_test = train_test_split(df_devtest, test_size=0.5, random_state=1)"
      ],
      "metadata": {
        "id": "hag683KM2qxZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# this code is adapted from  the Stopes repo of the NLLB team\n",
        "# https://github.com/facebookresearch/stopes/blob/main/stopes/pipelines/monolingual/monolingual_line_processor.py#L214\n",
        "\n",
        "import re\n",
        "import sys\n",
        "import typing as tp\n",
        "import unicodedata\n",
        "from sacremoses import MosesPunctNormalizer\n",
        "\n",
        "\n",
        "mpn = MosesPunctNormalizer(lang=\"en\")\n",
        "mpn.substitutions = [\n",
        "    (re.compile(r), sub) for r, sub in mpn.substitutions\n",
        "]\n",
        "\n",
        "\n",
        "def get_non_printing_char_replacer(replace_by: str = \" \") -> tp.Callable[[str], str]:\n",
        "    non_printable_map = {\n",
        "        ord(c): replace_by\n",
        "        for c in (chr(i) for i in range(sys.maxunicode + 1))\n",
        "        # same as \\p{C} in perl\n",
        "        # see https://www.unicode.org/reports/tr44/#General_Category_Values\n",
        "        if unicodedata.category(c) in {\"C\", \"Cc\", \"Cf\", \"Cs\", \"Co\", \"Cn\"}\n",
        "    }\n",
        "\n",
        "    def replace_non_printing_char(line) -> str:\n",
        "        return line.translate(non_printable_map)\n",
        "\n",
        "    return replace_non_printing_char\n",
        "\n",
        "replace_nonprint = get_non_printing_char_replacer(\" \")\n",
        "\n",
        "def preproc(text):\n",
        "    clean = mpn.normalize(text)\n",
        "    clean = replace_nonprint(clean)\n",
        "    # replace 𝓕𝔯𝔞𝔫𝔠𝔢𝔰𝔠𝔞 by Francesca\n",
        "    clean = unicodedata.normalize(\"NFKC\", clean)\n",
        "    return clean"
      ],
      "metadata": {
        "id": "0AXtm-Qf2wCR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def fix_tokenizer(tokenizer, new_lang='tyv_Cyrl'):\n",
        "    \"\"\" Add a new language token to the tokenizer vocabulary (this should be done each time after its initialization) \"\"\"\n",
        "    old_len = len(tokenizer) - int(new_lang in tokenizer.added_tokens_encoder)\n",
        "    tokenizer.lang_code_to_id[new_lang] = old_len-1\n",
        "    tokenizer.id_to_lang_code[old_len-1] = new_lang\n",
        "    # always move \"mask\" to the last position\n",
        "    tokenizer.fairseq_tokens_to_ids[\"<mask>\"] = len(tokenizer.sp_model) + len(tokenizer.lang_code_to_id) + tokenizer.fairseq_offset\n",
        "\n",
        "    tokenizer.fairseq_tokens_to_ids.update(tokenizer.lang_code_to_id)\n",
        "    tokenizer.fairseq_ids_to_tokens = {v: k for k, v in tokenizer.fairseq_tokens_to_ids.items()}\n",
        "    if new_lang not in tokenizer._additional_special_tokens:\n",
        "        tokenizer._additional_special_tokens.append(new_lang)\n",
        "    # clear the added token encoder; otherwise a new token may end up there by mistake\n",
        "    tokenizer.added_tokens_encoder = {}\n",
        "    tokenizer.added_tokens_decoder = {}"
      ],
      "metadata": {
        "id": "Wwb6ck8P25ZQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model_load_name = '/gd/MyDrive/models/nllb-rus-tyv-v1'\n",
        "model = AutoModelForSeq2SeqLM.from_pretrained(model_load_name).cuda()\n",
        "tokenizer = NllbTokenizer.from_pretrained(model_load_name)\n",
        "fix_tokenizer(tokenizer)"
      ],
      "metadata": {
        "id": "uY7nUGsX3NOM",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "84976f43-9775-443d-ba5e-7da564be2ed4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def translate(text, src_lang='rus_Cyrl', tgt_lang='eng_Latn', a=32, b=3, max_input_length=1024, num_beams=4, **kwargs):\n",
        "    tokenizer.src_lang = src_lang\n",
        "    tokenizer.tgt_lang = tgt_lang\n",
        "    inputs = tokenizer(text, return_tensors='pt', padding=True, truncation=True, max_length=max_input_length)\n",
        "    result = model.generate(\n",
        "        **inputs.to(model.device),\n",
        "        forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),\n",
        "        max_new_tokens=int(a + b * inputs.input_ids.shape[1]),\n",
        "        num_beams=num_beams,\n",
        "        **kwargs\n",
        "    )\n",
        "    return tokenizer.batch_decode(result, skip_special_tokens=True)"
      ],
      "metadata": {
        "id": "ZIsPI6YT3UG0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "t = \"мөңгүн үр чыткаш карарар\"\n",
        "print(translate(t, 'tyv_Cyrl', 'rus_Cyrl'))\n",
        "# ['серебро от времени чернеет']"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UJwLBH8M9XWW",
        "outputId": "8cd3007f-6b6e-4364-ca99-991efe0d719e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['серебро от времени чернеет']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "translate(t, 'tyv_Cyrl', 'rus_Cyrl', do_sample=True, num_beams=1, temperature=1.5)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "o9JFXvBS9xY7",
        "outputId": "09a8e62c-d727-4f72-8915-bed8a0e4498c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['серебро от ходения цвета']"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def batched_translate(texts, batch_size=16, **kwargs):\n",
        "    \"\"\"Translate texts in batches of similar length\"\"\"\n",
        "    idxs, texts2 = zip(*sorted(enumerate(texts), key=lambda p: len(p[1]), reverse=True))\n",
        "    results = []\n",
        "    for i in trange(0, len(texts2), batch_size):\n",
        "        results.extend(translate(texts2[i: i+batch_size], **kwargs))\n",
        "    return [p for i, p in sorted(zip(idxs, results))]"
      ],
      "metadata": {
        "id": "JoWvizFCRngQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "rus_translated = batched_translate(df_dev.tyv, src_lang='tyv_Cyrl', tgt_lang='rus_Cyrl')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "ef092016e6e64903b967170f1865e82f",
            "363e0d43df564854af2e6caa053e56c5",
            "7f363c96320042788c50184e91c8f154",
            "eb8a113b5f2c4d1997adafe1ad44946e",
            "9c7ab640a7e64a9db9e912a315f8929e",
            "3173699ecb454b75be1d493c0681beca",
            "5efff913d4dc4631939a6642c2343cb7",
            "cf9bdd81f6574f5bad3f9f878a6ad847",
            "48d4597749b5459b93f2480064e8015e",
            "2b40ca2c5862407ea4dfff008dbb6649",
            "1743e3e64cf540ac902d423a05b94305"
          ]
        },
        "id": "2-wPl4fTRlv2",
        "outputId": "279b6d2f-7786-4525-e72b-35780643785b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/16 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ef092016e6e64903b967170f1865e82f"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev['rus_translated'] = [translate(t, 'tyv_Cyrl', 'rus_Cyrl')[0] for t in tqdm(df_dev.tyv)]\n",
        "df_dev['tyv_translated'] = [translate(t, 'rus_Cyrl', 'tyv_Cyrl')[0] for t in tqdm(df_dev.ru)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81,
          "referenced_widgets": [
            "3a2c606d8688424aa3b325223cfc1f08",
            "5ddb689b97db48aaa93001a91b86a4a2",
            "01d711c111174832ab5bba94cfbca2a6",
            "b3aeb847f0374c2ba47ec22c60ab2f18",
            "b7466c53e33c4b23b9186821609eefa1",
            "fc8279d880854b5bb526cadc437bc317",
            "4b9195f13aa34a87a2859654902ef99a",
            "804c3b981cc740b79f700e7cecf9362e",
            "4ef9639ade9e49189272e6b8012223e8",
            "f87dfba0b04e42cabd525af31afae2ca",
            "a82b144e2c024fc8822def3701384c12",
            "0ea7235036c14d438a412ae6fe857ab0",
            "7b7348cd7da243158c0734488b422b74",
            "4aaad174030140a69db1e81361553902",
            "8d6093633eb44e2c9b499a1594e2b6dc",
            "2beeb8b43c3c4153b7723184ec6ab123",
            "2548884655614dfda0d263cacd665c07",
            "70099123340b4adca9f18dcd23cbb270",
            "f3782ed9cd074d43a69e2978eb7b0865",
            "3feb136a616d46b7a0100d2a507cbd95",
            "b9dfdf81307c4615b00ba80b3eef2958",
            "2ea8b05ab5f540efa609befac402a7d1"
          ]
        },
        "id": "k3S475uG3fBh",
        "outputId": "abacfc33-ac7e-4471-c5f1-e1a5c9f8aabc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/500 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3a2c606d8688424aa3b325223cfc1f08"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/500 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "0ea7235036c14d438a412ae6fe857ab0"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import sacrebleu\n",
        "bleu_calc = sacrebleu.BLEU()\n",
        "chrf_calc = sacrebleu.CHRF(word_order=2)  # this metric is called ChrF++"
      ],
      "metadata": {
        "id": "FMRSCWW732ya"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "xx, yy = ['течёт холод'], ['несёт холодом']\n",
        "print(bleu_calc.corpus_score(xx, [yy]))\n",
        "print(chrf_calc.corpus_score(xx, [yy]))\n",
        "print(chrf_calc.corpus_score(yy, [xx]))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gSwL7o6DMy8M",
        "outputId": "df5fea7c-814d-4a8e-ffa0-85ba7e84c3cf"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "BLEU = 0.00 0.0/0.0/0.0/0.0 (BP = 1.000 ratio = 1.000 hyp_len = 2 ref_len = 2)\n",
            "chrF2++ = 36.75\n",
            "chrF2++ = 42.30\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(bleu_calc.corpus_score(df_dev['rus_translated'].tolist(), [df_dev['ru'].tolist()]))\n",
        "print(chrf_calc.corpus_score(df_dev['rus_translated'].tolist(), [df_dev['ru'].tolist()]))\n",
        "print(bleu_calc.corpus_score(df_dev['tyv_translated'].tolist(), [df_dev['tyv'].tolist()]))\n",
        "print(chrf_calc.corpus_score(df_dev['tyv_translated'].tolist(), [df_dev['tyv'].tolist()]))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7NKKUyXZ4oXr",
        "outputId": "c3d0a0b6-9782-4aa1-e948-b35b06364ef9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "BLEU = 24.14 52.5/30.4/18.9/12.1 (BP = 0.981 ratio = 0.981 hyp_len = 2281 ref_len = 2324)\n",
            "chrF2++ = 49.49\n",
            "BLEU = 23.41 52.1/31.0/18.9/11.3 (BP = 0.966 ratio = 0.967 hyp_len = 2292 ref_len = 2371)\n",
            "chrF2++ = 50.89\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pd.options.display.max_colwidth = 100"
      ],
      "metadata": {
        "id": "svplVgTB5_Xq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev.sample(10, random_state=5)[['tyv', 'ru', 'tyv_translated', 'rus_translated']]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 481
        },
        "id": "i3bMbXUv5TsV",
        "outputId": "45dd9c76-35ec-45bf-b878-abfa8f5b53c7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                      tyv                            ru  \\\n",
              "row_id                                                                    \n",
              "37635                     ыяк сактып алыр                намотать на ус   \n",
              "42465                        чиик үлетпүр         лёгкая промышленность   \n",
              "27916                    аяк бусту берген               чашка разбилась   \n",
              "78401            оларны маш-салатче салыр      выложить их на маш-салат   \n",
              "74168   соус хоюг кылдыр хайынмаан шаанда  пока соус не заварится густо   \n",
              "61972                                шиви                           ель   \n",
              "8377          соондан арай боорда шимчээр                тащиться вслед   \n",
              "109242                                АКШ                           США   \n",
              "38832         бир-ле черден соок үрүп тур       откуда-то несёт холодом   \n",
              "54814                         быжыктырары                    укрепление   \n",
              "\n",
              "                        tyv_translated               rus_translated  \n",
              "row_id                                                               \n",
              "37635                  аксынга өттүрер             твёрдо запомнить  \n",
              "42465                     чиик үлетпүр        лёгкая промышленность  \n",
              "27916                 аяк бусту берген                чашка лопнула  \n",
              "78401         оларны маш-салатче салыр      выложить их в маш-салат  \n",
              "74168       соус хоюг хайынмаан шаанда  пока соус не сварится густо  \n",
              "61972                             шиви                еловая кобыла  \n",
              "8377                   соондан чүткүүр          еле двинуться вслед  \n",
              "109242                             АКШ                          США  \n",
              "38832   бир-ле черден соок аппар чыдыр        откуда-то течёт холод  \n",
              "54814                            шивээ                   укрепление  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-ebf39333-db66-45aa-9c32-f7e2c39164d7\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tyv</th>\n",
              "      <th>ru</th>\n",
              "      <th>tyv_translated</th>\n",
              "      <th>rus_translated</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>row_id</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>37635</th>\n",
              "      <td>ыяк сактып алыр</td>\n",
              "      <td>намотать на ус</td>\n",
              "      <td>аксынга өттүрер</td>\n",
              "      <td>твёрдо запомнить</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>42465</th>\n",
              "      <td>чиик үлетпүр</td>\n",
              "      <td>лёгкая промышленность</td>\n",
              "      <td>чиик үлетпүр</td>\n",
              "      <td>лёгкая промышленность</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27916</th>\n",
              "      <td>аяк бусту берген</td>\n",
              "      <td>чашка разбилась</td>\n",
              "      <td>аяк бусту берген</td>\n",
              "      <td>чашка лопнула</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>78401</th>\n",
              "      <td>оларны маш-салатче салыр</td>\n",
              "      <td>выложить их на маш-салат</td>\n",
              "      <td>оларны маш-салатче салыр</td>\n",
              "      <td>выложить их в маш-салат</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>74168</th>\n",
              "      <td>соус хоюг кылдыр хайынмаан шаанда</td>\n",
              "      <td>пока соус не заварится густо</td>\n",
              "      <td>соус хоюг хайынмаан шаанда</td>\n",
              "      <td>пока соус не сварится густо</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>61972</th>\n",
              "      <td>шиви</td>\n",
              "      <td>ель</td>\n",
              "      <td>шиви</td>\n",
              "      <td>еловая кобыла</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8377</th>\n",
              "      <td>соондан арай боорда шимчээр</td>\n",
              "      <td>тащиться вслед</td>\n",
              "      <td>соондан чүткүүр</td>\n",
              "      <td>еле двинуться вслед</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>109242</th>\n",
              "      <td>АКШ</td>\n",
              "      <td>США</td>\n",
              "      <td>АКШ</td>\n",
              "      <td>США</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>38832</th>\n",
              "      <td>бир-ле черден соок үрүп тур</td>\n",
              "      <td>откуда-то несёт холодом</td>\n",
              "      <td>бир-ле черден соок аппар чыдыр</td>\n",
              "      <td>откуда-то течёт холод</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>54814</th>\n",
              "      <td>быжыктырары</td>\n",
              "      <td>укрепление</td>\n",
              "      <td>шивээ</td>\n",
              "      <td>укрепление</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ebf39333-db66-45aa-9c32-f7e2c39164d7')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-ebf39333-db66-45aa-9c32-f7e2c39164d7 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-ebf39333-db66-45aa-9c32-f7e2c39164d7');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-daa8beea-0d43-47cb-9d2f-36d44dcfbc7d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-daa8beea-0d43-47cb-9d2f-36d44dcfbc7d')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-daa8beea-0d43-47cb-9d2f-36d44dcfbc7d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 64
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print((df_dev.ru == df_dev.rus_translated).mean())\n",
        "print((df_dev.tyv == df_dev.tyv_translated).mean())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uCEO4dSgJYKE",
        "outputId": "a1e96e0d-4760-4749-c54b-81bd272a205f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.26\n",
            "0.25\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install editdistance"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9NEfm2fmJm1S",
        "outputId": "e41a7dd6-4b67-4bcc-ba9b-61b01f319631"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: editdistance in /usr/local/lib/python3.10/dist-packages (0.6.2)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import editdistance\n",
        "\n",
        "def ed_similarity(text1, text2):\n",
        "    return max(0, 1 - editdistance.eval(text1, text2) / min(len(text1), len(text2)))\n",
        "\n",
        "print(ed_similarity('кот', 'собака'))\n",
        "print(ed_similarity('кот', 'кит'))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NxxFrdC7JrS6",
        "outputId": "52a5e77e-34e4-4733-aead-6f83d14be049"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0\n",
            "0.6666666666666667\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pd.Series([ed_similarity(row.ru, row.rus_translated) for row in df_dev.itertuples()]).describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EjkvAyN9JyXg",
        "outputId": "6df153cb-e719-4bbf-d7aa-78311cfdb05c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count    500.000000\n",
              "mean       0.516367\n",
              "std        0.392761\n",
              "min        0.000000\n",
              "25%        0.116013\n",
              "50%        0.507009\n",
              "75%        1.000000\n",
              "max        1.000000\n",
              "dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 74
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pd.Series([ed_similarity(row.tyv, row.tyv_translated) for row in df_dev.itertuples()]).describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NXm7oczQKknv",
        "outputId": "ff5566f1-2f21-4991-8bc9-7e3940c83a05"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count    500.000000\n",
              "mean       0.506007\n",
              "std        0.382357\n",
              "min        0.000000\n",
              "25%        0.111111\n",
              "50%        0.504902\n",
              "75%        0.979730\n",
              "max        1.000000\n",
              "dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev.index.name = \"row_id\""
      ],
      "metadata": {
        "id": "-OL-s6bK6UIE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev.to_csv(model_load_name + \"/dev_set_translated.tsv\", sep=\"\\t\")"
      ],
      "metadata": {
        "id": "DjmFAfB355Ss"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Evaluating another model (with extended vocabulary)"
      ],
      "metadata": {
        "id": "66sAzGnX6clC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model_load_name = '/gd/MyDrive/models/nllb-rus-tyv-v2-extvoc'"
      ],
      "metadata": {
        "id": "CsCfoNc26fhi"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "cfg = AutoConfig.from_pretrained(model_load_name)\n",
        "model = AutoModelForSeq2SeqLM.from_pretrained(model_load_name + \"/pytorch_model_60k.bin\", config=cfg).cuda()"
      ],
      "metadata": {
        "id": "7Ic0aPUj6kS5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer = NllbTokenizer.from_pretrained(model_load_name)\n",
        "fix_tokenizer(tokenizer)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "x0QRieQi6pMA",
        "outputId": "7485184b-576d-4e99-d7a4-c709ce37c585"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev['rus_translated2'] = [translate(t, 'tyv_Cyrl', 'rus_Cyrl')[0] for t in tqdm(df_dev.tyv)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "fa02790d3658464d85ba4fb817598142",
            "009f9de0888749509335e97d828bd8b8",
            "98f0b0ff99004eccbabba4d9859e7191",
            "7204e6c548c9423aadbecc7656976f0d",
            "4d25116204ee4c5999dac8e2281a1962",
            "8e7acded176c4fefa122b50b0d3126c6",
            "a4ff5a65bd224cb5bc201da43cab714f",
            "31f1fa84b19e4e6bb3e0391290539334",
            "c2928804b48a423a8c5d85dc8fd0d624",
            "c344baeff36447e49c4d99cadf39b721",
            "6c5aee3f42cb467e904e6c70e1123b4d"
          ]
        },
        "id": "5-s-VKmn7Lzk",
        "outputId": "a6bbec1b-a0e3-409e-8a11-c157674dc9ad"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/500 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "fa02790d3658464d85ba4fb817598142"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev['tyv_translated2'] = [translate(t, 'rus_Cyrl', 'tyv_Cyrl')[0] for t in tqdm(df_dev.ru)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "e0bf99ac22274196bd5fd39fd08c843e",
            "e64a9117c6b848fb914799426f87a000",
            "50dfdefc8d8149b3a1d5f2bee94e88ab",
            "3feb373ccc2d4999956a105e8502fd49",
            "7baf6b1140284f5780331f1a32d7be65",
            "e1f3cea2562f4e73b4e7158f6592c1df",
            "1c1fc5decda14cf6bc056a8da1f603f5",
            "49ed7c03056f42a281f323a0bf6fe81f",
            "c7d605ece9554217aeb6d75f468a541d",
            "aff14b0c3dbb40feb14bb235dcd53aa1",
            "055db7a3c88345af8025ad5b0d3b77b5"
          ]
        },
        "id": "F5qW8jzN7QmG",
        "outputId": "d56f8839-f3e8-48b4-e0c7-502ec1da0fee"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/500 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e0bf99ac22274196bd5fd39fd08c843e"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(bleu_calc.corpus_score(df_dev['rus_translated2'].tolist(), [df_dev['ru'].tolist()]))\n",
        "print(chrf_calc.corpus_score(df_dev['rus_translated2'].tolist(), [df_dev['ru'].tolist()]))\n",
        "print(bleu_calc.corpus_score(df_dev['tyv_translated2'].tolist(), [df_dev['tyv'].tolist()]))\n",
        "print(chrf_calc.corpus_score(df_dev['tyv_translated2'].tolist(), [df_dev['tyv'].tolist()]))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R5YCOHIw7TgW",
        "outputId": "35e04aac-d92f-4775-9655-49580721e8b2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "BLEU = 25.18 52.4/31.3/20.4/13.3 (BP = 0.976 ratio = 0.976 hyp_len = 2269 ref_len = 2324)\n",
            "chrF2++ = 49.85\n",
            "BLEU = 23.22 51.6/29.4/18.3/11.6 (BP = 0.975 ratio = 0.975 hyp_len = 2312 ref_len = 2371)\n",
            "chrF2++ = 49.87\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = AutoModelForSeq2SeqLM.from_pretrained(model_load_name).cuda()"
      ],
      "metadata": {
        "id": "g_HVwJbi88LF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev['rus_translated3'] = [translate(t, 'tyv_Cyrl', 'rus_Cyrl')[0] for t in tqdm(df_dev.tyv)]\n",
        "df_dev['tyv_translated3'] = [translate(t, 'rus_Cyrl', 'tyv_Cyrl')[0] for t in tqdm(df_dev.ru)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81,
          "referenced_widgets": [
            "0de79c6d3066479d993f394472b0ceb5",
            "4ca4884ea47145d5a5c5c77133e8d8d5",
            "7529d94c31de4517818a94ec4530dc47",
            "32f1be0d6f374067b074e69b65a465e5",
            "00e2e479e76746dc8eb8084df72c8cc9",
            "c33d68c000dd4497ad28d7f40c69b072",
            "f6d9757603664bafa0cc6120d2f649ba",
            "e5ed57e0c993466e86cc60724bcae077",
            "b63909b1caf44642a53f2141dc8a16f4",
            "24f4e5a4e3a44fa5b430bc93fc4c6b87",
            "813a875db57c4af899c352ef07099a41",
            "f4bdcbd34ff54db9ae7829801b4b7757",
            "95af726a251244d7bddac15e12198150",
            "a463b4ddc73c4eee908b4d22de34131e",
            "f65cd0a121514b7fae16225fdb171d90",
            "e8f5fe99f4cd49d49f0ca1c5ee76aa38",
            "dac6b73bd55845d683bf21eb49c5df37",
            "98439b2ffc2842188696838a66053b6f",
            "3f4cfab3834f4d18a921732098a82048",
            "a751fbe15eb949a5bd2b8f676f1cc8df",
            "b1b56592a439415ea0e63d4f55c2d58f",
            "ee24170941a14ecc80e5af8cd2370a93"
          ]
        },
        "id": "hhLc03vR7YBr",
        "outputId": "1d8a4e5f-4a88-4cd1-c67f-226b8cc1f0f3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/500 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "0de79c6d3066479d993f394472b0ceb5"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/500 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f4bdcbd34ff54db9ae7829801b4b7757"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(bleu_calc.corpus_score(df_dev['rus_translated3'].tolist(), [df_dev['ru'].tolist()]))\n",
        "print(chrf_calc.corpus_score(df_dev['rus_translated3'].tolist(), [df_dev['ru'].tolist()]))\n",
        "print(bleu_calc.corpus_score(df_dev['tyv_translated3'].tolist(), [df_dev['tyv'].tolist()]))\n",
        "print(chrf_calc.corpus_score(df_dev['tyv_translated3'].tolist(), [df_dev['tyv'].tolist()]))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ISur66Pl9KCX",
        "outputId": "e0cd245f-5508-4a07-9bbd-2639bd0c19dc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "BLEU = 23.06 51.1/29.1/18.1/11.5 (BP = 0.978 ratio = 0.978 hyp_len = 2273 ref_len = 2324)\n",
            "chrF2++ = 48.56\n",
            "BLEU = 26.12 53.4/32.5/21.0/13.6 (BP = 0.985 ratio = 0.985 hyp_len = 2336 ref_len = 2371)\n",
            "chrF2++ = 52.60\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev['rus2eng'] = [translate(t, 'tyv_Cyrl', 'eng_Latn')[0] for t in tqdm(df_dev.tyv)]\n",
        "df_dev['tyv2eng'] = [translate(t, 'rus_Cyrl', 'eng_Latn')[0] for t in tqdm(df_dev.ru)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81,
          "referenced_widgets": [
            "3417c70a8db54723804a9de92094eec6",
            "39fbccba9def4fce9e41edf0f931e667",
            "1555767cc3334aa185368367f27b49a6",
            "637865b7608e4aae82b8d87235d8172e",
            "7251bc0a86ce4739b512c814941eab7d",
            "9c2a4c33dc1d4eb2adf449272e1c1d7e",
            "db15b4bda74441d58fde31d2287c8cbe",
            "33daa47f93874045a29f7d77e5a9d7fc",
            "ed443ff78a184cc49632b65197010277",
            "53aefd2209db419181b816792cacfdd8",
            "025040b9a9e54109bc20ebd217bfd74c",
            "24f0dfcbe8b144828b5efb5d387c15d5",
            "39a4ac85caa7443f8348c2e2c68b64f2",
            "b2c8f8f89565450c85bf02e80cfadc39",
            "6f44ede97c0b4ca79dcbea7bb31402a9",
            "5558af48295945cb88aaefedc1f25ed4",
            "5e371b1b259248b1bb34ba66f9febdf5",
            "7009b83f25c147c8b173886f51ba21d1",
            "b1de5494ad734e44934ede08df117937",
            "002a75bfa1c943aca2e5c2ec06ad4900",
            "65d56cb8b3584d64b94f6194f76d0f4e",
            "f64c0ee1822f45c1a24f18765324cdd5"
          ]
        },
        "id": "GPU1GA1HDJ9h",
        "outputId": "c00c2fbd-c456-4660-d7fd-ee83e729e601"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/500 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3417c70a8db54723804a9de92094eec6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/500 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "24f0dfcbe8b144828b5efb5d387c15d5"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Results with num_beams=1:\n",
        "```\n",
        "V1\n",
        "BLEU = 23.21 51.2/29.1/18.0/11.8 (BP = 0.978 ratio = 0.978 hyp_len = 2273 ref_len = 2324)\n",
        "chrF2++ = 47.88\n",
        "BLEU = 22.03 51.5/29.7/17.9/10.4 (BP = 0.952 ratio = 0.953 hyp_len = 2260 ref_len = 2371)\n",
        "chrF2++ = 49.37\n",
        "V2\n",
        "BLEU = 24.08 50.9/29.5/19.1/12.3 (BP = 0.988 ratio = 0.988 hyp_len = 2297 ref_len = 2324)\n",
        "chrF2++ = 48.96\n",
        "BLEU = 22.50 50.5/28.5/17.7/11.1 (BP = 0.974 ratio = 0.974 hyp_len = 2310 ref_len = 2371)\n",
        "chrF2++ = 48.85\n",
        "V3\n",
        "BLEU = 22.25 49.8/27.8/17.2/11.0 (BP = 0.983 ratio = 0.983 hyp_len = 2284 ref_len = 2324)\n",
        "chrF2++ = 47.89\n",
        "BLEU = 25.28 52.2/31.2/20.0/13.1 (BP = 0.989 ratio = 0.989 hyp_len = 2346 ref_len = 2371)\n",
        "chrF2++ = 51.87\n",
        "````\n",
        "\n",
        "Results with 4 beams:\n",
        "```\n",
        "V1\n",
        "BLEU = 24.14 52.5/30.4/18.9/12.1 (BP = 0.981 ratio = 0.981 hyp_len = 2281 ref_len = 2324)\n",
        "chrF2++ = 49.49\n",
        "BLEU = 23.41 52.1/31.0/18.9/11.3 (BP = 0.966 ratio = 0.967 hyp_len = 2292 ref_len = 2371)\n",
        "chrF2++ = 50.89\n",
        "V2\n",
        "BLEU = 25.18 52.4/31.3/20.4/13.3 (BP = 0.976 ratio = 0.976 hyp_len = 2269 ref_len = 2324)\n",
        "chrF2++ = 49.85\n",
        "BLEU = 23.22 51.6/29.4/18.3/11.6 (BP = 0.975 ratio = 0.975 hyp_len = 2312 ref_len = 2371)\n",
        "chrF2++ = 49.87\n",
        "V3\n",
        "BLEU = 23.06 51.1/29.1/18.1/11.5 (BP = 0.978 ratio = 0.978 hyp_len = 2273 ref_len = 2324)\n",
        "chrF2++ = 48.56\n",
        "BLEU = 26.12 53.4/32.5/21.0/13.6 (BP = 0.985 ratio = 0.985 hyp_len = 2336 ref_len = 2371)\n",
        "chrF2++ = 52.60\n",
        "```\n",
        "\n",
        "Which means:\n",
        "* For all directions and models, beam search improves the results\n",
        "* Longer training builds up quality for Tyvan, but decreases it for Russian."
      ],
      "metadata": {
        "id": "Ux7sF9Cv7_o1"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "```\n",
        "                                  | tyv->rus | rus->tyv\n",
        "Model v1 (no vocabulary update):  |\n",
        "    no beam search                |   23.21  |  22.03\n",
        "    num_beams = 4                 |   24.14  |  23.41\n",
        "Model v2 (with vocabulary update):|\n",
        "    no beam search                |   24.08  |  22.50\n",
        "    num_beams = 4                 |   25.18  |  23.22\n",
        "```"
      ],
      "metadata": {
        "id": "ywUwR9KfP0yt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev.to_csv(model_load_name + \"/dev_set_translated.tsv\", sep=\"\\t\")"
      ],
      "metadata": {
        "id": "2JNisawc-hU4"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here are some examples of how translation has changed:"
      ],
      "metadata": {
        "id": "7VVicYZO9P3U"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev.sample(5, random_state=1)[['tyv', 'ru', 'rus_translated']]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 289
        },
        "id": "vf6rJm2PHcPZ",
        "outputId": "4accff57-b069-4e54-9f98-3f51473c308a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                                                          tyv  \\\n",
              "row_id                                                                                          \n",
              "5442    транспорт херекселдерин ажыглаарының база шимчээшкинниң айыыл чок чоруунуң дүрүмнери    \n",
              "57777                                                                           аъш-чем садыы   \n",
              "104130                                                             Бүгү чүве төнген, бойлаан.   \n",
              "49344                                                                          фокуска кирбес   \n",
              "28319                                                                         рекорд тургузар   \n",
              "\n",
              "                                                                        ru  \\\n",
              "row_id                                                                       \n",
              "5442     правила безопасности движения и эксплуатации транспортных средств   \n",
              "57777                                            продовольственный магазин   \n",
              "104130                                         Все было кончено, потеряно.   \n",
              "49344                                                   не попасть в фокус   \n",
              "28319                                                    установить рекорд   \n",
              "\n",
              "                                                           rus_translated  \n",
              "row_id                                                                     \n",
              "5442    правила безопасности движения и эксплуатации транспортных средств  \n",
              "57777                                           продовольственный магазин  \n",
              "104130                                  Все было кончено, самостоятельно.  \n",
              "49344                                                  не попасть в фокус  \n",
              "28319                                                    поставить рекорд  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8190efd5-dfe7-4c0f-8c34-2d57c1671dc3\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tyv</th>\n",
              "      <th>ru</th>\n",
              "      <th>rus_translated</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>row_id</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>5442</th>\n",
              "      <td>транспорт херекселдерин ажыглаарының база шимчээшкинниң айыыл чок чоруунуң дүрүмнери</td>\n",
              "      <td>правила безопасности движения и эксплуатации транспортных средств</td>\n",
              "      <td>правила безопасности движения и эксплуатации транспортных средств</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>57777</th>\n",
              "      <td>аъш-чем садыы</td>\n",
              "      <td>продовольственный магазин</td>\n",
              "      <td>продовольственный магазин</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>104130</th>\n",
              "      <td>Бүгү чүве төнген, бойлаан.</td>\n",
              "      <td>Все было кончено, потеряно.</td>\n",
              "      <td>Все было кончено, самостоятельно.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49344</th>\n",
              "      <td>фокуска кирбес</td>\n",
              "      <td>не попасть в фокус</td>\n",
              "      <td>не попасть в фокус</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28319</th>\n",
              "      <td>рекорд тургузар</td>\n",
              "      <td>установить рекорд</td>\n",
              "      <td>поставить рекорд</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8190efd5-dfe7-4c0f-8c34-2d57c1671dc3')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-8190efd5-dfe7-4c0f-8c34-2d57c1671dc3 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-8190efd5-dfe7-4c0f-8c34-2d57c1671dc3');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-a623c265-593c-4f74-be89-829d85c636ac\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a623c265-593c-4f74-be89-829d85c636ac')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-a623c265-593c-4f74-be89-829d85c636ac button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_dev.sample(20, random_state=1)[[\n",
        "    'tyv', 'tyv_translated', 'tyv_translated2', 'tyv_translated3', 'tyv2eng',\n",
        "    'ru', 'rus_translated', 'rus_translated2', 'rus_translated3', 'rus2eng',\n",
        "]]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "rrsuxzq9-kWl",
        "outputId": "d975f6b8-0dba-4deb-db89-31c44d932046"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                                                          tyv  \\\n",
              "row_id                                                                                          \n",
              "5442    транспорт херекселдерин ажыглаарының база шимчээшкинниң айыыл чок чоруунуң дүрүмнери    \n",
              "57777                                                                           аъш-чем садыы   \n",
              "104130                                                             Бүгү чүве төнген, бойлаан.   \n",
              "49344                                                                          фокуска кирбес   \n",
              "28319                                                                         рекорд тургузар   \n",
              "43534                                                                         чурукту делгээр   \n",
              "37159                                                                            ылап хөделир   \n",
              "36993                                                                  колдуктап алгаш чоруур   \n",
              "116009                                                                 Копривничко-Крижевачка   \n",
              "113178                                                                                 Лампаң   \n",
              "106939                                             Ону кым-даа томаартып чадап каан чүве-дир.   \n",
              "86499                                                                               Самбайлык   \n",
              "20503                                                       чалбыыш ыяштарны өрттендир чипкен   \n",
              "1784                                                                              бүзүредилге   \n",
              "34052                                                                          шын эвес адаар   \n",
              "83751                                                                              Калдар аът   \n",
              "68723                                                                           соора дыңнаар   \n",
              "84267                                                                           Коңга эдипкен   \n",
              "14467                                                   мен силерге кашты төлээр ужурлуг мен?   \n",
              "54719                                                                  кончуг берге байдалдар   \n",
              "\n",
              "                                                                         tyv_translated  \\\n",
              "row_id                                                                                    \n",
              "5442    шимчээшкинниң болгаш транспорт аймаан ажыглаарының айыыл чок чоруунуң дүрүмнери   \n",
              "57777                                                                     аъш-чем садыы   \n",
              "104130                                                    Шупту чүве доозулган, читкен.   \n",
              "49344                                                        илби-шидиге алзыр арга чок   \n",
              "28319                                                                   рекорд тургузар   \n",
              "43534                                                                   чурукту делгээр   \n",
              "37159                                                                      ылап хөделир   \n",
              "36993                                            шыңганның адаанга көдүрүп алгаш чоруур   \n",
              "116009                                                           Копривничко-Крижевачка   \n",
              "113178                                                                          Лампанг   \n",
              "106939                                                 Кым-даа ону оожургадып шыдаваан.   \n",
              "86499                                                                         Самбайлык   \n",
              "20503                                                        чалбыыш ыяштарны чиртипкен   \n",
              "1784                                                                        бүзүредилге   \n",
              "34052                                                                    шын эвес адаар   \n",
              "83751                                                                      Мухортай аът   \n",
              "68723                                                                        дыңнаалаар   \n",
              "84267                                                            Коңгулуур кыңгырт дээн   \n",
              "14467                                                          силерге чеже өрелиг мен?   \n",
              "54719                                                               кончуг берге байдал   \n",
              "\n",
              "                                                                         tyv_translated2  \\\n",
              "row_id                                                                                     \n",
              "5442    транспорт аймаан шимчээшкининиң болгаш ажыглаарының айыыл чок чоруунуң дүрүмнери   \n",
              "57777                                                                      аъш-чем садыы   \n",
              "104130                                                        Шупту чүве төнген, читкен.   \n",
              "49344                                                                     илбиге алыспас   \n",
              "28319                                                                    рекорд тургузар   \n",
              "43534                                                                      чурук делгээр   \n",
              "37159                                                                 бүзүрелдиг хөделир   \n",
              "36993                                                              колдук адаанга аппаар   \n",
              "116009                                                            Копривничко-Крижевачка   \n",
              "113178                                                                            Лампаң   \n",
              "106939                                                  Ону кым-даа оожургадып шыдаваан.   \n",
              "86499                                                                          Самбайлык   \n",
              "20503                                                            чалбыыш ыяштарны кыпкан   \n",
              "1784                                                                           бүзүрээри   \n",
              "34052                                                                     шын эвес адаар   \n",
              "83751                                                                       Дыйлагар аът   \n",
              "68723                                                                         дыңнаалаар   \n",
              "84267                                                                  Куңгур кызаш дээн   \n",
              "14467                                                      силерге чеже хире өренир мен?   \n",
              "54719                                                                     аажок кортканы   \n",
              "\n",
              "                                                                                          tyv_translated3  \\\n",
              "row_id                                                                                                      \n",
              "5442    транспорт аймаан шимчээшкининиң болгаш транспорт аймаан ажыглаарының айыыл чок чоруунуң дүрүмнери   \n",
              "57777                                                                                       аъш-чем садыы   \n",
              "104130                                                                         Бүгү чүве бойлаан, читкен.   \n",
              "49344                                                                               илби-шидиге туттурбас   \n",
              "28319                                                                                     рекорд тургузар   \n",
              "43534                                                                                     чурукту делгээр   \n",
              "37159                                                                                        ылап хөделир   \n",
              "36993                                                                              колдуктап алгаш чоруур   \n",
              "116009                                                                             Копривничко-Крижевачка   \n",
              "113178                                                                                            Лампанг   \n",
              "106939                                                                 Оглумну кым-даа оожургап шыдаваан.   \n",
              "86499                                                                                           Самбайлык   \n",
              "20503                                                                       чалбырааш ыяштарны хөме апкан   \n",
              "1784                                                                                              бүзүрел   \n",
              "34052                                                                                         соора адаар   \n",
              "83751                                                                                        Мухортая аът   \n",
              "68723                                                                                         дыңнап алыр   \n",
              "84267                                                                             Коңгурааш, кыңгырт дээн   \n",
              "14467                                                                     силерге чеже хире өртектиг мен?   \n",
              "54719                                                                                       аажок коргары   \n",
              "\n",
              "                                                                    tyv2eng  \\\n",
              "row_id                                                                        \n",
              "5442    ң болгаш транспорт аймаан ажыглаарының айыыл чок чоруунуң дүрүмнери   \n",
              "57777                                                                 садыы   \n",
              "104130                                                    -даа, читкен-даа.   \n",
              "49344                                                        гге күш четпес   \n",
              "28319                                                            г тургузар   \n",
              "43534                                                     ң чурукту делгээр   \n",
              "37159                                                             г хөделир   \n",
              "36993                                                             г шыгжаар   \n",
              "116009                                                       чко-Крижевачка   \n",
              "113178                                                                    ң   \n",
              "106939                                           кым-даа оожургап шыдаваан.   \n",
              "86499                                                                     ң   \n",
              "20503                                        г чалбырааш ыяштарны чиртипкен   \n",
              "1784                                                                          \n",
              "34052                                                               ң адаар   \n",
              "83751                                                                   аът   \n",
              "68723                                                               дыңнаар   \n",
              "84267                                              ң кыңгырткайны-ла берген   \n",
              "14467                                          каш хире өрелиг мен силерге?   \n",
              "54719                                                                         \n",
              "\n",
              "                                                                        ru  \\\n",
              "row_id                                                                       \n",
              "5442     правила безопасности движения и эксплуатации транспортных средств   \n",
              "57777                                            продовольственный магазин   \n",
              "104130                                         Все было кончено, потеряно.   \n",
              "49344                                                   не попасть в фокус   \n",
              "28319                                                    установить рекорд   \n",
              "43534                                                   выставлять картину   \n",
              "37159                                                действовать наверняка   \n",
              "36993                                                     нести под мышкой   \n",
              "116009                                              Копривничко-Крижевачка   \n",
              "113178                                                             Лампанг   \n",
              "106939                                И никто не в силах был укротить его.   \n",
              "86499                                                            Самбайлык   \n",
              "20503                                                пламя опалило деревья   \n",
              "1784                                                          доверенность   \n",
              "34052                                              неправильно произносить   \n",
              "83751                                                      Мухортая лошадь   \n",
              "68723                                                           прослушать   \n",
              "84267                                                     Звонок прозвенел   \n",
              "14467                                                сколько я вам должен?   \n",
              "54719                                                                 ужас   \n",
              "\n",
              "                                                           rus_translated  \\\n",
              "row_id                                                                      \n",
              "5442    правила безопасности движения и эксплуатации транспортных средств   \n",
              "57777                                           продовольственный магазин   \n",
              "104130                                  Все было кончено, самостоятельно.   \n",
              "49344                                                  не попасть в фокус   \n",
              "28319                                                    поставить рекорд   \n",
              "43534                                                  выставлять картину   \n",
              "37159                                               действовать аккуратно   \n",
              "36993                                                   нести под мышками   \n",
              "116009                                             Копривничко-Крижевачка   \n",
              "113178                                                            Лампанг   \n",
              "106939                                 И никто не мог оглянуться на него.   \n",
              "86499                                                           Самбайлык   \n",
              "20503                                                 пламясъегло деревья   \n",
              "1784                                                              доверие   \n",
              "34052                                              неправильно произнести   \n",
              "83751                                                     Верховая лошадь   \n",
              "68723                                                          ослышаться   \n",
              "84267                                                 Прикованный к конга   \n",
              "14467                                     сколько я вам должен заплатить?   \n",
              "54719                                                               ужасы   \n",
              "\n",
              "                                                          rus_translated2  \\\n",
              "row_id                                                                      \n",
              "5442    правила эксплуатации транспортных средств и безопасности движения   \n",
              "57777                                           продовольственный магазин   \n",
              "104130                                          Все кончилось, разошёлся.   \n",
              "49344                                               не попасться в фокусы   \n",
              "28319                                                   установить рекорд   \n",
              "43534                                                  развернуть картину   \n",
              "37159                                               действовать наверняка   \n",
              "36993                                                   носить под мышкой   \n",
              "116009                                             Копривничко-Крижевачка   \n",
              "113178                                                            Лампанг   \n",
              "106939                                Никто не мог на это остановить его.   \n",
              "86499                                                           Самбайлык   \n",
              "20503                                             пламя разъедало деревья   \n",
              "1784                                                            убеждение   \n",
              "34052                                              неправильно произнести   \n",
              "83751                                                      Верхняя лошадь   \n",
              "68723                                                          прослушать   \n",
              "84267                                                  Конга поправленный   \n",
              "14467                                               сколько я вам обязан?   \n",
              "54719                                                               ужасы   \n",
              "\n",
              "                                                          rus_translated3  \\\n",
              "row_id                                                                      \n",
              "5442    правила безопасности эксплуатации транспортных средств и движения   \n",
              "57777                                           продовольственный магазин   \n",
              "104130                                              Все кончено, кончено.   \n",
              "49344                                                 не попасть на фокус   \n",
              "28319                                                   установить рекорд   \n",
              "43534                                               экспонировать картину   \n",
              "37159                                               действовать наверняка   \n",
              "36993                                                    нести под мышкой   \n",
              "116009                                             Копривничко-Крижевачка   \n",
              "113178                                                            Лампанг   \n",
              "106939                                         Никто не мог его обуздать.   \n",
              "86499                                                           Самбайлык   \n",
              "20503                                                пламя выжгло деревья   \n",
              "1784                                                            заверения   \n",
              "34052                                             неправильно произносить   \n",
              "83751                                                     Рысистая лошадь   \n",
              "68723                                                          ослышаться   \n",
              "84267                                                     звонок исправил   \n",
              "14467                                       сколько я вам должен платить?   \n",
              "54719                                                      чёрные условия   \n",
              "\n",
              "                                                                                                rus2eng  \n",
              "row_id                                                                                                   \n",
              "5442    дүрүмнер транспорт херекселдерин ажыглаарының болгаш шимчээшкинниң айыыл чок чоруунуң дугайында  \n",
              "57777                                                                                             садыы  \n",
              "104130                                                                        -ла, бүгү чүве кончилось.  \n",
              "49344                                                                                                 г  \n",
              "28319                                                                                        г тургузар  \n",
              "43534                                                                         чурукту делгередип чуруур  \n",
              "37159                                                                                         г хөделир  \n",
              "36993                                                                                      алгаш чоруур  \n",
              "116009                                                                                вничко-Крижевачка  \n",
              "113178                                                                                                   \n",
              "106939                                               -ла, чүге дээрге кым-даа ону таарыштырып шыдаваан.  \n",
              "86499                                                                                                 г  \n",
              "20503                                                                         г ыяштарны чиртип каапкан  \n",
              "1784                                                                                                     \n",
              "34052                                                                                                    \n",
              "83751                                                                                               аът  \n",
              "68723                                                                                           дыңнаар  \n",
              "84267                                                                                            берген  \n",
              "14467                                                       ң дээш каш хире төлээр ужурлуг мен силерге?  \n",
              "54719                                                                                ть берге байдалдар  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-4a12ce7d-beba-485f-8e67-2c64f0a553d6\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tyv</th>\n",
              "      <th>tyv_translated</th>\n",
              "      <th>tyv_translated2</th>\n",
              "      <th>tyv_translated3</th>\n",
              "      <th>tyv2eng</th>\n",
              "      <th>ru</th>\n",
              "      <th>rus_translated</th>\n",
              "      <th>rus_translated2</th>\n",
              "      <th>rus_translated3</th>\n",
              "      <th>rus2eng</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>row_id</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>5442</th>\n",
              "      <td>транспорт херекселдерин ажыглаарының база шимчээшкинниң айыыл чок чоруунуң дүрүмнери</td>\n",
              "      <td>шимчээшкинниң болгаш транспорт аймаан ажыглаарының айыыл чок чоруунуң дүрүмнери</td>\n",
              "      <td>транспорт аймаан шимчээшкининиң болгаш ажыглаарының айыыл чок чоруунуң дүрүмнери</td>\n",
              "      <td>транспорт аймаан шимчээшкининиң болгаш транспорт аймаан ажыглаарының айыыл чок чоруунуң дүрүмнери</td>\n",
              "      <td>ң болгаш транспорт аймаан ажыглаарының айыыл чок чоруунуң дүрүмнери</td>\n",
              "      <td>правила безопасности движения и эксплуатации транспортных средств</td>\n",
              "      <td>правила безопасности движения и эксплуатации транспортных средств</td>\n",
              "      <td>правила эксплуатации транспортных средств и безопасности движения</td>\n",
              "      <td>правила безопасности эксплуатации транспортных средств и движения</td>\n",
              "      <td>дүрүмнер транспорт херекселдерин ажыглаарының болгаш шимчээшкинниң айыыл чок чоруунуң дугайында</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>57777</th>\n",
              "      <td>аъш-чем садыы</td>\n",
              "      <td>аъш-чем садыы</td>\n",
              "      <td>аъш-чем садыы</td>\n",
              "      <td>аъш-чем садыы</td>\n",
              "      <td>садыы</td>\n",
              "      <td>продовольственный магазин</td>\n",
              "      <td>продовольственный магазин</td>\n",
              "      <td>продовольственный магазин</td>\n",
              "      <td>продовольственный магазин</td>\n",
              "      <td>садыы</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>104130</th>\n",
              "      <td>Бүгү чүве төнген, бойлаан.</td>\n",
              "      <td>Шупту чүве доозулган, читкен.</td>\n",
              "      <td>Шупту чүве төнген, читкен.</td>\n",
              "      <td>Бүгү чүве бойлаан, читкен.</td>\n",
              "      <td>-даа, читкен-даа.</td>\n",
              "      <td>Все было кончено, потеряно.</td>\n",
              "      <td>Все было кончено, самостоятельно.</td>\n",
              "      <td>Все кончилось, разошёлся.</td>\n",
              "      <td>Все кончено, кончено.</td>\n",
              "      <td>-ла, бүгү чүве кончилось.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49344</th>\n",
              "      <td>фокуска кирбес</td>\n",
              "      <td>илби-шидиге алзыр арга чок</td>\n",
              "      <td>илбиге алыспас</td>\n",
              "      <td>илби-шидиге туттурбас</td>\n",
              "      <td>гге күш четпес</td>\n",
              "      <td>не попасть в фокус</td>\n",
              "      <td>не попасть в фокус</td>\n",
              "      <td>не попасться в фокусы</td>\n",
              "      <td>не попасть на фокус</td>\n",
              "      <td>г</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28319</th>\n",
              "      <td>рекорд тургузар</td>\n",
              "      <td>рекорд тургузар</td>\n",
              "      <td>рекорд тургузар</td>\n",
              "      <td>рекорд тургузар</td>\n",
              "      <td>г тургузар</td>\n",
              "      <td>установить рекорд</td>\n",
              "      <td>поставить рекорд</td>\n",
              "      <td>установить рекорд</td>\n",
              "      <td>установить рекорд</td>\n",
              "      <td>г тургузар</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>43534</th>\n",
              "      <td>чурукту делгээр</td>\n",
              "      <td>чурукту делгээр</td>\n",
              "      <td>чурук делгээр</td>\n",
              "      <td>чурукту делгээр</td>\n",
              "      <td>ң чурукту делгээр</td>\n",
              "      <td>выставлять картину</td>\n",
              "      <td>выставлять картину</td>\n",
              "      <td>развернуть картину</td>\n",
              "      <td>экспонировать картину</td>\n",
              "      <td>чурукту делгередип чуруур</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>37159</th>\n",
              "      <td>ылап хөделир</td>\n",
              "      <td>ылап хөделир</td>\n",
              "      <td>бүзүрелдиг хөделир</td>\n",
              "      <td>ылап хөделир</td>\n",
              "      <td>г хөделир</td>\n",
              "      <td>действовать наверняка</td>\n",
              "      <td>действовать аккуратно</td>\n",
              "      <td>действовать наверняка</td>\n",
              "      <td>действовать наверняка</td>\n",
              "      <td>г хөделир</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>36993</th>\n",
              "      <td>колдуктап алгаш чоруур</td>\n",
              "      <td>шыңганның адаанга көдүрүп алгаш чоруур</td>\n",
              "      <td>колдук адаанга аппаар</td>\n",
              "      <td>колдуктап алгаш чоруур</td>\n",
              "      <td>г шыгжаар</td>\n",
              "      <td>нести под мышкой</td>\n",
              "      <td>нести под мышками</td>\n",
              "      <td>носить под мышкой</td>\n",
              "      <td>нести под мышкой</td>\n",
              "      <td>алгаш чоруур</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>116009</th>\n",
              "      <td>Копривничко-Крижевачка</td>\n",
              "      <td>Копривничко-Крижевачка</td>\n",
              "      <td>Копривничко-Крижевачка</td>\n",
              "      <td>Копривничко-Крижевачка</td>\n",
              "      <td>чко-Крижевачка</td>\n",
              "      <td>Копривничко-Крижевачка</td>\n",
              "      <td>Копривничко-Крижевачка</td>\n",
              "      <td>Копривничко-Крижевачка</td>\n",
              "      <td>Копривничко-Крижевачка</td>\n",
              "      <td>вничко-Крижевачка</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>113178</th>\n",
              "      <td>Лампаң</td>\n",
              "      <td>Лампанг</td>\n",
              "      <td>Лампаң</td>\n",
              "      <td>Лампанг</td>\n",
              "      <td>ң</td>\n",
              "      <td>Лампанг</td>\n",
              "      <td>Лампанг</td>\n",
              "      <td>Лампанг</td>\n",
              "      <td>Лампанг</td>\n",
              "      <td></td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>106939</th>\n",
              "      <td>Ону кым-даа томаартып чадап каан чүве-дир.</td>\n",
              "      <td>Кым-даа ону оожургадып шыдаваан.</td>\n",
              "      <td>Ону кым-даа оожургадып шыдаваан.</td>\n",
              "      <td>Оглумну кым-даа оожургап шыдаваан.</td>\n",
              "      <td>кым-даа оожургап шыдаваан.</td>\n",
              "      <td>И никто не в силах был укротить его.</td>\n",
              "      <td>И никто не мог оглянуться на него.</td>\n",
              "      <td>Никто не мог на это остановить его.</td>\n",
              "      <td>Никто не мог его обуздать.</td>\n",
              "      <td>-ла, чүге дээрге кым-даа ону таарыштырып шыдаваан.</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>86499</th>\n",
              "      <td>Самбайлык</td>\n",
              "      <td>Самбайлык</td>\n",
              "      <td>Самбайлык</td>\n",
              "      <td>Самбайлык</td>\n",
              "      <td>ң</td>\n",
              "      <td>Самбайлык</td>\n",
              "      <td>Самбайлык</td>\n",
              "      <td>Самбайлык</td>\n",
              "      <td>Самбайлык</td>\n",
              "      <td>г</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20503</th>\n",
              "      <td>чалбыыш ыяштарны өрттендир чипкен</td>\n",
              "      <td>чалбыыш ыяштарны чиртипкен</td>\n",
              "      <td>чалбыыш ыяштарны кыпкан</td>\n",
              "      <td>чалбырааш ыяштарны хөме апкан</td>\n",
              "      <td>г чалбырааш ыяштарны чиртипкен</td>\n",
              "      <td>пламя опалило деревья</td>\n",
              "      <td>пламясъегло деревья</td>\n",
              "      <td>пламя разъедало деревья</td>\n",
              "      <td>пламя выжгло деревья</td>\n",
              "      <td>г ыяштарны чиртип каапкан</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1784</th>\n",
              "      <td>бүзүредилге</td>\n",
              "      <td>бүзүредилге</td>\n",
              "      <td>бүзүрээри</td>\n",
              "      <td>бүзүрел</td>\n",
              "      <td></td>\n",
              "      <td>доверенность</td>\n",
              "      <td>доверие</td>\n",
              "      <td>убеждение</td>\n",
              "      <td>заверения</td>\n",
              "      <td></td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>34052</th>\n",
              "      <td>шын эвес адаар</td>\n",
              "      <td>шын эвес адаар</td>\n",
              "      <td>шын эвес адаар</td>\n",
              "      <td>соора адаар</td>\n",
              "      <td>ң адаар</td>\n",
              "      <td>неправильно произносить</td>\n",
              "      <td>неправильно произнести</td>\n",
              "      <td>неправильно произнести</td>\n",
              "      <td>неправильно произносить</td>\n",
              "      <td></td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>83751</th>\n",
              "      <td>Калдар аът</td>\n",
              "      <td>Мухортай аът</td>\n",
              "      <td>Дыйлагар аът</td>\n",
              "      <td>Мухортая аът</td>\n",
              "      <td>аът</td>\n",
              "      <td>Мухортая лошадь</td>\n",
              "      <td>Верховая лошадь</td>\n",
              "      <td>Верхняя лошадь</td>\n",
              "      <td>Рысистая лошадь</td>\n",
              "      <td>аът</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>68723</th>\n",
              "      <td>соора дыңнаар</td>\n",
              "      <td>дыңнаалаар</td>\n",
              "      <td>дыңнаалаар</td>\n",
              "      <td>дыңнап алыр</td>\n",
              "      <td>дыңнаар</td>\n",
              "      <td>прослушать</td>\n",
              "      <td>ослышаться</td>\n",
              "      <td>прослушать</td>\n",
              "      <td>ослышаться</td>\n",
              "      <td>дыңнаар</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>84267</th>\n",
              "      <td>Коңга эдипкен</td>\n",
              "      <td>Коңгулуур кыңгырт дээн</td>\n",
              "      <td>Куңгур кызаш дээн</td>\n",
              "      <td>Коңгурааш, кыңгырт дээн</td>\n",
              "      <td>ң кыңгырткайны-ла берген</td>\n",
              "      <td>Звонок прозвенел</td>\n",
              "      <td>Прикованный к конга</td>\n",
              "      <td>Конга поправленный</td>\n",
              "      <td>звонок исправил</td>\n",
              "      <td>берген</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14467</th>\n",
              "      <td>мен силерге кашты төлээр ужурлуг мен?</td>\n",
              "      <td>силерге чеже өрелиг мен?</td>\n",
              "      <td>силерге чеже хире өренир мен?</td>\n",
              "      <td>силерге чеже хире өртектиг мен?</td>\n",
              "      <td>каш хире өрелиг мен силерге?</td>\n",
              "      <td>сколько я вам должен?</td>\n",
              "      <td>сколько я вам должен заплатить?</td>\n",
              "      <td>сколько я вам обязан?</td>\n",
              "      <td>сколько я вам должен платить?</td>\n",
              "      <td>ң дээш каш хире төлээр ужурлуг мен силерге?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>54719</th>\n",
              "      <td>кончуг берге байдалдар</td>\n",
              "      <td>кончуг берге байдал</td>\n",
              "      <td>аажок кортканы</td>\n",
              "      <td>аажок коргары</td>\n",
              "      <td></td>\n",
              "      <td>ужас</td>\n",
              "      <td>ужасы</td>\n",
              "      <td>ужасы</td>\n",
              "      <td>чёрные условия</td>\n",
              "      <td>ть берге байдалдар</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4a12ce7d-beba-485f-8e67-2c64f0a553d6')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-4a12ce7d-beba-485f-8e67-2c64f0a553d6 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-4a12ce7d-beba-485f-8e67-2c64f0a553d6');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-4e5f649f-f630-497d-ac7b-d213d1ebb02b\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4e5f649f-f630-497d-ac7b-d213d1ebb02b')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-4e5f649f-f630-497d-ac7b-d213d1ebb02b button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "cols = ['ind', 'tyv', 'ru']\n",
        "splits = {'train': df_train[df_train.index<=49_454], 'test': df_test, 'dev': df_dev}\n",
        "df_joint = []\n",
        "for k, v in splits.items():\n",
        "    v = v[cols].copy()\n",
        "    v.index.name = \"row_id\"\n",
        "    v['split'] = k\n",
        "    df_joint.append(v)\n",
        "df_joint = pd.concat(df_joint)\n",
        "df_joint.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gTHS_RygGLgV",
        "outputId": "640f2f2f-20d9-4a75-d3b8-ded11ab0b7a9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(50000, 4)"
            ]
          },
          "metadata": {},
          "execution_count": 70
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_joint.sample(5)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 237
        },
        "id": "Yz6ShvR3HGk3",
        "outputId": "216b9e1d-90b7-40a7-a976-207fda948148"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          ind  \\\n",
              "row_id          \n",
              "314       328   \n",
              "4376     4390   \n",
              "13377   13392   \n",
              "91144   97279   \n",
              "307       321   \n",
              "\n",
              "                                                                                   tyv  \\\n",
              "row_id                                                                                   \n",
              "314                      Өг-бүле бүрүзү эвээш дээрге-ле 500-600 ивини азырап өстүрзүн.   \n",
              "4376                                                              Өрээл аяннаны берген   \n",
              "13377                                           кым-бир кижи-биле силер деп чугаалажыр   \n",
              "91144   Идээледир чемнениринде база эки чүү-даа чүве чок, ынчангаш арай эвээшти чиңер.   \n",
              "307          Оларның аразында 14 суурда 500 четпес, а 8 суурда 250 хире чурттакчы бар.   \n",
              "\n",
              "                                                                   ru  split  \n",
              "row_id                                                                        \n",
              "314             Пусть на каждую семью было хотя бы по 500-600 оленей.  train  \n",
              "4376                                      Комната приняла хороший вид  train  \n",
              "13377                                           быть на вы с кем-либо  train  \n",
              "91144     Ничего хорошего нет и в переедании, так что ешьте поменьше.    dev  \n",
              "307     Среди них в 14 селах менее 500, в восьми - менее 250 человек.  train  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e2ff1a85-e59b-42eb-a460-c613f4696ac6\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>ind</th>\n",
              "      <th>tyv</th>\n",
              "      <th>ru</th>\n",
              "      <th>split</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>row_id</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>314</th>\n",
              "      <td>328</td>\n",
              "      <td>Өг-бүле бүрүзү эвээш дээрге-ле 500-600 ивини азырап өстүрзүн.</td>\n",
              "      <td>Пусть на каждую семью было хотя бы по 500-600 оленей.</td>\n",
              "      <td>train</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4376</th>\n",
              "      <td>4390</td>\n",
              "      <td>Өрээл аяннаны берген</td>\n",
              "      <td>Комната приняла хороший вид</td>\n",
              "      <td>train</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13377</th>\n",
              "      <td>13392</td>\n",
              "      <td>кым-бир кижи-биле силер деп чугаалажыр</td>\n",
              "      <td>быть на вы с кем-либо</td>\n",
              "      <td>train</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>91144</th>\n",
              "      <td>97279</td>\n",
              "      <td>Идээледир чемнениринде база эки чүү-даа чүве чок, ынчангаш арай эвээшти чиңер.</td>\n",
              "      <td>Ничего хорошего нет и в переедании, так что ешьте поменьше.</td>\n",
              "      <td>dev</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>307</th>\n",
              "      <td>321</td>\n",
              "      <td>Оларның аразында 14 суурда 500 четпес, а 8 суурда 250 хире чурттакчы бар.</td>\n",
              "      <td>Среди них в 14 селах менее 500, в восьми - менее 250 человек.</td>\n",
              "      <td>train</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e2ff1a85-e59b-42eb-a460-c613f4696ac6')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e2ff1a85-e59b-42eb-a460-c613f4696ac6 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e2ff1a85-e59b-42eb-a460-c613f4696ac6');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-42e1d4cc-d2d9-49f4-9c8c-d77ea07917b0\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-42e1d4cc-d2d9-49f4-9c8c-d77ea07917b0')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-42e1d4cc-d2d9-49f4-9c8c-d77ea07917b0 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 76
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "df_joint.to_csv(\"/gd/MyDrive/datasets/nlp/tyvan/rus_tyv_parallel_50k.tsv\", sep=\"\\t\")"
      ],
      "metadata": {
        "id": "9qfu9FPwGSQu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Publishing the model to HF"
      ],
      "metadata": {
        "id": "Z1mVG4Gy9KYK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!huggingface-cli login"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "r5PuJmlJ994_",
        "outputId": "ebfdfb8f-085a-43ad-ad86-dc1090ddddc3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|\n",
            "    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|\n",
            "    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|\n",
            "    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|\n",
            "    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|\n",
            "    \n",
            "    A token is already saved on your machine. Run `huggingface-cli whoami` to get more information or `huggingface-cli logout` if you want to log out.\n",
            "    Setting a new token will erase the existing one.\n",
            "    To login, `huggingface_hub` requires a token generated from https://huggingface.co/settings/tokens .\n",
            "Token: \n",
            "Add token as git credential? (Y/n) Y\n",
            "Token is valid (permission: write).\n",
            "Your token has been saved in your configured git credential helpers (store).\n",
            "Your token has been saved to /root/.cache/huggingface/token\n",
            "Login successful\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from transformers import NllbTokenizer, AutoModelForSeq2SeqLM, AutoConfig"
      ],
      "metadata": {
        "id": "vtrBFlQp9hSb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def fix_tokenizer(tokenizer, new_lang='tyv_Cyrl'):\n",
        "    \"\"\" Add a new language token to the tokenizer vocabulary (this should be done each time after its initialization) \"\"\"\n",
        "    old_len = len(tokenizer) - int(new_lang in tokenizer.added_tokens_encoder)\n",
        "    tokenizer.lang_code_to_id[new_lang] = old_len-1\n",
        "    tokenizer.id_to_lang_code[old_len-1] = new_lang\n",
        "    # always move \"mask\" to the last position\n",
        "    tokenizer.fairseq_tokens_to_ids[\"<mask>\"] = len(tokenizer.sp_model) + len(tokenizer.lang_code_to_id) + tokenizer.fairseq_offset\n",
        "\n",
        "    tokenizer.fairseq_tokens_to_ids.update(tokenizer.lang_code_to_id)\n",
        "    tokenizer.fairseq_ids_to_tokens = {v: k for k, v in tokenizer.fairseq_tokens_to_ids.items()}\n",
        "    if new_lang not in tokenizer._additional_special_tokens:\n",
        "        tokenizer._additional_special_tokens.append(new_lang)\n",
        "    # clear the added token encoder; otherwise a new token may end up there by mistake\n",
        "    tokenizer.added_tokens_encoder = {}\n",
        "    tokenizer.added_tokens_decoder = {}"
      ],
      "metadata": {
        "id": "RmGVeHIzFiuA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model_load_name = '/gd/MyDrive/models/nllb-rus-tyv-v1'\n",
        "model = AutoModelForSeq2SeqLM.from_pretrained(model_load_name)\n",
        "tokenizer = NllbTokenizer.from_pretrained(model_load_name)\n",
        "fix_tokenizer(tokenizer)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "r6OR05X89a71",
        "outputId": "8b2a98f7-09c8-4461-9e8e-191c32d0d9d1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "upload_repo = \"slone/nllb-rus-tyv-v1\"\n",
        "tokenizer.push_to_hub(upload_repo)\n",
        "model.push_to_hub(upload_repo)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 150,
          "referenced_widgets": [
            "79d5f10ea79041c1bac7288656bca516",
            "ecb3db3792174c1a981da47b05bcf803",
            "ce3b6707fde344c2a716878a8cde3402",
            "cb9e3644d0fc4f3fa45efaa34cfc13c4",
            "57040992a43947ffa6ead81e184f7036",
            "5f244ff98c44422384025a25ad8fc779",
            "37bade299f834601b876cc1f200063fd",
            "aef8675be22c4b688533791c58d84dbb",
            "b9eb136229e949da91c0a03ad0f7f7af",
            "bc082bba84164c6e855253cc71801f3c",
            "6731bcd3e5104732b6c6b3917a5e2f90",
            "6cd6374c78f244548b79c11bcd089d16",
            "6902215d7f92404f99489865748286e3",
            "99389cae0f304a03819cd882f4ade40d",
            "ddc9b3ac87cb40b8b41a5d4bca98b2af",
            "310ac5268d70468a8ff4b53a938553af",
            "c687e45084444bc9a4bed6abe593f2ba",
            "9cf39baa725b4e0784c0113644efb60f",
            "b5bd0ea3f2ca4ee6b7a1f342274ff63a",
            "5a421731b36648168c7c417388219dad",
            "226a4c8a97ef4a359858d2cbbd13dcea",
            "bcaf9ac3a03d436b9322a14ce025611e"
          ]
        },
        "id": "zf0U6Vgf9qu-",
        "outputId": "3e80561f-a03a-4551-e6ad-3b81ee8253f6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "sentencepiece.bpe.model:   0%|          | 0.00/4.85M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "79d5f10ea79041c1bac7288656bca516"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "pytorch_model.bin:   0%|          | 0.00/2.46G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "6cd6374c78f244548b79c11bcd089d16"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "CommitInfo(commit_url='https://huggingface.co/slone/nllb-rus-tyv-v1/commit/e45e25aa62c8487aaa7050d804f9851ae01e353a', commit_message='Upload M2M100ForConditionalGeneration', commit_description='', oid='e45e25aa62c8487aaa7050d804f9851ae01e353a', pr_url=None, pr_revision=None, pr_num=None)"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model_load_name = '/gd/MyDrive/models/nllb-rus-tyv-v2-extvoc'\n",
        "tokenizer = NllbTokenizer.from_pretrained(model_load_name)\n",
        "cfg = AutoConfig.from_pretrained(model_load_name)\n",
        "model = AutoModelForSeq2SeqLM.from_pretrained(model_load_name + \"/pytorch_model_60k.bin\", config=cfg)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eSKL2avG-WX_",
        "outputId": "effced5b-a9ac-4d41-ef2d-fd015680094b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "upload_repo = \"slone/nllb-rus-tyv-v2-extvoc\"\n",
        "tokenizer.push_to_hub(upload_repo)\n",
        "model.push_to_hub(upload_repo)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 150,
          "referenced_widgets": [
            "362af541ede543768ce93e17f1d61bb2",
            "cc2ecd822b9544ca959fd9261616bc48",
            "3777e3a308fd4dda90ee586e5c10be1e",
            "96b23ce96ca24a48a394ffa896ba826b",
            "05a688fb436f4f198ae33b23a2a597d8",
            "d031b9c0721f49e3a6b0d834de8509b2",
            "3ae752dc3dee4c588a271c331c281e53",
            "b840fe71ba454d81ab26e64a6076e2b4",
            "9654591de7d44e20b22c88de1bb65bd1",
            "148b54598c4149b7975dd2a61c2b3b3e",
            "52a3480806dc48dd9223362e733ac3d7",
            "c1b1401e2ff848eaa56cd481811f5cb7",
            "ec6fdf7481424d66adac2b89f44945e4",
            "17d6237bbad7414c9d262d7834f568df",
            "d6769dad0c024ae786aca7a29f5f3c10",
            "2b8ede97bb094c2d8bcb24e00dd08ffe",
            "1722ec3afe0b4f439519c967038301cc",
            "954f57b73e554d09b6949aa58194008b",
            "dcc9e4ea553f4f5fa9aeeb38a945575a",
            "9bb9853a89704244bcbf0c1081402e0e",
            "1267c4ccdc1042f38f52bb0199ad0324",
            "d1f47279843e45c8b525ad14491b4e1d"
          ]
        },
        "id": "uldyz3aG-Y4H",
        "outputId": "717530ec-0240-4b83-b322-c310e4575906"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "sentencepiece.bpe.model:   0%|          | 0.00/5.14M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "362af541ede543768ce93e17f1d61bb2"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "pytorch_model.bin:   0%|          | 0.00/2.51G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c1b1401e2ff848eaa56cd481811f5cb7"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "CommitInfo(commit_url='https://huggingface.co/slone/nllb-rus-tyv-v2-extvoc/commit/48e9b1269b037fe08280bfec990c189e5748bccd', commit_message='Upload M2M100ForConditionalGeneration', commit_description='', oid='48e9b1269b037fe08280bfec990c189e5748bccd', pr_url=None, pr_revision=None, pr_num=None)"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(tokenizer.convert_ids_to_tokens([256202, 256203, 256204])) # ['zul_Latn', 'tyv_Cyrl', '<mask>']\n",
        "print(tokenizer.convert_tokens_to_ids(['zul_Latn', 'tyv_Cyrl', '<mask>'])) # [256202, 256203, 256204]\n",
        "# this is consistent now, wow!"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3zygd7HmFrLJ",
        "outputId": "e711c9d7-02b8-4886-911b-994915aac481"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['zul_Latn', '<mask>', 'tyv_Cyrl']\n",
            "[256202, 256204, 256203]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Testing that it works"
      ],
      "metadata": {
        "id": "qUQCkKIzO7EB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "MODEL_URL = 'slone/nllb-rus-tyv-v1'\n",
        "model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_URL)\n",
        "tokenizer = NllbTokenizer.from_pretrained(MODEL_URL, force_download=True)\n",
        "fix_tokenizer(tokenizer)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 246,
          "referenced_widgets": [
            "50a0ef4daefd47b7b7b64009f358dc41",
            "24abf099591e4a899885de04b3fd8321",
            "6859e3c6c0984676b917f79b21b56dc8",
            "c8a6ec22038d400aa37d98c894dea92a",
            "f3e7dd3724cb4ad49a8bac488e245698",
            "dd4bc071514a47ef919e5590845b2a78",
            "73d6676d04a448ebbfc80102eb3a0e58",
            "1b0e75b61e574f678b9e3d36e0802ac3",
            "b17359c60aa24e029e112fddc7f28826",
            "90b1696cb54a4bcbba6c7402bba4b649",
            "7a6767005260467bbe539944b8b6e369",
            "1423cc55b5814e4c895bdee24201f0ec",
            "d545c961cdea43ad85d78b18691d72ab",
            "a67208795d664dcb9effab4a65e3b79e",
            "507cc289eb8a4fc19864d163d95ba5fc",
            "6e82afd16a3a4c9797fd8382c78e6046",
            "5aef36278ed942789d24bacd873c6f0b",
            "f3b8e4f9b9264146b2465c708351f65d",
            "3c460f721f424ac99c91dcac2fa4bbfa",
            "e48127d45b8e433e8d2ea1eae029c781",
            "cffce888c2604c8b9fe59c858603ddf9",
            "d2ecd924600a49908c9729c0903e35c4",
            "09961195b22648ea859f993290dd6dd3",
            "7b30ceacc6ac40a48ea32ff82b34428c",
            "08536b0168fe47c993102b0f46581842",
            "6f0bebdc37c64942923c577dada7d5b9",
            "41e13138581b4197a82e91cea90194a2",
            "ef47628e909d471781ce6f83ae88f2a9",
            "38d68291eac247cf85bfb1de50d8b120",
            "083bb600d64848589c02b9d772736a8c",
            "d062239d38e74048aadcc946bb974cb7",
            "3053da3f6f31407fb5ee0dc0402bfdb0",
            "ee2c90d133ed4fa69ffa5427e0e71f4e",
            "55593e579b654a4686a97175bc9372d2",
            "5313e9e56e3549068ae32738baf0b306",
            "d2b42ed99d254880b96d6bf5813cef56",
            "875fe414f74d4de989dcf249e1653aa6",
            "e620ef40b21640079d41d28b863eb013",
            "d1f84a81069042bdae75373de6536172",
            "eb5c4067edac458fbe71d8a324aa7408",
            "1617ee5bd47d403d874325bc0770c5b4",
            "9e9b0df4e1db4685b5fe64b0df76f522",
            "258e87e4744e4621acd3cbc64b5d2618",
            "8ab6cd9e36dc4da1b90a878b72504d0e",
            "43d03284a26148f89020a1001e226801",
            "19efa6d1430e4947888691bd06bc984f",
            "20a7f65a15be4081ba5b2848eb08a652",
            "5ebeb9328cb64491b80a8e6f956e74c9",
            "1ae9df2028994482a4a0c8ea90e94295",
            "14368928461046b8a478c6850a8c0309",
            "9b7cbfa1b1a0423a978fd657004ef995",
            "8284e1e6aea14607a8cb9d39d5fa8303",
            "491343173f17475b909aaa57a445b68b",
            "006504cc2cc34662a7686b5199b3fb3a",
            "7861e68bd3654ac580a4b70a5d9fa6f9",
            "9adc761447984736adc1d059197f389e",
            "f680c6102c2f4396ad9f0b6a5842a544",
            "c24bf754dca14ea6b2929391ecfb5e39",
            "5789aacf1599425e84eb25f6d0e1374a",
            "6f37a2f65b304e93bdb952fb76e4928a",
            "af0177c68cd9469c9f3229a12d5bbc00",
            "f742607592864f01befe3ac663f5e8b7",
            "4e32326f49d84389b198eb08ae529acd",
            "ed8ff220b68f46399040a0864a1434fc",
            "8e7b505016044ab18fffa81d80e12b24",
            "553cc9502a9a4ccc9c21efdcb08aa451"
          ]
        },
        "id": "0obaABVcQ3N_",
        "outputId": "39b0b63d-7986-454e-aa11-5b997ad337d1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)lve/main/config.json:   0%|          | 0.00/898 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "50a0ef4daefd47b7b7b64009f358dc41"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading pytorch_model.bin:   0%|          | 0.00/2.46G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "1423cc55b5814e4c895bdee24201f0ec"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)neration_config.json:   0%|          | 0.00/184 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "09961195b22648ea859f993290dd6dd3"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)tencepiece.bpe.model:   0%|          | 0.00/4.85M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "55593e579b654a4686a97175bc9372d2"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)cial_tokens_map.json:   0%|          | 0.00/3.56k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "43d03284a26148f89020a1001e226801"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)okenizer_config.json:   0%|          | 0.00/570 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9adc761447984736adc1d059197f389e"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def translate(\n",
        "    text,\n",
        "    model,\n",
        "    tokenizer,\n",
        "    src_lang='rus_Cyrl',\n",
        "    tgt_lang='tyv_Cyrl',\n",
        "    max_length='auto',\n",
        "    num_beams=4,\n",
        "    no_repeat_ngram_size=4,\n",
        "    n_out=None,\n",
        "    **kwargs\n",
        "):\n",
        "    tokenizer.src_lang = src_lang\n",
        "    encoded = tokenizer(text, return_tensors=\"pt\", truncation=True, max_length=512)\n",
        "    if max_length == 'auto':\n",
        "        max_length = int(32 + 2.0 * encoded.input_ids.shape[1])\n",
        "    model.eval()\n",
        "    generated_tokens = model.generate(\n",
        "        **encoded.to(model.device),\n",
        "        forced_bos_token_id=tokenizer.lang_code_to_id[tgt_lang],\n",
        "        max_length=max_length,\n",
        "        num_beams=num_beams,\n",
        "        no_repeat_ngram_size=no_repeat_ngram_size,\n",
        "        num_return_sequences=n_out or 1,\n",
        "        **kwargs\n",
        "    )\n",
        "    out = tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)\n",
        "    if isinstance(text, str) and n_out is None:\n",
        "        return out[0]\n",
        "    return out"
      ],
      "metadata": {
        "id": "VIGg9OfFQ_vb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "translate(\"красная птица\", model=model, tokenizer=tokenizer)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "79sB2Zd6RA49",
        "outputId": "89766263-f107-4fa5-e4ed-c27a4141f672"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'кызыл куш'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "translate(\"красная птица\", model=model, tokenizer=tokenizer, tgt_lang='eng_Latn')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "m0SbeZ_jUEVV",
        "outputId": "053b868d-fd88-4bc3-81b4-3f17732eae80"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "': кызыл куш'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "lang_to_code = {\n",
        "    'Русский | Russian': 'rus_Cyrl',\n",
        "    'Тувинский | Tyvan': 'tyv_Cyrl',\n",
        "}"
      ],
      "metadata": {
        "id": "8UZldyYURnSL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def translate_wrapper(text, src, trg, correct=None):\n",
        "    src_lang = lang_to_code.get(src)\n",
        "    tgt_lang = lang_to_code.get(trg)\n",
        "    if src == trg:\n",
        "        return 'Please choose two different languages'\n",
        "    print(text, src, trg)\n",
        "    result = translate(\n",
        "        text=text,\n",
        "        model=model,\n",
        "        tokenizer=tokenizer,\n",
        "        src_lang=src_lang,\n",
        "        tgt_lang=tgt_lang,\n",
        "    )\n",
        "    return result"
      ],
      "metadata": {
        "id": "Mba9l9M9RfGG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "translate_wrapper(\"красная птица\", 'Русский | Russian', 'Тувинский | Tyvan')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "WKc3iTz6RFck",
        "outputId": "63a58904-794f-44a0-d0b4-b5afcbabc304"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "красная птица Русский | Russian Тувинский | Tyvan\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'кызыл куш'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "p2DWArceRsM-"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "3a2c606d8688424aa3b325223cfc1f08": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5ddb689b97db48aaa93001a91b86a4a2",
              "IPY_MODEL_01d711c111174832ab5bba94cfbca2a6",
              "IPY_MODEL_b3aeb847f0374c2ba47ec22c60ab2f18"
            ],
            "layout": "IPY_MODEL_b7466c53e33c4b23b9186821609eefa1"
          }
        },
        "5ddb689b97db48aaa93001a91b86a4a2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fc8279d880854b5bb526cadc437bc317",
            "placeholder": "​",
            "style": "IPY_MODEL_4b9195f13aa34a87a2859654902ef99a",
            "value": "100%"
          }
        },
        "01d711c111174832ab5bba94cfbca2a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_804c3b981cc740b79f700e7cecf9362e",
            "max": 500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4ef9639ade9e49189272e6b8012223e8",
            "value": 500
          }
        },
        "b3aeb847f0374c2ba47ec22c60ab2f18": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f87dfba0b04e42cabd525af31afae2ca",
            "placeholder": "​",
            "style": "IPY_MODEL_a82b144e2c024fc8822def3701384c12",
            "value": " 500/500 [03:58&lt;00:00,  2.46it/s]"
          }
        },
        "b7466c53e33c4b23b9186821609eefa1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fc8279d880854b5bb526cadc437bc317": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4b9195f13aa34a87a2859654902ef99a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "804c3b981cc740b79f700e7cecf9362e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4ef9639ade9e49189272e6b8012223e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f87dfba0b04e42cabd525af31afae2ca": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a82b144e2c024fc8822def3701384c12": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0ea7235036c14d438a412ae6fe857ab0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7b7348cd7da243158c0734488b422b74",
              "IPY_MODEL_4aaad174030140a69db1e81361553902",
              "IPY_MODEL_8d6093633eb44e2c9b499a1594e2b6dc"
            ],
            "layout": "IPY_MODEL_2beeb8b43c3c4153b7723184ec6ab123"
          }
        },
        "7b7348cd7da243158c0734488b422b74": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2548884655614dfda0d263cacd665c07",
            "placeholder": "​",
            "style": "IPY_MODEL_70099123340b4adca9f18dcd23cbb270",
            "value": "100%"
          }
        },
        "4aaad174030140a69db1e81361553902": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f3782ed9cd074d43a69e2978eb7b0865",
            "max": 500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3feb136a616d46b7a0100d2a507cbd95",
            "value": 500
          }
        },
        "8d6093633eb44e2c9b499a1594e2b6dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b9dfdf81307c4615b00ba80b3eef2958",
            "placeholder": "​",
            "style": "IPY_MODEL_2ea8b05ab5f540efa609befac402a7d1",
            "value": " 500/500 [04:41&lt;00:00,  1.44it/s]"
          }
        },
        "2beeb8b43c3c4153b7723184ec6ab123": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2548884655614dfda0d263cacd665c07": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "70099123340b4adca9f18dcd23cbb270": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f3782ed9cd074d43a69e2978eb7b0865": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3feb136a616d46b7a0100d2a507cbd95": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b9dfdf81307c4615b00ba80b3eef2958": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2ea8b05ab5f540efa609befac402a7d1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fa02790d3658464d85ba4fb817598142": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_009f9de0888749509335e97d828bd8b8",
              "IPY_MODEL_98f0b0ff99004eccbabba4d9859e7191",
              "IPY_MODEL_7204e6c548c9423aadbecc7656976f0d"
            ],
            "layout": "IPY_MODEL_4d25116204ee4c5999dac8e2281a1962"
          }
        },
        "009f9de0888749509335e97d828bd8b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8e7acded176c4fefa122b50b0d3126c6",
            "placeholder": "​",
            "style": "IPY_MODEL_a4ff5a65bd224cb5bc201da43cab714f",
            "value": "100%"
          }
        },
        "98f0b0ff99004eccbabba4d9859e7191": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_31f1fa84b19e4e6bb3e0391290539334",
            "max": 500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c2928804b48a423a8c5d85dc8fd0d624",
            "value": 500
          }
        },
        "7204e6c548c9423aadbecc7656976f0d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c344baeff36447e49c4d99cadf39b721",
            "placeholder": "​",
            "style": "IPY_MODEL_6c5aee3f42cb467e904e6c70e1123b4d",
            "value": " 500/500 [03:24&lt;00:00,  2.76it/s]"
          }
        },
        "4d25116204ee4c5999dac8e2281a1962": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8e7acded176c4fefa122b50b0d3126c6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a4ff5a65bd224cb5bc201da43cab714f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "31f1fa84b19e4e6bb3e0391290539334": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c2928804b48a423a8c5d85dc8fd0d624": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c344baeff36447e49c4d99cadf39b721": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6c5aee3f42cb467e904e6c70e1123b4d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e0bf99ac22274196bd5fd39fd08c843e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e64a9117c6b848fb914799426f87a000",
              "IPY_MODEL_50dfdefc8d8149b3a1d5f2bee94e88ab",
              "IPY_MODEL_3feb373ccc2d4999956a105e8502fd49"
            ],
            "layout": "IPY_MODEL_7baf6b1140284f5780331f1a32d7be65"
          }
        },
        "e64a9117c6b848fb914799426f87a000": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e1f3cea2562f4e73b4e7158f6592c1df",
            "placeholder": "​",
            "style": "IPY_MODEL_1c1fc5decda14cf6bc056a8da1f603f5",
            "value": "100%"
          }
        },
        "50dfdefc8d8149b3a1d5f2bee94e88ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_49ed7c03056f42a281f323a0bf6fe81f",
            "max": 500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c7d605ece9554217aeb6d75f468a541d",
            "value": 500
          }
        },
        "3feb373ccc2d4999956a105e8502fd49": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aff14b0c3dbb40feb14bb235dcd53aa1",
            "placeholder": "​",
            "style": "IPY_MODEL_055db7a3c88345af8025ad5b0d3b77b5",
            "value": " 500/500 [03:28&lt;00:00,  1.67it/s]"
          }
        },
        "7baf6b1140284f5780331f1a32d7be65": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e1f3cea2562f4e73b4e7158f6592c1df": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1c1fc5decda14cf6bc056a8da1f603f5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "49ed7c03056f42a281f323a0bf6fe81f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c7d605ece9554217aeb6d75f468a541d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "aff14b0c3dbb40feb14bb235dcd53aa1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "055db7a3c88345af8025ad5b0d3b77b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0de79c6d3066479d993f394472b0ceb5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4ca4884ea47145d5a5c5c77133e8d8d5",
              "IPY_MODEL_7529d94c31de4517818a94ec4530dc47",
              "IPY_MODEL_32f1be0d6f374067b074e69b65a465e5"
            ],
            "layout": "IPY_MODEL_00e2e479e76746dc8eb8084df72c8cc9"
          }
        },
        "4ca4884ea47145d5a5c5c77133e8d8d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c33d68c000dd4497ad28d7f40c69b072",
            "placeholder": "​",
            "style": "IPY_MODEL_f6d9757603664bafa0cc6120d2f649ba",
            "value": "100%"
          }
        },
        "7529d94c31de4517818a94ec4530dc47": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e5ed57e0c993466e86cc60724bcae077",
            "max": 500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b63909b1caf44642a53f2141dc8a16f4",
            "value": 500
          }
        },
        "32f1be0d6f374067b074e69b65a465e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_24f4e5a4e3a44fa5b430bc93fc4c6b87",
            "placeholder": "​",
            "style": "IPY_MODEL_813a875db57c4af899c352ef07099a41",
            "value": " 500/500 [03:14&lt;00:00,  2.77it/s]"
          }
        },
        "00e2e479e76746dc8eb8084df72c8cc9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c33d68c000dd4497ad28d7f40c69b072": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f6d9757603664bafa0cc6120d2f649ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e5ed57e0c993466e86cc60724bcae077": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b63909b1caf44642a53f2141dc8a16f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "24f4e5a4e3a44fa5b430bc93fc4c6b87": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "813a875db57c4af899c352ef07099a41": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f4bdcbd34ff54db9ae7829801b4b7757": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_95af726a251244d7bddac15e12198150",
              "IPY_MODEL_a463b4ddc73c4eee908b4d22de34131e",
              "IPY_MODEL_f65cd0a121514b7fae16225fdb171d90"
            ],
            "layout": "IPY_MODEL_e8f5fe99f4cd49d49f0ca1c5ee76aa38"
          }
        },
        "95af726a251244d7bddac15e12198150": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dac6b73bd55845d683bf21eb49c5df37",
            "placeholder": "​",
            "style": "IPY_MODEL_98439b2ffc2842188696838a66053b6f",
            "value": "100%"
          }
        },
        "a463b4ddc73c4eee908b4d22de34131e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3f4cfab3834f4d18a921732098a82048",
            "max": 500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a751fbe15eb949a5bd2b8f676f1cc8df",
            "value": 500
          }
        },
        "f65cd0a121514b7fae16225fdb171d90": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b1b56592a439415ea0e63d4f55c2d58f",
            "placeholder": "​",
            "style": "IPY_MODEL_ee24170941a14ecc80e5af8cd2370a93",
            "value": " 500/500 [03:30&lt;00:00,  1.75it/s]"
          }
        },
        "e8f5fe99f4cd49d49f0ca1c5ee76aa38": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dac6b73bd55845d683bf21eb49c5df37": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "98439b2ffc2842188696838a66053b6f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3f4cfab3834f4d18a921732098a82048": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a751fbe15eb949a5bd2b8f676f1cc8df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b1b56592a439415ea0e63d4f55c2d58f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ee24170941a14ecc80e5af8cd2370a93": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3417c70a8db54723804a9de92094eec6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_39fbccba9def4fce9e41edf0f931e667",
              "IPY_MODEL_1555767cc3334aa185368367f27b49a6",
              "IPY_MODEL_637865b7608e4aae82b8d87235d8172e"
            ],
            "layout": "IPY_MODEL_7251bc0a86ce4739b512c814941eab7d"
          }
        },
        "39fbccba9def4fce9e41edf0f931e667": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9c2a4c33dc1d4eb2adf449272e1c1d7e",
            "placeholder": "​",
            "style": "IPY_MODEL_db15b4bda74441d58fde31d2287c8cbe",
            "value": "100%"
          }
        },
        "1555767cc3334aa185368367f27b49a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_33daa47f93874045a29f7d77e5a9d7fc",
            "max": 500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ed443ff78a184cc49632b65197010277",
            "value": 500
          }
        },
        "637865b7608e4aae82b8d87235d8172e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_53aefd2209db419181b816792cacfdd8",
            "placeholder": "​",
            "style": "IPY_MODEL_025040b9a9e54109bc20ebd217bfd74c",
            "value": " 500/500 [03:21&lt;00:00,  2.91it/s]"
          }
        },
        "7251bc0a86ce4739b512c814941eab7d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9c2a4c33dc1d4eb2adf449272e1c1d7e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "db15b4bda74441d58fde31d2287c8cbe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "33daa47f93874045a29f7d77e5a9d7fc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ed443ff78a184cc49632b65197010277": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "53aefd2209db419181b816792cacfdd8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "025040b9a9e54109bc20ebd217bfd74c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "24f0dfcbe8b144828b5efb5d387c15d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_39a4ac85caa7443f8348c2e2c68b64f2",
              "IPY_MODEL_b2c8f8f89565450c85bf02e80cfadc39",
              "IPY_MODEL_6f44ede97c0b4ca79dcbea7bb31402a9"
            ],
            "layout": "IPY_MODEL_5558af48295945cb88aaefedc1f25ed4"
          }
        },
        "39a4ac85caa7443f8348c2e2c68b64f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5e371b1b259248b1bb34ba66f9febdf5",
            "placeholder": "​",
            "style": "IPY_MODEL_7009b83f25c147c8b173886f51ba21d1",
            "value": "100%"
          }
        },
        "b2c8f8f89565450c85bf02e80cfadc39": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b1de5494ad734e44934ede08df117937",
            "max": 500,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_002a75bfa1c943aca2e5c2ec06ad4900",
            "value": 500
          }
        },
        "6f44ede97c0b4ca79dcbea7bb31402a9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_65d56cb8b3584d64b94f6194f76d0f4e",
            "placeholder": "​",
            "style": "IPY_MODEL_f64c0ee1822f45c1a24f18765324cdd5",
            "value": " 500/500 [03:35&lt;00:00,  2.75it/s]"
          }
        },
        "5558af48295945cb88aaefedc1f25ed4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5e371b1b259248b1bb34ba66f9febdf5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7009b83f25c147c8b173886f51ba21d1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b1de5494ad734e44934ede08df117937": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "002a75bfa1c943aca2e5c2ec06ad4900": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "65d56cb8b3584d64b94f6194f76d0f4e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f64c0ee1822f45c1a24f18765324cdd5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "79d5f10ea79041c1bac7288656bca516": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ecb3db3792174c1a981da47b05bcf803",
              "IPY_MODEL_ce3b6707fde344c2a716878a8cde3402",
              "IPY_MODEL_cb9e3644d0fc4f3fa45efaa34cfc13c4"
            ],
            "layout": "IPY_MODEL_57040992a43947ffa6ead81e184f7036"
          }
        },
        "ecb3db3792174c1a981da47b05bcf803": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5f244ff98c44422384025a25ad8fc779",
            "placeholder": "​",
            "style": "IPY_MODEL_37bade299f834601b876cc1f200063fd",
            "value": "sentencepiece.bpe.model: 100%"
          }
        },
        "ce3b6707fde344c2a716878a8cde3402": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aef8675be22c4b688533791c58d84dbb",
            "max": 4852054,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b9eb136229e949da91c0a03ad0f7f7af",
            "value": 4852054
          }
        },
        "cb9e3644d0fc4f3fa45efaa34cfc13c4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bc082bba84164c6e855253cc71801f3c",
            "placeholder": "​",
            "style": "IPY_MODEL_6731bcd3e5104732b6c6b3917a5e2f90",
            "value": " 4.85M/4.85M [00:01&lt;00:00, 6.01MB/s]"
          }
        },
        "57040992a43947ffa6ead81e184f7036": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5f244ff98c44422384025a25ad8fc779": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "37bade299f834601b876cc1f200063fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "aef8675be22c4b688533791c58d84dbb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b9eb136229e949da91c0a03ad0f7f7af": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "bc082bba84164c6e855253cc71801f3c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6731bcd3e5104732b6c6b3917a5e2f90": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6cd6374c78f244548b79c11bcd089d16": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6902215d7f92404f99489865748286e3",
              "IPY_MODEL_99389cae0f304a03819cd882f4ade40d",
              "IPY_MODEL_ddc9b3ac87cb40b8b41a5d4bca98b2af"
            ],
            "layout": "IPY_MODEL_310ac5268d70468a8ff4b53a938553af"
          }
        },
        "6902215d7f92404f99489865748286e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c687e45084444bc9a4bed6abe593f2ba",
            "placeholder": "​",
            "style": "IPY_MODEL_9cf39baa725b4e0784c0113644efb60f",
            "value": "pytorch_model.bin: 100%"
          }
        },
        "99389cae0f304a03819cd882f4ade40d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b5bd0ea3f2ca4ee6b7a1f342274ff63a",
            "max": 2460457473,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_5a421731b36648168c7c417388219dad",
            "value": 2460457473
          }
        },
        "ddc9b3ac87cb40b8b41a5d4bca98b2af": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_226a4c8a97ef4a359858d2cbbd13dcea",
            "placeholder": "​",
            "style": "IPY_MODEL_bcaf9ac3a03d436b9322a14ce025611e",
            "value": " 2.46G/2.46G [01:22&lt;00:00, 37.3MB/s]"
          }
        },
        "310ac5268d70468a8ff4b53a938553af": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c687e45084444bc9a4bed6abe593f2ba": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9cf39baa725b4e0784c0113644efb60f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b5bd0ea3f2ca4ee6b7a1f342274ff63a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5a421731b36648168c7c417388219dad": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "226a4c8a97ef4a359858d2cbbd13dcea": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bcaf9ac3a03d436b9322a14ce025611e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "362af541ede543768ce93e17f1d61bb2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_cc2ecd822b9544ca959fd9261616bc48",
              "IPY_MODEL_3777e3a308fd4dda90ee586e5c10be1e",
              "IPY_MODEL_96b23ce96ca24a48a394ffa896ba826b"
            ],
            "layout": "IPY_MODEL_05a688fb436f4f198ae33b23a2a597d8"
          }
        },
        "cc2ecd822b9544ca959fd9261616bc48": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d031b9c0721f49e3a6b0d834de8509b2",
            "placeholder": "​",
            "style": "IPY_MODEL_3ae752dc3dee4c588a271c331c281e53",
            "value": "sentencepiece.bpe.model: 100%"
          }
        },
        "3777e3a308fd4dda90ee586e5c10be1e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b840fe71ba454d81ab26e64a6076e2b4",
            "max": 5141129,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9654591de7d44e20b22c88de1bb65bd1",
            "value": 5141129
          }
        },
        "96b23ce96ca24a48a394ffa896ba826b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_148b54598c4149b7975dd2a61c2b3b3e",
            "placeholder": "​",
            "style": "IPY_MODEL_52a3480806dc48dd9223362e733ac3d7",
            "value": " 5.14M/5.14M [00:01&lt;00:00, 5.36MB/s]"
          }
        },
        "05a688fb436f4f198ae33b23a2a597d8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d031b9c0721f49e3a6b0d834de8509b2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3ae752dc3dee4c588a271c331c281e53": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b840fe71ba454d81ab26e64a6076e2b4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9654591de7d44e20b22c88de1bb65bd1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "148b54598c4149b7975dd2a61c2b3b3e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "52a3480806dc48dd9223362e733ac3d7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c1b1401e2ff848eaa56cd481811f5cb7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ec6fdf7481424d66adac2b89f44945e4",
              "IPY_MODEL_17d6237bbad7414c9d262d7834f568df",
              "IPY_MODEL_d6769dad0c024ae786aca7a29f5f3c10"
            ],
            "layout": "IPY_MODEL_2b8ede97bb094c2d8bcb24e00dd08ffe"
          }
        },
        "ec6fdf7481424d66adac2b89f44945e4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1722ec3afe0b4f439519c967038301cc",
            "placeholder": "​",
            "style": "IPY_MODEL_954f57b73e554d09b6949aa58194008b",
            "value": "pytorch_model.bin: 100%"
          }
        },
        "17d6237bbad7414c9d262d7834f568df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dcc9e4ea553f4f5fa9aeeb38a945575a",
            "max": 2511063553,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9bb9853a89704244bcbf0c1081402e0e",
            "value": 2511063553
          }
        },
        "d6769dad0c024ae786aca7a29f5f3c10": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1267c4ccdc1042f38f52bb0199ad0324",
            "placeholder": "​",
            "style": "IPY_MODEL_d1f47279843e45c8b525ad14491b4e1d",
            "value": " 2.51G/2.51G [01:23&lt;00:00, 37.6MB/s]"
          }
        },
        "2b8ede97bb094c2d8bcb24e00dd08ffe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1722ec3afe0b4f439519c967038301cc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "954f57b73e554d09b6949aa58194008b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "dcc9e4ea553f4f5fa9aeeb38a945575a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9bb9853a89704244bcbf0c1081402e0e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1267c4ccdc1042f38f52bb0199ad0324": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d1f47279843e45c8b525ad14491b4e1d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "50a0ef4daefd47b7b7b64009f358dc41": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_24abf099591e4a899885de04b3fd8321",
              "IPY_MODEL_6859e3c6c0984676b917f79b21b56dc8",
              "IPY_MODEL_c8a6ec22038d400aa37d98c894dea92a"
            ],
            "layout": "IPY_MODEL_f3e7dd3724cb4ad49a8bac488e245698"
          }
        },
        "24abf099591e4a899885de04b3fd8321": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dd4bc071514a47ef919e5590845b2a78",
            "placeholder": "​",
            "style": "IPY_MODEL_73d6676d04a448ebbfc80102eb3a0e58",
            "value": "Downloading (…)lve/main/config.json: 100%"
          }
        },
        "6859e3c6c0984676b917f79b21b56dc8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1b0e75b61e574f678b9e3d36e0802ac3",
            "max": 898,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b17359c60aa24e029e112fddc7f28826",
            "value": 898
          }
        },
        "c8a6ec22038d400aa37d98c894dea92a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_90b1696cb54a4bcbba6c7402bba4b649",
            "placeholder": "​",
            "style": "IPY_MODEL_7a6767005260467bbe539944b8b6e369",
            "value": " 898/898 [00:00&lt;00:00, 31.5kB/s]"
          }
        },
        "f3e7dd3724cb4ad49a8bac488e245698": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dd4bc071514a47ef919e5590845b2a78": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "73d6676d04a448ebbfc80102eb3a0e58": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1b0e75b61e574f678b9e3d36e0802ac3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b17359c60aa24e029e112fddc7f28826": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "90b1696cb54a4bcbba6c7402bba4b649": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7a6767005260467bbe539944b8b6e369": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1423cc55b5814e4c895bdee24201f0ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d545c961cdea43ad85d78b18691d72ab",
              "IPY_MODEL_a67208795d664dcb9effab4a65e3b79e",
              "IPY_MODEL_507cc289eb8a4fc19864d163d95ba5fc"
            ],
            "layout": "IPY_MODEL_6e82afd16a3a4c9797fd8382c78e6046"
          }
        },
        "d545c961cdea43ad85d78b18691d72ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5aef36278ed942789d24bacd873c6f0b",
            "placeholder": "​",
            "style": "IPY_MODEL_f3b8e4f9b9264146b2465c708351f65d",
            "value": "Downloading pytorch_model.bin: 100%"
          }
        },
        "a67208795d664dcb9effab4a65e3b79e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3c460f721f424ac99c91dcac2fa4bbfa",
            "max": 2460457473,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e48127d45b8e433e8d2ea1eae029c781",
            "value": 2460457473
          }
        },
        "507cc289eb8a4fc19864d163d95ba5fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cffce888c2604c8b9fe59c858603ddf9",
            "placeholder": "​",
            "style": "IPY_MODEL_d2ecd924600a49908c9729c0903e35c4",
            "value": " 2.46G/2.46G [00:49&lt;00:00, 48.3MB/s]"
          }
        },
        "6e82afd16a3a4c9797fd8382c78e6046": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5aef36278ed942789d24bacd873c6f0b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f3b8e4f9b9264146b2465c708351f65d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3c460f721f424ac99c91dcac2fa4bbfa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e48127d45b8e433e8d2ea1eae029c781": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "cffce888c2604c8b9fe59c858603ddf9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d2ecd924600a49908c9729c0903e35c4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "09961195b22648ea859f993290dd6dd3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7b30ceacc6ac40a48ea32ff82b34428c",
              "IPY_MODEL_08536b0168fe47c993102b0f46581842",
              "IPY_MODEL_6f0bebdc37c64942923c577dada7d5b9"
            ],
            "layout": "IPY_MODEL_41e13138581b4197a82e91cea90194a2"
          }
        },
        "7b30ceacc6ac40a48ea32ff82b34428c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ef47628e909d471781ce6f83ae88f2a9",
            "placeholder": "​",
            "style": "IPY_MODEL_38d68291eac247cf85bfb1de50d8b120",
            "value": "Downloading (…)neration_config.json: 100%"
          }
        },
        "08536b0168fe47c993102b0f46581842": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_083bb600d64848589c02b9d772736a8c",
            "max": 184,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d062239d38e74048aadcc946bb974cb7",
            "value": 184
          }
        },
        "6f0bebdc37c64942923c577dada7d5b9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3053da3f6f31407fb5ee0dc0402bfdb0",
            "placeholder": "​",
            "style": "IPY_MODEL_ee2c90d133ed4fa69ffa5427e0e71f4e",
            "value": " 184/184 [00:00&lt;00:00, 5.65kB/s]"
          }
        },
        "41e13138581b4197a82e91cea90194a2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ef47628e909d471781ce6f83ae88f2a9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "38d68291eac247cf85bfb1de50d8b120": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "083bb600d64848589c02b9d772736a8c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d062239d38e74048aadcc946bb974cb7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3053da3f6f31407fb5ee0dc0402bfdb0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ee2c90d133ed4fa69ffa5427e0e71f4e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "55593e579b654a4686a97175bc9372d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5313e9e56e3549068ae32738baf0b306",
              "IPY_MODEL_d2b42ed99d254880b96d6bf5813cef56",
              "IPY_MODEL_875fe414f74d4de989dcf249e1653aa6"
            ],
            "layout": "IPY_MODEL_e620ef40b21640079d41d28b863eb013"
          }
        },
        "5313e9e56e3549068ae32738baf0b306": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d1f84a81069042bdae75373de6536172",
            "placeholder": "​",
            "style": "IPY_MODEL_eb5c4067edac458fbe71d8a324aa7408",
            "value": "Downloading (…)tencepiece.bpe.model: 100%"
          }
        },
        "d2b42ed99d254880b96d6bf5813cef56": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1617ee5bd47d403d874325bc0770c5b4",
            "max": 4852054,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9e9b0df4e1db4685b5fe64b0df76f522",
            "value": 4852054
          }
        },
        "875fe414f74d4de989dcf249e1653aa6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_258e87e4744e4621acd3cbc64b5d2618",
            "placeholder": "​",
            "style": "IPY_MODEL_8ab6cd9e36dc4da1b90a878b72504d0e",
            "value": " 4.85M/4.85M [00:00&lt;00:00, 10.2MB/s]"
          }
        },
        "e620ef40b21640079d41d28b863eb013": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d1f84a81069042bdae75373de6536172": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eb5c4067edac458fbe71d8a324aa7408": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1617ee5bd47d403d874325bc0770c5b4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9e9b0df4e1db4685b5fe64b0df76f522": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "258e87e4744e4621acd3cbc64b5d2618": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8ab6cd9e36dc4da1b90a878b72504d0e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "43d03284a26148f89020a1001e226801": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_19efa6d1430e4947888691bd06bc984f",
              "IPY_MODEL_20a7f65a15be4081ba5b2848eb08a652",
              "IPY_MODEL_5ebeb9328cb64491b80a8e6f956e74c9"
            ],
            "layout": "IPY_MODEL_1ae9df2028994482a4a0c8ea90e94295"
          }
        },
        "19efa6d1430e4947888691bd06bc984f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_14368928461046b8a478c6850a8c0309",
            "placeholder": "​",
            "style": "IPY_MODEL_9b7cbfa1b1a0423a978fd657004ef995",
            "value": "Downloading (…)cial_tokens_map.json: 100%"
          }
        },
        "20a7f65a15be4081ba5b2848eb08a652": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8284e1e6aea14607a8cb9d39d5fa8303",
            "max": 3564,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_491343173f17475b909aaa57a445b68b",
            "value": 3564
          }
        },
        "5ebeb9328cb64491b80a8e6f956e74c9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_006504cc2cc34662a7686b5199b3fb3a",
            "placeholder": "​",
            "style": "IPY_MODEL_7861e68bd3654ac580a4b70a5d9fa6f9",
            "value": " 3.56k/3.56k [00:00&lt;00:00, 73.9kB/s]"
          }
        },
        "1ae9df2028994482a4a0c8ea90e94295": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "14368928461046b8a478c6850a8c0309": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9b7cbfa1b1a0423a978fd657004ef995": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8284e1e6aea14607a8cb9d39d5fa8303": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "491343173f17475b909aaa57a445b68b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "006504cc2cc34662a7686b5199b3fb3a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7861e68bd3654ac580a4b70a5d9fa6f9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9adc761447984736adc1d059197f389e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f680c6102c2f4396ad9f0b6a5842a544",
              "IPY_MODEL_c24bf754dca14ea6b2929391ecfb5e39",
              "IPY_MODEL_5789aacf1599425e84eb25f6d0e1374a"
            ],
            "layout": "IPY_MODEL_6f37a2f65b304e93bdb952fb76e4928a"
          }
        },
        "f680c6102c2f4396ad9f0b6a5842a544": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_af0177c68cd9469c9f3229a12d5bbc00",
            "placeholder": "​",
            "style": "IPY_MODEL_f742607592864f01befe3ac663f5e8b7",
            "value": "Downloading (…)okenizer_config.json: 100%"
          }
        },
        "c24bf754dca14ea6b2929391ecfb5e39": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4e32326f49d84389b198eb08ae529acd",
            "max": 570,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ed8ff220b68f46399040a0864a1434fc",
            "value": 570
          }
        },
        "5789aacf1599425e84eb25f6d0e1374a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8e7b505016044ab18fffa81d80e12b24",
            "placeholder": "​",
            "style": "IPY_MODEL_553cc9502a9a4ccc9c21efdcb08aa451",
            "value": " 570/570 [00:00&lt;00:00, 10.1kB/s]"
          }
        },
        "6f37a2f65b304e93bdb952fb76e4928a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "af0177c68cd9469c9f3229a12d5bbc00": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f742607592864f01befe3ac663f5e8b7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4e32326f49d84389b198eb08ae529acd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ed8ff220b68f46399040a0864a1434fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8e7b505016044ab18fffa81d80e12b24": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "553cc9502a9a4ccc9c21efdcb08aa451": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ef092016e6e64903b967170f1865e82f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_363e0d43df564854af2e6caa053e56c5",
              "IPY_MODEL_7f363c96320042788c50184e91c8f154",
              "IPY_MODEL_eb8a113b5f2c4d1997adafe1ad44946e"
            ],
            "layout": "IPY_MODEL_9c7ab640a7e64a9db9e912a315f8929e"
          }
        },
        "363e0d43df564854af2e6caa053e56c5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3173699ecb454b75be1d493c0681beca",
            "placeholder": "​",
            "style": "IPY_MODEL_5efff913d4dc4631939a6642c2343cb7",
            "value": "100%"
          }
        },
        "7f363c96320042788c50184e91c8f154": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cf9bdd81f6574f5bad3f9f878a6ad847",
            "max": 16,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_48d4597749b5459b93f2480064e8015e",
            "value": 16
          }
        },
        "eb8a113b5f2c4d1997adafe1ad44946e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2b40ca2c5862407ea4dfff008dbb6649",
            "placeholder": "​",
            "style": "IPY_MODEL_1743e3e64cf540ac902d423a05b94305",
            "value": " 16/16 [00:31&lt;00:00,  1.21s/it]"
          }
        },
        "9c7ab640a7e64a9db9e912a315f8929e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3173699ecb454b75be1d493c0681beca": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5efff913d4dc4631939a6642c2343cb7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cf9bdd81f6574f5bad3f9f878a6ad847": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "48d4597749b5459b93f2480064e8015e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2b40ca2c5862407ea4dfff008dbb6649": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1743e3e64cf540ac902d423a05b94305": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a3cce029e48b451ab7e3b3cd6de9fbb5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_df100e6cd14344d9850db922a5e13450",
              "IPY_MODEL_48c6fab6b4c6428abcc5208231fbe054",
              "IPY_MODEL_caa478c43f7c4034b483a2e2697265fb"
            ],
            "layout": "IPY_MODEL_e99da02112534d7c9a1eac3bb414ece1"
          }
        },
        "df100e6cd14344d9850db922a5e13450": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ff797d41acda4a5f930c379d86514042",
            "placeholder": "​",
            "style": "IPY_MODEL_5a0058193e6e4ad19169f63a65f3bf51",
            "value": "sentencepiece.bpe.model: 100%"
          }
        },
        "48c6fab6b4c6428abcc5208231fbe054": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_26469d725ff040279e89d6553ca51218",
            "max": 4852054,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ff9672ff7b154d1f811984b723c497d1",
            "value": 4852054
          }
        },
        "caa478c43f7c4034b483a2e2697265fb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fa16d40023fe4587986cfee853426a48",
            "placeholder": "​",
            "style": "IPY_MODEL_665cc2bf3da6485aac757b3db5e0472c",
            "value": " 4.85M/4.85M [00:00&lt;00:00, 107MB/s]"
          }
        },
        "e99da02112534d7c9a1eac3bb414ece1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ff797d41acda4a5f930c379d86514042": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5a0058193e6e4ad19169f63a65f3bf51": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "26469d725ff040279e89d6553ca51218": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ff9672ff7b154d1f811984b723c497d1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "fa16d40023fe4587986cfee853426a48": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "665cc2bf3da6485aac757b3db5e0472c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a092f5d3de9b4de183c6c24bbb94fd61": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0b3e1204ef4049e492b98f6a04f23800",
              "IPY_MODEL_40bdb135046f4cdea1bfca440fec082b",
              "IPY_MODEL_1fdff1ba87df48338d0d5eb3b47ec3a0"
            ],
            "layout": "IPY_MODEL_9deec01ab0ce44d890a3195b8ae2a3ab"
          }
        },
        "0b3e1204ef4049e492b98f6a04f23800": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_45ffe8bd2c21400187966e6fa7fb1aa9",
            "placeholder": "​",
            "style": "IPY_MODEL_ce35a79a4256487f9c8129cb88696fc1",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "40bdb135046f4cdea1bfca440fec082b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7b74fe6ab70a4dfd8ca86fc41d4aa11c",
            "max": 3548,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0b3ac9ae5cec4aefbd73834d3e5c6e7e",
            "value": 3548
          }
        },
        "1fdff1ba87df48338d0d5eb3b47ec3a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c999c707f6174d0f8d1d7c4488dc06eb",
            "placeholder": "​",
            "style": "IPY_MODEL_2fad97512f17469aa5c6ed607e72247b",
            "value": " 3.55k/3.55k [00:00&lt;00:00, 55.8kB/s]"
          }
        },
        "9deec01ab0ce44d890a3195b8ae2a3ab": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "45ffe8bd2c21400187966e6fa7fb1aa9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ce35a79a4256487f9c8129cb88696fc1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7b74fe6ab70a4dfd8ca86fc41d4aa11c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0b3ac9ae5cec4aefbd73834d3e5c6e7e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c999c707f6174d0f8d1d7c4488dc06eb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2fad97512f17469aa5c6ed607e72247b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "23f7497f85ee46d5ba558b61c883b431": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2fea688288aa4b0e9ce4b9a3d26d2d6a",
              "IPY_MODEL_917536ea52a84e1ea2e708a5786bc79d",
              "IPY_MODEL_debb6444da7c4410b2a137fb2f2db62f"
            ],
            "layout": "IPY_MODEL_b465d4f10c10450baa1ed660487b3929"
          }
        },
        "2fea688288aa4b0e9ce4b9a3d26d2d6a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7ced34a412a74e0ea53e07e6624d3b85",
            "placeholder": "​",
            "style": "IPY_MODEL_0492f2a503284dbdb105b8c443dc53b0",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "917536ea52a84e1ea2e708a5786bc79d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fd5e94baf2e445e89e6ea77f14e22647",
            "max": 564,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f5674c11b43e4eec9964626fb3ba60e8",
            "value": 564
          }
        },
        "debb6444da7c4410b2a137fb2f2db62f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e1af2035af3c4c2dba8ae5da974978e9",
            "placeholder": "​",
            "style": "IPY_MODEL_08f92e34af284860a3d72c6c24899aec",
            "value": " 564/564 [00:00&lt;00:00, 13.4kB/s]"
          }
        },
        "b465d4f10c10450baa1ed660487b3929": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7ced34a412a74e0ea53e07e6624d3b85": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0492f2a503284dbdb105b8c443dc53b0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fd5e94baf2e445e89e6ea77f14e22647": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f5674c11b43e4eec9964626fb3ba60e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e1af2035af3c4c2dba8ae5da974978e9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "08f92e34af284860a3d72c6c24899aec": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "38a8a1febe7a4c15be012c046d943b7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_363bc9e667054eb1b3697e627c87373f",
              "IPY_MODEL_04ab7ee6804a4faab6eb88a91915ea88",
              "IPY_MODEL_4506edf6e191456b833683bdcf99ebd5"
            ],
            "layout": "IPY_MODEL_5ef9c560df0a46eb995d1ecba63ac8c5"
          }
        },
        "363bc9e667054eb1b3697e627c87373f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ec638390ce3a4a509ad29a8d8c71b63e",
            "placeholder": "​",
            "style": "IPY_MODEL_301bff8171df4572bc8ec9f969110a69",
            "value": "100%"
          }
        },
        "04ab7ee6804a4faab6eb88a91915ea88": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_75ca7cf36acc4ef4ab29217dae170984",
            "max": 15021,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e849b6a3c8bb4b73ab742c8db6ad2323",
            "value": 15021
          }
        },
        "4506edf6e191456b833683bdcf99ebd5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5dee707988dc4a1b885c785f1c5fdaab",
            "placeholder": "​",
            "style": "IPY_MODEL_aa8d19e5f70d4a5a98ac828a4c5eb55f",
            "value": " 15021/15021 [00:14&lt;00:00, 2445.58it/s]"
          }
        },
        "5ef9c560df0a46eb995d1ecba63ac8c5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ec638390ce3a4a509ad29a8d8c71b63e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "301bff8171df4572bc8ec9f969110a69": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "75ca7cf36acc4ef4ab29217dae170984": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e849b6a3c8bb4b73ab742c8db6ad2323": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5dee707988dc4a1b885c785f1c5fdaab": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa8d19e5f70d4a5a98ac828a4c5eb55f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b358392d266c451694a78d1c9303b4fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_160ccde1b33044c59b0cd27f81e3a358",
              "IPY_MODEL_40fe8b64f2414c8eaffc672494f8e115",
              "IPY_MODEL_1a265a8a39e5445183f1d73147715aa8"
            ],
            "layout": "IPY_MODEL_e139fa59521b49f6908081473c20b9f8"
          }
        },
        "160ccde1b33044c59b0cd27f81e3a358": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6e5db85fe0ca462da2a32b7bd32c7ec2",
            "placeholder": "​",
            "style": "IPY_MODEL_f3f7440ec22c4c8483d4e7744395980b",
            "value": "100%"
          }
        },
        "40fe8b64f2414c8eaffc672494f8e115": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_53e3a3e3444c41d4a3f5f1941bc0c19f",
            "max": 389,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1589724f361a42b0990b83d9e8d12aa5",
            "value": 389
          }
        },
        "1a265a8a39e5445183f1d73147715aa8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_219fe916d2094b56a81087231bcd7604",
            "placeholder": "​",
            "style": "IPY_MODEL_85d483f550da4b8481106489f44a794d",
            "value": " 389/389 [00:00&lt;00:00, 1352.75it/s]"
          }
        },
        "e139fa59521b49f6908081473c20b9f8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6e5db85fe0ca462da2a32b7bd32c7ec2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f3f7440ec22c4c8483d4e7744395980b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "53e3a3e3444c41d4a3f5f1941bc0c19f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1589724f361a42b0990b83d9e8d12aa5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "219fe916d2094b56a81087231bcd7604": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "85d483f550da4b8481106489f44a794d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e95932f91bb6487585b1dac0e17af7b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_b556be65181f436f995eb21e7f458b00",
              "IPY_MODEL_114aaf3174eb4f838e50c19558383b04",
              "IPY_MODEL_6bdba58bd25444f6b8a61e4958459662"
            ],
            "layout": "IPY_MODEL_2c71903d089f478eac895388be3f789e"
          }
        },
        "b556be65181f436f995eb21e7f458b00": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1b69592b3eee4327a4aa87b9b1351eee",
            "placeholder": "​",
            "style": "IPY_MODEL_14aa13da61c740b5ad53eaf1780f083b",
            "value": "config.json: 100%"
          }
        },
        "114aaf3174eb4f838e50c19558383b04": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f66db39bad81474aaf634825fd88ee36",
            "max": 846,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_eb86a7722b894e978400e65198a1c7f7",
            "value": 846
          }
        },
        "6bdba58bd25444f6b8a61e4958459662": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e7dfbbced006476dae5c160f1d363a85",
            "placeholder": "​",
            "style": "IPY_MODEL_3a3374913d8c440794587bb7a2859154",
            "value": " 846/846 [00:00&lt;00:00, 26.6kB/s]"
          }
        },
        "2c71903d089f478eac895388be3f789e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1b69592b3eee4327a4aa87b9b1351eee": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "14aa13da61c740b5ad53eaf1780f083b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f66db39bad81474aaf634825fd88ee36": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eb86a7722b894e978400e65198a1c7f7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e7dfbbced006476dae5c160f1d363a85": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3a3374913d8c440794587bb7a2859154": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "18eb8063745a4566abee6f69eb7bc2b9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_752a364aaaff4cd3a54fb256bb8b5f65",
              "IPY_MODEL_b1b5b3c9ed3e4aa08db37615af52a291",
              "IPY_MODEL_518b3c406c2346d7ad279bcdc2243d70"
            ],
            "layout": "IPY_MODEL_fb75663870b346ac930e68431c8810d8"
          }
        },
        "752a364aaaff4cd3a54fb256bb8b5f65": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bba9a22a4b6c49a7b725d58fe8bf4414",
            "placeholder": "​",
            "style": "IPY_MODEL_5407739c7a044787a388cbee927136d0",
            "value": "pytorch_model.bin: 100%"
          }
        },
        "b1b5b3c9ed3e4aa08db37615af52a291": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c5333d0501d748d7bd99ca3f3358751d",
            "max": 2460457927,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0223df94c49445b0abc6fc7986f8c749",
            "value": 2460457927
          }
        },
        "518b3c406c2346d7ad279bcdc2243d70": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5941bc78a34c4cf5a9d22a811380c9c7",
            "placeholder": "​",
            "style": "IPY_MODEL_2a50c218d62b4e279f4bd470c8ceb772",
            "value": " 2.46G/2.46G [00:13&lt;00:00, 223MB/s]"
          }
        },
        "fb75663870b346ac930e68431c8810d8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bba9a22a4b6c49a7b725d58fe8bf4414": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5407739c7a044787a388cbee927136d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c5333d0501d748d7bd99ca3f3358751d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0223df94c49445b0abc6fc7986f8c749": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5941bc78a34c4cf5a9d22a811380c9c7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2a50c218d62b4e279f4bd470c8ceb772": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5f6ba05a2caf43658644638b293e5dd4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6e0c3d6f87774c528ba452c8a07ba3a1",
              "IPY_MODEL_d7e06ccdd1bd4315a16392b34ea8bf52",
              "IPY_MODEL_8487a78750d841cd9c402ba016be5b09"
            ],
            "layout": "IPY_MODEL_7838363f24944d1184758a7745cdb056"
          }
        },
        "6e0c3d6f87774c528ba452c8a07ba3a1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4dace0ae80cb47ce9879a94028826e35",
            "placeholder": "​",
            "style": "IPY_MODEL_8fc2ec641a7d446ca9bac60c82d557dd",
            "value": "generation_config.json: 100%"
          }
        },
        "d7e06ccdd1bd4315a16392b34ea8bf52": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_87c2086cf6b14c198da5fe35b59b6dcc",
            "max": 189,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_bee75c5460ea459e960e4c56af75c35d",
            "value": 189
          }
        },
        "8487a78750d841cd9c402ba016be5b09": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1b8f53ff04434cb59a4b4ab3adecfd38",
            "placeholder": "​",
            "style": "IPY_MODEL_4b5a8dd1723d41a2b74e37eff62a7395",
            "value": " 189/189 [00:00&lt;00:00, 13.3kB/s]"
          }
        },
        "7838363f24944d1184758a7745cdb056": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4dace0ae80cb47ce9879a94028826e35": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8fc2ec641a7d446ca9bac60c82d557dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "87c2086cf6b14c198da5fe35b59b6dcc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bee75c5460ea459e960e4c56af75c35d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1b8f53ff04434cb59a4b4ab3adecfd38": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4b5a8dd1723d41a2b74e37eff62a7395": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "26b724eadf264317bff2da2b3cc7fa59": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_18cebb22a2df4e458694eea2d28a451f",
              "IPY_MODEL_599293ac93044a4dba47d5b98ee38b57",
              "IPY_MODEL_227b1591dbde44c3afa0e45ab95be0d7"
            ],
            "layout": "IPY_MODEL_abdca147cab842788870a2849696e071"
          }
        },
        "18cebb22a2df4e458694eea2d28a451f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6a436d06080743fba25772d2df3ce5c5",
            "placeholder": "​",
            "style": "IPY_MODEL_48b626d1d8f349e3982c3793ca7f006e",
            "value": "  4%"
          }
        },
        "599293ac93044a4dba47d5b98ee38b57": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_080f970af40d498ab21ecc31365b6f7c",
            "max": 57000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_71a00bde017844919bfb981d179a2eb2",
            "value": 2301
          }
        },
        "227b1591dbde44c3afa0e45ab95be0d7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6217008254a542128962af75eee3b979",
            "placeholder": "​",
            "style": "IPY_MODEL_2eaed57764954dbe9772864307668094",
            "value": " 2301/57000 [1:09:57&lt;26:25:09,  1.74s/it]"
          }
        },
        "abdca147cab842788870a2849696e071": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6a436d06080743fba25772d2df3ce5c5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "48b626d1d8f349e3982c3793ca7f006e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "080f970af40d498ab21ecc31365b6f7c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "71a00bde017844919bfb981d179a2eb2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6217008254a542128962af75eee3b979": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2eaed57764954dbe9772864307668094": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}