![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
]
},
{
"cell_type": "markdown",
"source": [
"# [COSCUP 2022] Google Colab + Hugging Face：帶你快速認識NLP\n",
"\n",
"作者：江尚軒 (Andy Chiang)\n",
"\n",
"這是 **Google Colab + Hugging Face：帶你快速認識NLP** 的範例程式碼。"
],
"metadata": {
"id": "3jqxrl1neWBa"
}
},
{
"cell_type": "markdown",
"source": [
"## Before you start...\n",
"\n",
"1. 請先點 **檔案 >> 在雲端硬碟中儲存副本**\n",
"2. 然後切換到你自己的副本\n",
"3. 接著點 **執行階段 >> 變更執行階段類型**\n",
"4. 硬體加速器選 **GPU**\n",
"5. 點擊右上角 **連線** 就可以開始執行了!"
],
"metadata": {
"id": "KpmiLig_qhPE"
}
},
{
"cell_type": "markdown",
"source": [
"## Install packages"
],
"metadata": {
"id": "YTUdiW8ce5t_"
}
},
{
"cell_type": "markdown",
"source": [
"因為 Colab 內建沒有 transformers 和 datasets 套件，所以要先用底下的指令下載。\n",
"\n",
"(如果出現紅字，是因為套件版本有衝突，重跑一次就好了)"
],
"metadata": {
"id": "ehDtyU0moRXw"
}
},
{
"cell_type": "code",
"source": [
"!pip install transformers datasets"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "tn4bMD5geVme",
"outputId": "a2352614-2042-4d32-90ef-010a03a2c26a"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Requirement already satisfied: transformers in /usr/local/lib/python3.7/dist-packages (4.20.1)\n",
"Requirement already satisfied: datasets in /usr/local/lib/python3.7/dist-packages (2.3.2)\n",
"Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers) (4.12.0)\n",
"Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.64.0)\n",
"Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (21.3)\n",
"Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.7.1)\n",
"Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (6.0)\n",
"Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in /usr/local/lib/python3.7/dist-packages (from transformers) (0.12.1)\n",
"Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (0.8.1)\n",
"Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.21.6)\n",
"Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2022.6.2)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)\n",
"Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (4.1.1)\n",
"Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->transformers) (3.0.9)\n",
"Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.7/dist-packages (from datasets) (6.0.1)\n",
"Requirement already satisfied: xxhash in /usr/local/lib/python3.7/dist-packages (from datasets) (3.0.0)\n",
"Requirement already satisfied: aiohttp in /usr/local/lib/python3.7/dist-packages (from datasets) (3.8.1)\n",
"Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.7/dist-packages (from datasets) (0.18.0)\n",
"Requirement already satisfied: dill<0.3.6 in /usr/local/lib/python3.7/dist-packages (from datasets) (0.3.5.1)\n",
"Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from datasets) (1.3.5)\n",
"Requirement already satisfied: fsspec[http]>=2021.05.0 in /usr/local/lib/python3.7/dist-packages (from datasets) (2022.5.0)\n",
"Requirement already satisfied: multiprocess in /usr/local/lib/python3.7/dist-packages (from datasets) (0.70.13)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4)\n",
"Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.25.11)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2022.6.15)\n",
"Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (2.1.0)\n",
"Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (21.4.0)\n",
"Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (1.7.2)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (6.0.2)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (1.2.0)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (1.3.0)\n",
"Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (0.13.0)\n",
"Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp->datasets) (4.0.2)\n",
"Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers) (3.8.1)\n",
"Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas->datasets) (2.8.2)\n",
"Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->datasets) (2022.1)\n",
"Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas->datasets) (1.15.0)\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Pipeline"
],
"metadata": {
"id": "tJ_j0hmLhA7b"
}
},
{
"cell_type": "markdown",
"source": [
"使用 `pipeline()` 是完成NLP任務最簡單的方式。"
],
"metadata": {
"id": "cVWMLvYFnUwx"
}
},
{
"cell_type": "markdown",
"source": [
"### 各種NLP任務"
],
"metadata": {
"id": "nFcGlo0rhKEY"
}
},
{
"cell_type": "markdown",
"source": [
"首先，引用 `pipeline()` 並指定任務為**情感分析 (sentiment-analysis)**，它會自動幫你抓最好的模型下來。"
],
"metadata": {
"id": "0RTh1fpXnmMV"
}
},
{
"cell_type": "code",
"source": [
"from transformers import pipeline\n",
"\n",
"classifier = pipeline(\"sentiment-analysis\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 163,
"referenced_widgets": [
"f602f949cd594cd0bf0dfbbae9bcd9e0",
"55d8ae5ff4ab4099bc1352b4d6497a4a",
"1a128a2a7f8c445982962b13971ae6cf",
"1aa019b555504c68855b1ff9531d854f",
"486d11f3671148ec9c2024aff9fabbeb",
"1ef3c14595fe4ed99a4523e2258b3224",
"8ebc4f6df9914495acd5162fe1479782",
"52a65f4c763b436581ef9bef05244ae2",
"77a3e87b77ed457fa55639c8bd4de3fa",
"175911d3dfc04c22b35445baddb24b12",
"336f5a23aedb45ab9152960e31f38fab",
"45479de6f6b54521894e6996c24e8934",
"d564f957d044412496d2ac6f5f07c049",
"f2c8494db4284ab987c0615b0c0fffe3",
"1ea805fc7bdb469f9307a874a4c04fa5",
"56334fb1ca0e408facd2e027e96715c0",
"589dcdf169a24cc789ca89d2805ff39b",
"a110726cb6e94a4c9adb36a5bbbc71cb",
"2e638f4fcde5419a87a03f7d8f02f4f2",
"d9b64a337030487b949be00e70c3299e",
"ecd4d5bae822461fb37f4217c398c66c",
"d45e2d4af612498a8ab207ae61507f04",
"5c3c45b6c2164147a523897ba75a8cf5",
"5ca3853207ff4ff5afece51cbf33403d",
"9295ef73b84a40fdb3d73d3d530bef5c",
"2dbe19a662a14e3a989f75ab41fa047e",
"fcdd7662c41448409f7abb15d26136ca",
"cc7ebe90ca65417bb84d92d5c31438f0",
"1decd1c2c17149f7999267992635f559",
"4a680ae5b5104d08a72cc439c36493ee",
"cb6329fbfcb841c0b468683f3005d590",
"0e5182e8fa7346d4ba8e3c98310ccee2",
"abf334d96bd6471182529961b777759e",
"f0d04dbb645f491484e7dabb23edfa36",
"fc7556c8855b45b6bb13d4decfe1eb14",
"c0d04e7690a84689a0fa7268d2c3ced0",
"081c0b3f6fae4b2399dae705ebb0e9d1",
"2e3eb512e5284d259a385bccb2606fe0",
"fcb4cde7336140429631e884daf82b98",
"a6aaeefeb5684951b8ddfa59e70e6b25",
"69e7e20b083a4203b9bc8fdec49f8d2c",
"e776e2f02015479e9dfb4d729e0d3079",
"dad8f2119f9b4352959362bb17af7659",
"0778e32476cd436590fdabf40bc4fb6b"
]
},
"id": "jrhkESwTnfI_",
"outputId": "4ce38f92-896c-49fd-c09f-ec46eedbe89a"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"No model was supplied, defaulted to distilbert-base-uncased-finetuned-sst-2-english (https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english)\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading: 0%| | 0.00/629 [00:00 of the transform datasets.arrow_dataset.Dataset._map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed.\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
" 0%| | 0/4 [00:00"
],
"text/html": [
"\n",
" | Epoch | \n", "Training Loss | \n", "Validation Loss | \n", "Accuracy | \n", "
|---|---|---|---|
| 1 | \n", "1.307200 | \n", "0.988094 | \n", "0.577000 | \n", "
| 2 | \n", "0.936900 | \n", "0.970908 | \n", "0.596000 | \n", "
| 3 | \n", "0.621400 | \n", "1.052324 | \n", "0.606000 | \n", "
"
]
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"Saving model checkpoint to test_trainer/checkpoint-500\n",
"Configuration saved in test_trainer/checkpoint-500/config.json\n",
"Model weights saved in test_trainer/checkpoint-500/pytorch_model.bin\n",
"The following columns in the evaluation set don't have a corresponding argument in `BertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `BertForSequenceClassification.forward`, you can safely ignore this message.\n",
"***** Running Evaluation *****\n",
" Num examples = 1000\n",
" Batch size = 8\n",
"Saving model checkpoint to test_trainer/checkpoint-1000\n",
"Configuration saved in test_trainer/checkpoint-1000/config.json\n",
"Model weights saved in test_trainer/checkpoint-1000/pytorch_model.bin\n",
"The following columns in the evaluation set don't have a corresponding argument in `BertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `BertForSequenceClassification.forward`, you can safely ignore this message.\n",
"***** Running Evaluation *****\n",
" Num examples = 1000\n",
" Batch size = 8\n",
"Saving model checkpoint to test_trainer/checkpoint-1500\n",
"Configuration saved in test_trainer/checkpoint-1500/config.json\n",
"Model weights saved in test_trainer/checkpoint-1500/pytorch_model.bin\n",
"The following columns in the evaluation set don't have a corresponding argument in `BertForSequenceClassification.forward` and have been ignored: text. If text are not expected by `BertForSequenceClassification.forward`, you can safely ignore this message.\n",
"***** Running Evaluation *****\n",
" Num examples = 1000\n",
" Batch size = 8\n",
"\n",
"\n",
"Training completed. Do not forget to share your model on huggingface.co/models =)\n",
"\n",
"\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"TrainOutput(global_step=1500, training_loss=0.9551491088867188, metrics={'train_runtime': 1340.1922, 'train_samples_per_second': 8.954, 'train_steps_per_second': 1.119, 'total_flos': 3157417709568000.0, 'train_loss': 0.9551491088867188, 'epoch': 3.0})"
]
},
"metadata": {},
"execution_count": 17
}
]
},
{
"cell_type": "markdown",
"source": [
"## Save a model"
],
"metadata": {
"id": "Idm7wPjpldQ_"
}
},
{
"cell_type": "markdown",
"source": [
"訓練完後，可以使用 `save_model()` 這個函數將模型下載下來，模型會下載在 Colab 的虛擬機器中。"
],
"metadata": {
"id": "idvD6Kt05hYi"
}
},
{
"cell_type": "code",
"source": [
"trainer.save_model(\"./test_model\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-voH-zI9nNFl",
"outputId": "cd3b28a1-ad45-4083-80d8-3fda695c02e0"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"Saving model checkpoint to ./test_model\n",
"Configuration saved in ./test_model/config.json\n",
"Model weights saved in ./test_model/pytorch_model.bin\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"使用 `from_pretrained()` 就可以載入模型來使用。"
],
"metadata": {
"id": "IMaD8cAf6MYG"
}
},
{
"cell_type": "code",
"source": [
"from transformers import AutoModelForSequenceClassification\n",
"\n",
"pt_model = AutoModelForSequenceClassification.from_pretrained(\"./test_model\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "8kD97C6ltjL8",
"outputId": "bb50ea10-d658-41d7-b1de-b22d23fbf9b7"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"loading configuration file ./test_model/config.json\n",
"Model config BertConfig {\n",
" \"_name_or_path\": \"./test_model\",\n",
" \"architectures\": [\n",
" \"BertForSequenceClassification\"\n",
" ],\n",
" \"attention_probs_dropout_prob\": 0.1,\n",
" \"classifier_dropout\": null,\n",
" \"gradient_checkpointing\": false,\n",
" \"hidden_act\": \"gelu\",\n",
" \"hidden_dropout_prob\": 0.1,\n",
" \"hidden_size\": 768,\n",
" \"id2label\": {\n",
" \"0\": \"LABEL_0\",\n",
" \"1\": \"LABEL_1\",\n",
" \"2\": \"LABEL_2\",\n",
" \"3\": \"LABEL_3\",\n",
" \"4\": \"LABEL_4\"\n",
" },\n",
" \"initializer_range\": 0.02,\n",
" \"intermediate_size\": 3072,\n",
" \"label2id\": {\n",
" \"LABEL_0\": 0,\n",
" \"LABEL_1\": 1,\n",
" \"LABEL_2\": 2,\n",
" \"LABEL_3\": 3,\n",
" \"LABEL_4\": 4\n",
" },\n",
" \"layer_norm_eps\": 1e-12,\n",
" \"max_position_embeddings\": 512,\n",
" \"model_type\": \"bert\",\n",
" \"num_attention_heads\": 12,\n",
" \"num_hidden_layers\": 12,\n",
" \"pad_token_id\": 0,\n",
" \"position_embedding_type\": \"absolute\",\n",
" \"problem_type\": \"single_label_classification\",\n",
" \"torch_dtype\": \"float32\",\n",
" \"transformers_version\": \"4.20.1\",\n",
" \"type_vocab_size\": 2,\n",
" \"use_cache\": true,\n",
" \"vocab_size\": 28996\n",
"}\n",
"\n",
"loading weights file ./test_model/pytorch_model.bin\n",
"All model checkpoint weights were used when initializing BertForSequenceClassification.\n",
"\n",
"All the weights of BertForSequenceClassification were initialized from the model checkpoint at ./test_model.\n",
"If your task is similar to the task the model of the checkpoint was trained on, you can already use BertForSequenceClassification for predictions without further training.\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Hugging Face 還有一個很強大的功能，就在於 Pytorch 和 TensorFlow 的模型可以輕易轉換。\n",
"\n",
"剛才下載的是 Pytorch 的模型，但只要在使用 TensorFlow 的模型載入時加上 `from_pt=True`，就可以轉換成功了!"
],
"metadata": {
"id": "KRE0lwaQ6gUr"
}
},
{
"cell_type": "code",
"source": [
"from transformers import TFAutoModelForSequenceClassification\n",
"\n",
"tf_model = TFAutoModelForSequenceClassification.from_pretrained(\"./test_model\", from_pt=True)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "7AFZulgYubU1",
"outputId": "1e43e816-2ae7-49bc-a92f-3b1d578a66ff"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"loading configuration file ./test_model/config.json\n",
"Model config BertConfig {\n",
" \"_name_or_path\": \"./test_model\",\n",
" \"architectures\": [\n",
" \"BertForSequenceClassification\"\n",
" ],\n",
" \"attention_probs_dropout_prob\": 0.1,\n",
" \"classifier_dropout\": null,\n",
" \"gradient_checkpointing\": false,\n",
" \"hidden_act\": \"gelu\",\n",
" \"hidden_dropout_prob\": 0.1,\n",
" \"hidden_size\": 768,\n",
" \"id2label\": {\n",
" \"0\": \"LABEL_0\",\n",
" \"1\": \"LABEL_1\",\n",
" \"2\": \"LABEL_2\",\n",
" \"3\": \"LABEL_3\",\n",
" \"4\": \"LABEL_4\"\n",
" },\n",
" \"initializer_range\": 0.02,\n",
" \"intermediate_size\": 3072,\n",
" \"label2id\": {\n",
" \"LABEL_0\": 0,\n",
" \"LABEL_1\": 1,\n",
" \"LABEL_2\": 2,\n",
" \"LABEL_3\": 3,\n",
" \"LABEL_4\": 4\n",
" },\n",
" \"layer_norm_eps\": 1e-12,\n",
" \"max_position_embeddings\": 512,\n",
" \"model_type\": \"bert\",\n",
" \"num_attention_heads\": 12,\n",
" \"num_hidden_layers\": 12,\n",
" \"pad_token_id\": 0,\n",
" \"position_embedding_type\": \"absolute\",\n",
" \"problem_type\": \"single_label_classification\",\n",
" \"torch_dtype\": \"float32\",\n",
" \"transformers_version\": \"4.20.1\",\n",
" \"type_vocab_size\": 2,\n",
" \"use_cache\": true,\n",
" \"vocab_size\": 28996\n",
"}\n",
"\n",
"loading weights file ./test_model/pytorch_model.bin\n",
"Loading PyTorch weights from /content/test_model/pytorch_model.bin\n",
"PyTorch checkpoint contains 108,314,629 parameters\n",
"Loaded 108,314,117 parameters in the TF 2.0 model.\n",
"Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertForSequenceClassification: ['bert.embeddings.position_ids']\n",
"- This IS expected if you are initializing TFBertForSequenceClassification from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).\n",
"- This IS NOT expected if you are initializing TFBertForSequenceClassification from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).\n",
"All the weights of TFBertForSequenceClassification were initialized from the PyTorch model.\n",
"If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Share a model"
],
"metadata": {
"id": "aNB3ePpalkOT"
}
},
{
"cell_type": "markdown",
"source": [
"都來 COSCUP 了，就來講點開源的事情吧! 相信我，這非常的簡單!\n",
"\n",
"如果你想把 Fine-tune 好的模型分享給全世界的人，首先你需要下載 **huggingface_hub** 套件。"
],
"metadata": {
"id": "swTuNhoY7hY0"
}
},
{
"cell_type": "code",
"source": [
"!pip install huggingface_hub"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "COsGev99vUCM",
"outputId": "8a6cd398-954c-4920-a271-64e17f673ba2"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting huggingface_hub\n",
" Downloading huggingface_hub-0.8.1-py3-none-any.whl (101 kB)\n",
"\u001b[K |████████████████████████████████| 101 kB 4.5 MB/s \n",
"\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from huggingface_hub) (3.7.1)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from huggingface_hub) (2.23.0)\n",
"Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from huggingface_hub) (4.12.0)\n",
"Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface_hub) (4.1.1)\n",
"Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.7/dist-packages (from huggingface_hub) (21.3)\n",
"Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from huggingface_hub) (4.64.0)\n",
"Collecting pyyaml>=5.1\n",
" Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)\n",
"\u001b[K |████████████████████████████████| 596 kB 28.5 MB/s \n",
"\u001b[?25hRequirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.9->huggingface_hub) (3.0.9)\n",
"Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->huggingface_hub) (3.8.1)\n",
"Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->huggingface_hub) (1.24.3)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->huggingface_hub) (2.10)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->huggingface_hub) (3.0.4)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->huggingface_hub) (2022.6.15)\n",
"Installing collected packages: pyyaml, huggingface-hub\n",
" Attempting uninstall: pyyaml\n",
" Found existing installation: PyYAML 3.13\n",
" Uninstalling PyYAML-3.13:\n",
" Successfully uninstalled PyYAML-3.13\n",
"Successfully installed huggingface-hub-0.8.1 pyyaml-6.0\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"下一步要登入，執行後會出現如同下面的提示，要求你輸入一個 access token。\n",
"\n",
"\n",
"\n",
"點擊提示裡的[連結](https://huggingface.co/settings/tokens)，會傳到 Hugging Face 的網站，新增一個權限是 **write** 的 token。\n",
"\n",
"\n",
"\n",
"然後將這個 token 填入文字框內並按 login，就能成功登入了。如果沒有帳號必須先註冊一個 (怎麼註冊應該不用講了吧)。"
],
"metadata": {
"id": "dWvaSnL98_jA"
}
},
{
"cell_type": "code",
"source": [
"from huggingface_hub import notebook_login\n",
"\n",
"notebook_login()"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 327,
"referenced_widgets": [
"4f5b1a4e61754da8851266ede056b3c6",
"b71a44a3dc4948dfb5daf0d15366af10",
"81af08976e194678baa35196491400f0",
"c9e3b3b5c89f4bdc916b4372ab9b818a",
"4aec6cb6b5dc4daa9ebd77ad1bf99a0d",
"ec9c8eb821f04491842b2b0a7ba0e4bd",
"3f2a0f4179c64d5ca36c5f590a998b97",
"e0b6ce45f30e4ad18a3cf4c8110b306b",
"bde111164051410087d6c0c5e23e0d8c",
"88d6a78f810e4482b46989c8a3f15904",
"c7cd0883d6004088b8dfffe043f76a4e",
"e6b3738fdc3d4274b0a71b409ff7c115",
"fd9238a04a784291904dd65a7c29df98",
"c7ec0643a67b4d98a5f5057bbbdf62ca"
]
},
"id": "5TbbbRSSvX8b",
"outputId": "bbd467f5-72dc-409c-bbeb-d9273de1f12b"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Login successful\n",
"Your token has been saved to /root/.huggingface/token\n",
"\u001b[1m\u001b[31mAuthenticated through git-credential store but this isn't the helper defined on your machine.\n",
"You might have to re-authenticate when pushing to the Hugging Face Hub. Run the following command in your terminal in case you want to set this credential helper as the default\n",
"\n",
"git config --global credential.helper store\u001b[0m\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"再來就只要把剛才 Fine-tune 好的模型 push 到 Hugging Face 上就行了，它會自動地幫你創立新的 repo。"
],
"metadata": {
"id": "xG9ZQNby_gJX"
}
},
{
"cell_type": "code",
"source": [
"pt_model.push_to_hub(\"my-test-model\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 261,
"referenced_widgets": [
"4c715ce23a384f448845dc1493c4a849",
"17abab63bfcb497590f44ef330e8ca2e",
"3c5ef466e72246f8bbc0ad095af6f9ee",
"eb269b3cf4fe4dff937ada08b1950767",
"1c46c9e226744a6cbc28cbdc52b6a5c0",
"a32ea7dfcaaa4122b9b46f7f45407ac5",
"ea7f091d485c42288f5727274728c997",
"5c924e59b5064bae9eb03b775cff0f0d",
"d369f843eec242cb9cd754faed4f4bc9",
"7a048bed626e4fee9b97d2167b564365",
"766baf87a4c74108babaa2ad2d52d37b"
]
},
"id": "03iWyqDcv66s",
"outputId": "c3748b8b-478d-4d81-cd99-665fa2f370da"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.7/dist-packages/huggingface_hub/utils/_deprecation.py:43: FutureWarning: Pass token='my-test-model' as keyword args. From version 0.8 passing these as positional arguments will result in an error\n",
" FutureWarning,\n",
"/usr/local/lib/python3.7/dist-packages/huggingface_hub/hf_api.py:644: FutureWarning: `create_repo` now takes `token` as an optional positional argument. Be sure to adapt your code!\n",
" FutureWarning,\n",
"Cloning https://huggingface.co/AndyChiang/my-test-model into local empty directory.\n",
"Configuration saved in my-test-model/config.json\n",
"Model weights saved in my-test-model/pytorch_model.bin\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Upload file pytorch_model.bin: 0%| | 3.34k/413M [00:00 main\n",
"\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"'https://huggingface.co/AndyChiang/my-test-model/commit/9035379b12a638eeb8e54d38cdc3f47474e03b8d'"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
}
},
"metadata": {},
"execution_count": 25
}
]
},
{
"cell_type": "markdown",
"source": [
"雖然說 Transformers 可以自動轉換 Pytorch 和 TensorFlow 的模型，但會比較花時間。所以為了使用者方便，基本上建議你將兩種模型都 push 上去比較好。"
],
"metadata": {
"id": "Fql0XckdAQJg"
}
},
{
"cell_type": "code",
"source": [
"tf_model.push_to_hub(\"my-test-model\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 154,
"referenced_widgets": [
"98b846347ef246bab2bb8a208fcb7287",
"399c274cb7274a2bb00f264d916a9c28",
"a45338072a1143119d59ea535b0910de",
"c274eb8c30f84bb69733fcd150e9bcc9",
"efd3c382afc24d2fb8e0b15d6e3b67a2",
"51065558b9ea4868b1a63924ef6ad6a1",
"d5aaf4dc530f42cd9df17b56a6e5d17c",
"68dae8899a504c8b8df9260374f77391",
"c39561accbc748beb17800d66212994f",
"0d48ffc914e34b0da29445d2bd3acc25",
"a8d9585f598d4b00a5089c202de1cf27"
]
},
"id": "FjxjyncmwpEz",
"outputId": "7ba5febd-2f5b-4460-9e0d-718576299419"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"Configuration saved in my-test-model/config.json\n",
"Model weights saved in my-test-model/tf_model.h5\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"Upload file tf_model.h5: 0%| | 3.34k/413M [00:00 main\n",
"\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"'https://huggingface.co/AndyChiang/my-test-model/commit/e6392a97d572dd50121bb398803a008dc230bb60'"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
}
},
"metadata": {},
"execution_count": 27
}
]
},
{
"cell_type": "markdown",
"source": [
"也把剛才用的 tokenizer 給 push 上去。"
],
"metadata": {
"id": "yK5RtjdLBN7b"
}
},
{
"cell_type": "code",
"source": [
"tokenizer.push_to_hub(\"my-test-model\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 122
},
"id": "xXVf2GZAwoVY",
"outputId": "3bc5c7dd-0e76-4457-f7d8-82d33ca80664"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"tokenizer config file saved in my-test-model/tokenizer_config.json\n",
"Special tokens file saved in my-test-model/special_tokens_map.json\n",
"To https://huggingface.co/AndyChiang/my-test-model\n",
" 9035379..9c4ac9f main -> main\n",
"\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"'https://huggingface.co/AndyChiang/my-test-model/commit/9c4ac9fee5b0a98c227c918bef6280cc1d695c1a'"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
}
},
"metadata": {},
"execution_count": 26
}
]
},
{
"cell_type": "markdown",
"source": [
"以後就可以下載自己的模型來用啦! 名稱是 `