Spaces:
Runtime error
Runtime error
File size: 12,568 Bytes
94ece44 bbdf699 94ece44 bbdf699 d803879 94ece44 e200f8c 94ece44 bbdf699 d803879 94ece44 a1771a7 94ece44 e200f8c 94ece44 90c428d 94ece44 b676b21 a1771a7 b676b21 94ece44 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 |
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"private_outputs": true,
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"gpuClass": "standard",
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/zetavg/LLaMA-LoRA-Tuner/blob/main/LLaMA_LoRA.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"# 🦙🎛️ LLaMA-LoRA Tuner\n",
"\n",
"TL;DR: **Runtime > Run All** (`⌘/Ctrl+F9`). Takes about 5 minutes to start. You will be promped to authorize Google Drive access."
],
"metadata": {
"id": "bb4nzBvLfZUj"
}
},
{
"cell_type": "code",
"source": [
"#@markdown To prevent Colab from disconnecting you, here is a music player that will loop infinitely (it's silent):\n",
"%%html\n",
"<audio src=\"https://github.com/anars/blank-audio/raw/master/1-hour-of-silence.mp3\" autoplay muted loop controls />"
],
"metadata": {
"id": "DwarOgXbG77C",
"cellView": "form"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# @title A small workaround { display-mode: \"form\" }\n",
"# @markdown Don't panic if you see an error here. Just click the `RESTART RUNTIME` button in the output below, then Run All again.\n",
"# @markdown The error will disappear on the next run.\n",
"!pip install Pillow==9.3.0\n",
"import PIL\n",
"major, minor = map(float, PIL.__version__.split(\".\")[:2])\n",
"version_float = major + minor / 10**len(str(minor))\n",
"print(version_float)\n",
"if version_float < 9.003:\n",
" raise Exception(\"Restart the runtime by clicking the 'RESTART RUNTIME' button above (or Runtime > Restart Runtime).\")"
],
"metadata": {
"id": "XcJ4WO3KhOX1"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# Config\n",
"\n",
"Some configurations to run this notebook. "
],
"metadata": {
"id": "5uS5jJ8063f_"
}
},
{
"cell_type": "code",
"source": [
"# @title Git/Project { display-mode: \"form\", run: \"auto\" }\n",
"# @markdown Project settings.\n",
"\n",
"# @markdown The URL of the LLaMA-LoRA-Tuner project<br> (default: `https://github.com/zetavg/LLaMA-LoRA-Tuner.git`):\n",
"llama_lora_project_url = \"https://github.com/zetavg/LLaMA-LoRA-Tuner.git\" # @param {type:\"string\"}\n",
"# @markdown The branch to use for LLaMA-LoRA-Tuner project:\n",
"llama_lora_project_branch = \"main\" # @param {type:\"string\"}\n",
"\n",
"# # @markdown Forces the local directory to be updated by the remote branch:\n",
"# force_update = True # @param {type:\"boolean\"}\n"
],
"metadata": {
"id": "v3ZCPW0JBCcH"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# @title Google Drive { display-mode: \"form\", run: \"auto\" }\n",
"# @markdown Google Drive will be used to store data that is used or outputed by this notebook. you will be prompted to authorize access while running this notebook. \n",
"#\n",
"# @markdown Currently, it's not possible to access only a specific folder of Google Drive, we have no choice to mount the entire Google Drive, but will only access given folder.\n",
"#\n",
"# @markdown You can customize the location of the stored data here.\n",
"\n",
"# @markdown The folder in Google Drive where Colab Notebook data are stored<br /> **(WARNING: The content of this folder will be modified by this notebook)**:\n",
"google_drive_folder = \"Colab Data/LLaMA-LoRA Tuner\" # @param {type:\"string\"}\n",
"# google_drive_colab_data_folder = \"Colab Notebooks/Notebook Data\"\n",
"\n",
"# Where Google Drive will be mounted in the Colab runtime.\n",
"google_drive_mount_path = \"/content/drive\"\n",
"\n",
"from requests import get\n",
"from socket import gethostname, gethostbyname\n",
"host_ip = gethostbyname(gethostname())\n",
"colab_notebook_filename = get(f\"http://{host_ip}:9000/api/sessions\").json()[0][\"name\"]\n",
"\n",
"# def remove_ipynb_extension(filename: str) -> str:\n",
"# extension = \".ipynb\"\n",
"# if filename.endswith(extension):\n",
"# return filename[:-len(extension)]\n",
"# return filename\n",
"\n",
"# colab_notebook_name = remove_ipynb_extension(colab_notebook_filename)\n",
"\n",
"from google.colab import drive\n",
"drive.mount(google_drive_mount_path)\n",
"\n",
"# google_drive_data_directory_relative_path = f\"{google_drive_colab_data_folder}/{colab_notebook_name}\"\n",
"google_drive_data_directory_relative_path = google_drive_folder\n",
"google_drive_data_directory_path = f\"{google_drive_mount_path}/My Drive/{google_drive_data_directory_relative_path}\"\n",
"!mkdir -p \"{google_drive_data_directory_path}\"\n",
"!ln -nsf \"{google_drive_data_directory_path}\" ./data\n",
"!touch \"data/This folder is used by the Colab notebook \\\"{colab_notebook_filename}\\\".txt\"\n",
"!echo \"Data will be stored in Google Drive folder: \\\"{google_drive_data_directory_relative_path}\\\", which is mounted under \\\"{google_drive_data_directory_path}\\\"\"\n"
],
"metadata": {
"id": "iZmRtUY68U5f"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# @title Model/Training Settings { display-mode: \"form\", run: \"auto\" }\n",
"\n",
"base_model = \"decapoda-research/llama-7b-hf\" # @param {type:\"string\"}"
],
"metadata": {
"id": "Ep3Qhwj0Bzwf"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# Runtime Info\n",
"\n",
"Print out some information about the Colab runtime. Code from https://colab.research.google.com/notebooks/pro.ipynb."
],
"metadata": {
"id": "Qg8tzrgb6ya_"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AHbRt8sK6YWy"
},
"outputs": [],
"source": [
"# @title GPU Info { display-mode: \"form\" }\n",
"#\n",
"# @markdown By running this cell, you can see what GPU you've been assigned.\n",
"#\n",
"# @markdown If the execution result of running the code cell below is \"Not connected to a GPU\", you can change the runtime by going to `Runtime / Change runtime type` in the menu to enable a GPU accelerator, and then re-execute the code cell.\n",
"\n",
"\n",
"gpu_info = !nvidia-smi\n",
"gpu_info = '\\n'.join(gpu_info)\n",
"if gpu_info.find('failed') >= 0:\n",
" print('Not connected to a GPU')\n",
"else:\n",
" print(gpu_info)"
]
},
{
"cell_type": "code",
"source": [
"# @title RAM Info { display-mode: \"form\" }\n",
"#\n",
"# @markdown By running this code cell, You can see how much memory you have available.\n",
"#\n",
"# @markdown Normally, a high-RAM runtime is not needed, but if you need more RAM, you can enable a high-RAM runtime via `Runtime / Change runtime type` in the menu. Then select High-RAM in the Runtime shape dropdown. After, re-execute the code cell.\n",
"\n",
"from psutil import virtual_memory\n",
"ram_gb = virtual_memory().total / 1e9\n",
"print('Your runtime has {:.1f} gigabytes of available RAM\\n'.format(ram_gb))\n",
"\n",
"if ram_gb < 20:\n",
" print('Not using a high-RAM runtime')\n",
"else:\n",
" print('You are using a high-RAM runtime!')"
],
"metadata": {
"id": "rGM5MYjR7yeS"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# Prepare the Project\n",
"\n",
"Clone the project and install dependencies (takes about 5m on the first run)."
],
"metadata": {
"id": "8vSPMNtNAqOo"
}
},
{
"cell_type": "code",
"source": [
"![ ! -d llama_lora ] && git clone -b {llama_lora_project_branch} --filter=tree:0 {llama_lora_project_url} llama_lora\n",
"!cd llama_lora && git add --all && git stash && git fetch origin {llama_lora_project_branch} && git checkout {llama_lora_project_branch} && git reset origin/{llama_lora_project_branch} --hard\n",
"![ ! -f llama-lora-requirements-installed ] && cd llama_lora && pip install -r requirements.lock.txt && touch ../llama-lora-requirements-installed"
],
"metadata": {
"id": "JGYz2VDoAzC8"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"# Launch"
],
"metadata": {
"id": "o90p1eYQimyr"
}
},
{
"cell_type": "code",
"source": [
"# The following command will launch the app in one shot, but we will not do this here.\n",
"# Instead, we will import and run Python code from the runtime, so that each part\n",
"# can be reloaded easily in the Colab notebook and provide readable outputs.\n",
"# It also resolves the GPU out-of-memory issue on training.\n",
"# !python llama_lora/app.py --base_model='{base_model}' --data_dir='./data' --share"
],
"metadata": {
"id": "HYVjcvwXimB6"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"# @title Load the App (set config, prepare data dir, load base bodel)\n",
"\n",
"# @markdown For a LLaMA-7B model, it will take about ~5m to load for the first execution,\n",
"# @markdown including download. Subsequent executions will take about 2m to load.\n",
"\n",
"# Set Configs\n",
"from llama_lora.llama_lora.globals import Global\n",
"Global.default_base_model_name = base_model\n",
"data_dir_realpath = !realpath ./data\n",
"Global.data_dir = data_dir_realpath[0]\n",
"Global.load_8bit = True\n",
"\n",
"# Prepare Data Dir\n",
"import os\n",
"from llama_lora.llama_lora.utils.data import init_data_dir\n",
"init_data_dir()\n",
"\n",
"# Load the Base Model\n",
"from llama_lora.llama_lora.models import prepare_base_model\n",
"prepare_base_model()"
],
"metadata": {
"id": "Yf6g248ylteP"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Start Gradio UI 🚀 (open the app from the URL output-ed here)\n",
"\n",
"You will see `Running on public URL: https://...` in the output of the following code cell, click on it to open the Gradio UI."
],
"metadata": {
"id": "K-hCouBClKAe"
}
},
{
"cell_type": "code",
"source": [
"import gradio as gr\n",
"from llama_lora.llama_lora.ui.main_page import main_page, get_page_title, main_page_custom_css\n",
"\n",
"with gr.Blocks(title=get_page_title(), css=main_page_custom_css()) as app:\n",
" main_page()\n",
"\n",
"app.queue(concurrency_count=1).launch(share=True, debug=True, server_name=\"127.0.0.1\")"
],
"metadata": {
"id": "iLygNTcHk0N8"
},
"execution_count": null,
"outputs": []
}
]
}
|