text-generation-webui / docs /09 - Docker.md
GitNeu256
Load
cfbfaf1
|
raw
history blame
7.04 kB

Docker Compose is a way of installing and launching the web UI in an isolated Ubuntu image using only a few commands.

Installing Docker Compose

In order to create the image as described in the main README, you must have Docker Compose installed (2.17 or higher is recommended):

~$ docker compose version
Docker Compose version v2.21.0

The installation instructions for various Linux distributions can be found here:

https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository

Launching the image

Use these commands to launch the image:

cd text-generation-webui
ln -s docker/{nvidia/Dockerfile,docker-compose.yml,.dockerignore} .
cp docker/.env.example .env
# Edit .env and set TORCH_CUDA_ARCH_LIST based on your GPU model
docker compose up --build

More detailed installation instructions

By @loeken.

Ubuntu 22.04

0. youtube video

A video walking you through the setup can be found here:

oobabooga text-generation-webui setup in docker on ubuntu 22.04

1. update the drivers

in the the β€œsoftware updater” update drivers to the last version of the prop driver.

2. reboot

to switch using to new driver

3. install docker

sudo apt update
sudo apt-get install curl
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-compose -y
sudo usermod -aG docker $USER
newgrp docker

4. docker & container toolkit

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/stable/ubuntu22.04/amd64 /" | \
sudo tee /etc/apt/sources.list.d/nvidia.list > /dev/null 
sudo apt update
sudo apt install nvidia-docker2 nvidia-container-runtime -y
sudo systemctl restart docker

5. clone the repo

git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui

6. prepare models

download and place the models inside the models folder. tested with:

4bit https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617 https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105

8bit: https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789

7. prepare .env file

edit .env values to your needs.

cp .env.example .env
nano .env

8. startup docker container

docker compose up --build

Manjaro

manjaro/arch is similar to ubuntu just the dependency installation is more convenient

update the drivers

sudo mhwd -a pci nonfree 0300

reboot

reboot

docker & container toolkit

yay -S docker docker-compose buildkit gcc nvidia-docker
sudo usermod -aG docker $USER
newgrp docker
sudo systemctl restart docker # required by nvidia-container-runtime

continue with ubuntu task

continue at 5. clone the repo

Windows

0. youtube video

A video walking you through the setup can be found here: oobabooga text-generation-webui setup in docker on windows 11

1. choco package manager

install package manager (https://chocolatey.org/ )

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))

2. install drivers/dependencies

choco install nvidia-display-driver cuda git docker-desktop

3. install wsl

wsl --install

4. reboot

after reboot enter username/password in wsl

5. git clone && startup

clone the repo and edit .env values to your needs.

cd Desktop
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
COPY .env.example .env
notepad .env

6. prepare models

download and place the models inside the models folder. tested with:

4bit https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483891617 https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105

8bit: https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1484235789

7. startup

docker compose up

notes

on older ubuntus you can manually install the docker compose plugin like this:

DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/v2.17.2/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
export PATH="$HOME/.docker/cli-plugins:$PATH"

Dedicated docker repository

An external repository maintains a docker wrapper for this project as well as several pre-configured 'one-click' docker compose variants (e.g., updated branches of GPTQ). It can be found at: Atinoda/text-generation-webui-docker.