File size: 18,151 Bytes
de0f783 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 |
Collecting vllm
Using cached vllm-0.5.3.post1-cp38-abi3-manylinux1_x86_64.whl.metadata (1.8 kB)
Requirement already satisfied: cmake>=3.21 in /opt/conda/lib/python3.10/site-packages (from vllm) (3.30.1)
Requirement already satisfied: ninja in /opt/conda/lib/python3.10/site-packages (from vllm) (1.11.1.1)
Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from vllm) (5.9.0)
Requirement already satisfied: sentencepiece in /opt/conda/lib/python3.10/site-packages (from vllm) (0.1.99)
Requirement already satisfied: numpy<2.0.0 in /opt/conda/lib/python3.10/site-packages (from vllm) (1.26.3)
Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from vllm) (2.32.3)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.10/site-packages (from vllm) (4.66.4)
Requirement already satisfied: py-cpuinfo in /opt/conda/lib/python3.10/site-packages (from vllm) (9.0.0)
Requirement already satisfied: transformers>=4.42.4 in /opt/conda/lib/python3.10/site-packages (from vllm) (4.43.2)
Requirement already satisfied: tokenizers>=0.19.1 in /opt/conda/lib/python3.10/site-packages (from vllm) (0.19.1)
Collecting fastapi (from vllm)
Using cached fastapi-0.111.1-py3-none-any.whl.metadata (26 kB)
Requirement already satisfied: aiohttp in /opt/conda/lib/python3.10/site-packages (from vllm) (3.9.5)
Collecting openai (from vllm)
Using cached openai-1.37.1-py3-none-any.whl.metadata (22 kB)
Requirement already satisfied: uvicorn[standard] in /opt/conda/lib/python3.10/site-packages (from vllm) (0.30.3)
Collecting pydantic>=2.0 (from vllm)
Using cached pydantic-2.8.2-py3-none-any.whl.metadata (125 kB)
Requirement already satisfied: pillow in /opt/conda/lib/python3.10/site-packages (from vllm) (10.2.0)
Requirement already satisfied: prometheus-client>=0.18.0 in /opt/conda/lib/python3.10/site-packages (from vllm) (0.20.0)
Collecting prometheus-fastapi-instrumentator>=7.0.0 (from vllm)
Using cached prometheus_fastapi_instrumentator-7.0.0-py3-none-any.whl.metadata (13 kB)
Collecting tiktoken>=0.6.0 (from vllm)
Using cached tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting lm-format-enforcer==0.10.3 (from vllm)
Using cached lm_format_enforcer-0.10.3-py3-none-any.whl.metadata (16 kB)
Collecting outlines<0.1,>=0.0.43 (from vllm)
Using cached outlines-0.0.46-py3-none-any.whl.metadata (15 kB)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.10/site-packages (from vllm) (4.9.0)
Requirement already satisfied: filelock>=3.10.4 in /opt/conda/lib/python3.10/site-packages (from vllm) (3.13.1)
Requirement already satisfied: pyzmq in /opt/conda/lib/python3.10/site-packages (from vllm) (26.0.3)
Collecting ray>=2.9 (from vllm)
Using cached ray-2.33.0-cp310-cp310-manylinux2014_x86_64.whl.metadata (13 kB)
Requirement already satisfied: nvidia-ml-py in /opt/conda/lib/python3.10/site-packages (from vllm) (12.555.43)
Collecting torch==2.3.1 (from vllm)
Using cached torch-2.3.1-cp310-cp310-manylinux1_x86_64.whl.metadata (26 kB)
Collecting torchvision==0.18.1 (from vllm)
Using cached torchvision-0.18.1-cp310-cp310-manylinux1_x86_64.whl.metadata (6.6 kB)
Collecting xformers==0.0.27 (from vllm)
Using cached xformers-0.0.27-cp310-cp310-manylinux2014_x86_64.whl.metadata (1.0 kB)
Collecting vllm-flash-attn==2.5.9.post1 (from vllm)
Using cached vllm_flash_attn-2.5.9.post1-cp310-cp310-manylinux1_x86_64.whl.metadata (482 bytes)
Requirement already satisfied: interegular>=0.3.2 in /opt/conda/lib/python3.10/site-packages (from lm-format-enforcer==0.10.3->vllm) (0.3.3)
Requirement already satisfied: packaging in /opt/conda/lib/python3.10/site-packages (from lm-format-enforcer==0.10.3->vllm) (23.1)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from lm-format-enforcer==0.10.3->vllm) (6.0.1)
Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (1.12)
Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (3.1)
Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (3.1.3)
Requirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (2023.10.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.105)
Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.3.1->vllm)
Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (10.3.2.106)
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.3.1->vllm)
Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.3.1->vllm)
Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)
Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (2.20.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (12.1.105)
Requirement already satisfied: triton==2.3.1 in /opt/conda/lib/python3.10/site-packages (from torch==2.3.1->vllm) (2.3.1)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /opt/conda/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch==2.3.1->vllm) (12.5.82)
Requirement already satisfied: lark in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (1.1.9)
Requirement already satisfied: nest-asyncio in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (1.6.0)
Collecting cloudpickle (from outlines<0.1,>=0.0.43->vllm)
Using cached cloudpickle-3.0.0-py3-none-any.whl.metadata (7.0 kB)
Requirement already satisfied: diskcache in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (5.6.3)
Requirement already satisfied: numba in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (0.60.0)
Requirement already satisfied: referencing in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (0.30.2)
Requirement already satisfied: jsonschema in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (4.19.2)
Requirement already satisfied: datasets in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (2.20.0)
Requirement already satisfied: pycountry in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (24.6.1)
Requirement already satisfied: pyairports in /opt/conda/lib/python3.10/site-packages (from outlines<0.1,>=0.0.43->vllm) (2.1.1)
Collecting starlette<1.0.0,>=0.30.0 (from prometheus-fastapi-instrumentator>=7.0.0->vllm)
Using cached starlette-0.38.1-py3-none-any.whl.metadata (5.9 kB)
Collecting annotated-types>=0.4.0 (from pydantic>=2.0->vllm)
Using cached annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)
Requirement already satisfied: pydantic-core==2.20.1 in /opt/conda/lib/python3.10/site-packages (from pydantic>=2.0->vllm) (2.20.1)
Requirement already satisfied: click>=7.0 in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (8.1.7)
Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (1.0.8)
Requirement already satisfied: protobuf!=3.19.5,>=3.15.3 in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (5.27.2)
Requirement already satisfied: aiosignal in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (1.3.1)
Requirement already satisfied: frozenlist in /opt/conda/lib/python3.10/site-packages (from ray>=2.9->vllm) (1.4.1)
Requirement already satisfied: regex>=2022.1.18 in /opt/conda/lib/python3.10/site-packages (from tiktoken>=0.6.0->vllm) (2024.7.24)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->vllm) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->vllm) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->vllm) (2.1.0)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->vllm) (2024.2.2)
Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /opt/conda/lib/python3.10/site-packages (from tokenizers>=0.19.1->vllm) (0.24.2)
Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers>=4.42.4->vllm) (0.4.3)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->vllm) (23.1.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp->vllm) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->vllm) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp->vllm) (4.0.3)
Collecting starlette<1.0.0,>=0.30.0 (from prometheus-fastapi-instrumentator>=7.0.0->vllm)
Using cached starlette-0.37.2-py3-none-any.whl.metadata (5.9 kB)
Collecting fastapi-cli>=0.0.2 (from fastapi->vllm)
Using cached fastapi_cli-0.0.4-py3-none-any.whl.metadata (7.0 kB)
Requirement already satisfied: httpx>=0.23.0 in /opt/conda/lib/python3.10/site-packages (from fastapi->vllm) (0.27.0)
Requirement already satisfied: python-multipart>=0.0.7 in /opt/conda/lib/python3.10/site-packages (from fastapi->vllm) (0.0.9)
Requirement already satisfied: email_validator>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from fastapi->vllm) (2.2.0)
Requirement already satisfied: h11>=0.8 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (0.14.0)
Requirement already satisfied: httptools>=0.5.0 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (0.6.1)
Requirement already satisfied: python-dotenv>=0.13 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (1.0.1)
Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (0.19.0)
Collecting watchfiles>=0.13 (from uvicorn[standard]->vllm)
Using cached watchfiles-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.9 kB)
Requirement already satisfied: websockets>=10.4 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]->vllm) (12.0)
Requirement already satisfied: anyio<5,>=3.5.0 in /opt/conda/lib/python3.10/site-packages (from openai->vllm) (4.4.0)
Requirement already satisfied: distro<2,>=1.7.0 in /opt/conda/lib/python3.10/site-packages (from openai->vllm) (1.8.0)
Requirement already satisfied: sniffio in /opt/conda/lib/python3.10/site-packages (from openai->vllm) (1.3.1)
Requirement already satisfied: exceptiongroup>=1.0.2 in /opt/conda/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai->vllm) (1.2.0)
Requirement already satisfied: dnspython>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from email_validator>=2.0.0->fastapi->vllm) (2.6.1)
Collecting typer>=0.12.3 (from fastapi-cli>=0.0.2->fastapi->vllm)
Using cached typer-0.12.3-py3-none-any.whl.metadata (15 kB)
Requirement already satisfied: httpcore==1.* in /opt/conda/lib/python3.10/site-packages (from httpx>=0.23.0->fastapi->vllm) (1.0.5)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch==2.3.1->vllm) (2.1.3)
Requirement already satisfied: pyarrow>=15.0.0 in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (17.0.0)
Requirement already satisfied: pyarrow-hotfix in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (0.6)
Requirement already satisfied: dill<0.3.9,>=0.3.0 in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (0.3.7)
Requirement already satisfied: pandas in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (2.2.2)
Requirement already satisfied: xxhash in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (3.4.1)
Requirement already satisfied: multiprocess in /opt/conda/lib/python3.10/site-packages (from datasets->outlines<0.1,>=0.0.43->vllm) (0.70.15)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/conda/lib/python3.10/site-packages (from jsonschema->outlines<0.1,>=0.0.43->vllm) (2023.7.1)
Requirement already satisfied: rpds-py>=0.7.1 in /opt/conda/lib/python3.10/site-packages (from jsonschema->outlines<0.1,>=0.0.43->vllm) (0.10.6)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /opt/conda/lib/python3.10/site-packages (from numba->outlines<0.1,>=0.0.43->vllm) (0.43.0)
Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch==2.3.1->vllm) (1.3.0)
Requirement already satisfied: shellingham>=1.3.0 in /opt/conda/lib/python3.10/site-packages (from typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm) (1.5.4)
Collecting rich>=10.11.0 (from typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm)
Using cached rich-13.7.1-py3-none-any.whl.metadata (18 kB)
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.10/site-packages (from pandas->datasets->outlines<0.1,>=0.0.43->vllm) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.10/site-packages (from pandas->datasets->outlines<0.1,>=0.0.43->vllm) (2023.3.post1)
Requirement already satisfied: tzdata>=2022.7 in /opt/conda/lib/python3.10/site-packages (from pandas->datasets->outlines<0.1,>=0.0.43->vllm) (2024.1)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->datasets->outlines<0.1,>=0.0.43->vllm) (1.16.0)
Collecting markdown-it-py>=2.2.0 (from rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm)
Using cached markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /opt/conda/lib/python3.10/site-packages (from rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm) (2.15.1)
Requirement already satisfied: mdurl~=0.1 in /opt/conda/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich>=10.11.0->typer>=0.12.3->fastapi-cli>=0.0.2->fastapi->vllm) (0.1.2)
Using cached vllm-0.5.3.post1-cp38-abi3-manylinux1_x86_64.whl (158.3 MB)
Using cached lm_format_enforcer-0.10.3-py3-none-any.whl (43 kB)
Using cached torch-2.3.1-cp310-cp310-manylinux1_x86_64.whl (779.1 MB)
Using cached torchvision-0.18.1-cp310-cp310-manylinux1_x86_64.whl (7.0 MB)
Using cached vllm_flash_attn-2.5.9.post1-cp310-cp310-manylinux1_x86_64.whl (37.1 MB)
Using cached xformers-0.0.27-cp310-cp310-manylinux2014_x86_64.whl (164.1 MB)
Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
Using cached outlines-0.0.46-py3-none-any.whl (101 kB)
Using cached prometheus_fastapi_instrumentator-7.0.0-py3-none-any.whl (19 kB)
Using cached pydantic-2.8.2-py3-none-any.whl (423 kB)
Using cached ray-2.33.0-cp310-cp310-manylinux2014_x86_64.whl (64.8 MB)
Using cached tiktoken-0.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Using cached fastapi-0.111.1-py3-none-any.whl (92 kB)
Using cached openai-1.37.1-py3-none-any.whl (337 kB)
Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB)
Using cached fastapi_cli-0.0.4-py3-none-any.whl (9.5 kB)
Using cached starlette-0.37.2-py3-none-any.whl (71 kB)
Using cached watchfiles-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Using cached cloudpickle-3.0.0-py3-none-any.whl (20 kB)
Using cached typer-0.12.3-py3-none-any.whl (47 kB)
Using cached rich-13.7.1-py3-none-any.whl (240 kB)
Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Installing collected packages: nvidia-cusparse-cu12, nvidia-cudnn-cu12, markdown-it-py, cloudpickle, annotated-types, watchfiles, tiktoken, starlette, rich, pydantic, nvidia-cusolver-cu12, typer, torch, prometheus-fastapi-instrumentator, openai, lm-format-enforcer, xformers, vllm-flash-attn, torchvision, ray, fastapi-cli, outlines, fastapi, vllm
Attempting uninstall: torch
Found existing installation: torch 2.2.1
Can't uninstall 'torch'. No files were found to uninstall.
Attempting uninstall: torchvision
Found existing installation: torchvision 0.17.1
Can't uninstall 'torchvision'. No files were found to uninstall.
Successfully installed annotated-types-0.7.0 cloudpickle-3.0.0 fastapi-0.111.1 fastapi-cli-0.0.4 lm-format-enforcer-0.10.3 markdown-it-py-3.0.0 nvidia-cudnn-cu12-8.9.2.26 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 openai-1.37.1 outlines-0.0.46 prometheus-fastapi-instrumentator-7.0.0 pydantic-2.8.2 ray-2.33.0 rich-13.7.1 starlette-0.37.2 tiktoken-0.7.0 torch-2.3.1 torchvision-0.18.1 typer-0.12.3 vllm-0.5.3.post1 vllm-flash-attn-2.5.9.post1 watchfiles-0.22.0 xformers-0.0.27
|