joaoalvarenga
commited on
Commit
·
1a3d1db
1
Parent(s):
8b76f74
Update README.md
Browse files
README.md
CHANGED
@@ -52,7 +52,7 @@ pipeline_tag: text-generation
|
|
52 |
|
53 |
Heavily inspired by [Hivemind's GPT-J-6B with 8-bit weights](https://huggingface.co/hivemind/gpt-j-6B-8bit), this is a version of [bigscience/bloom](https://huggingface.co/bigscience/bloom) a ~176 billion parameters language model that you run and fine-tune with less memory.
|
54 |
|
55 |
-
Here, we also apply [LoRA (Low Rank
|
56 |
|
57 |
Our main goal is to generate a model compressed enough to be deployed in a traditional Kubernetes cluster.
|
58 |
|
|
|
52 |
|
53 |
Heavily inspired by [Hivemind's GPT-J-6B with 8-bit weights](https://huggingface.co/hivemind/gpt-j-6B-8bit), this is a version of [bigscience/bloom](https://huggingface.co/bigscience/bloom) a ~176 billion parameters language model that you run and fine-tune with less memory.
|
54 |
|
55 |
+
Here, we also apply [LoRA (Low Rank Adaptation)](https://arxiv.org/abs/2106.09685) to reduce model size. The original version takes \~353GB memory, this version takes **\~180GB**.
|
56 |
|
57 |
Our main goal is to generate a model compressed enough to be deployed in a traditional Kubernetes cluster.
|
58 |
|