Spaces:
Runtime error
Runtime error
Commit
·
a8fc3e9
1
Parent(s):
9757e0e
removed model
Browse files- .DS_Store +0 -0
- models/.DS_Store +0 -0
- models/PygmalionAI_pygmalion-350m/README.md +0 -24
- models/PygmalionAI_pygmalion-350m/config.json +0 -28
- models/PygmalionAI_pygmalion-350m/huggingface-metadata.txt +0 -5
- models/PygmalionAI_pygmalion-350m/merges.txt +0 -0
- models/PygmalionAI_pygmalion-350m/pytorch_model.bin +0 -3
- models/PygmalionAI_pygmalion-350m/special_tokens_map.json +0 -1
- models/PygmalionAI_pygmalion-350m/tokenizer_config.json +0 -1
- models/PygmalionAI_pygmalion-350m/vocab.json +0 -0
.DS_Store
CHANGED
Binary files a/.DS_Store and b/.DS_Store differ
|
|
models/.DS_Store
ADDED
Binary file (6.15 kB). View file
|
|
models/PygmalionAI_pygmalion-350m/README.md
DELETED
@@ -1,24 +0,0 @@
|
|
1 |
-
---
|
2 |
-
language:
|
3 |
-
- en
|
4 |
-
thumbnail:
|
5 |
-
tags:
|
6 |
-
- convAI
|
7 |
-
- conversational
|
8 |
-
inference: false
|
9 |
-
---
|
10 |
-
# pygmalion-350m
|
11 |
-
|
12 |
-
# Model description
|
13 |
-
|
14 |
-
This is a proof-of-concept fine-tune of Facebook's OPT-350M model optimized for dialogue, to be used as a stepping stone to higher parameter models.
|
15 |
-
|
16 |
-
**Disclaimer:** NSFW data was included in the fine-tuning of this model. Although SFW inputs will usually result in SFW outputs, you are advised to **chat at your own risk. This model is not suitable for use by minors.**
|
17 |
-
|
18 |
-
# Fine-tuning process
|
19 |
-
|
20 |
-
This model was much easier than expected to create.
|
21 |
-
|
22 |
-
We used the [ColossalAI](https://www.colossalai.org/) library to fine-tune the [OPT-350M](https://huggingface.co/facebook/opt-350m) model originally trained by Facebook on The Pile. Though our initial dataset was sets of dialogue gathered from various sources totaling about 50 MB in size, early training runs revealed that the model converged after only 7% of the dataset was passed through. To alleviate this, we massively reduced the size of the dataset to only 273 KB.
|
23 |
-
|
24 |
-
ColossalAI's magic allowed for something incredible: this entire model was fine-tuned on a singular GPU with only 6 GB ***(!)*** of VRAM. Fine-tuning took less than an hour to complete.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
models/PygmalionAI_pygmalion-350m/config.json
DELETED
@@ -1,28 +0,0 @@
|
|
1 |
-
{
|
2 |
-
"_name_or_path": "opt-350m",
|
3 |
-
"activation_dropout": 0.0,
|
4 |
-
"activation_function": "relu",
|
5 |
-
"architectures": [
|
6 |
-
"OPTForCausalLM"
|
7 |
-
],
|
8 |
-
"attention_dropout": 0.0,
|
9 |
-
"bos_token_id": 2,
|
10 |
-
"do_layer_norm_before": false,
|
11 |
-
"dropout": 0.1,
|
12 |
-
"eos_token_id": 2,
|
13 |
-
"ffn_dim": 4096,
|
14 |
-
"hidden_size": 1024,
|
15 |
-
"init_std": 0.02,
|
16 |
-
"layerdrop": 0.0,
|
17 |
-
"max_position_embeddings": 2048,
|
18 |
-
"model_type": "opt",
|
19 |
-
"num_attention_heads": 16,
|
20 |
-
"num_hidden_layers": 24,
|
21 |
-
"pad_token_id": 1,
|
22 |
-
"prefix": "</s>",
|
23 |
-
"torch_dtype": "float16",
|
24 |
-
"transformers_version": "4.20.0.dev0",
|
25 |
-
"use_cache": true,
|
26 |
-
"vocab_size": 50272,
|
27 |
-
"word_embed_proj_dim": 512
|
28 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
models/PygmalionAI_pygmalion-350m/huggingface-metadata.txt
DELETED
@@ -1,5 +0,0 @@
|
|
1 |
-
url: https://huggingface.co/PygmalionAI/pygmalion-350m
|
2 |
-
branch: main
|
3 |
-
download date: 2023-07-15 12:39:01
|
4 |
-
sha256sum:
|
5 |
-
356aa4ab61193d13e3e7a097bb5f2c025dc2536d5f127154889202ba3c735ae2 pytorch_model.bin
|
|
|
|
|
|
|
|
|
|
|
|
models/PygmalionAI_pygmalion-350m/merges.txt
DELETED
The diff for this file is too large to render.
See raw diff
|
|
models/PygmalionAI_pygmalion-350m/pytorch_model.bin
DELETED
@@ -1,3 +0,0 @@
|
|
1 |
-
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:356aa4ab61193d13e3e7a097bb5f2c025dc2536d5f127154889202ba3c735ae2
|
3 |
-
size 1324917213
|
|
|
|
|
|
|
|
models/PygmalionAI_pygmalion-350m/special_tokens_map.json
DELETED
@@ -1 +0,0 @@
|
|
1 |
-
{"bos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "eos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "unk_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}, "pad_token": {"content": "<pad>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true}}
|
|
|
|
models/PygmalionAI_pygmalion-350m/tokenizer_config.json
DELETED
@@ -1 +0,0 @@
|
|
1 |
-
{"errors": "replace", "unk_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "bos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "eos_token": {"content": "</s>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "pad_token": {"content": "<pad>", "single_word": false, "lstrip": false, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "add_prefix_space": false, "add_bos_token": true, "special_tokens_map_file": null, "name_or_path": "patrickvonplaten/opt-30b"}
|
|
|
|
models/PygmalionAI_pygmalion-350m/vocab.json
DELETED
The diff for this file is too large to render.
See raw diff
|
|