amy-hyunji-lee commited on
Commit
185dfab
1 Parent(s): dd31dc0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -1,3 +1,23 @@
1
  ---
2
  license: cc-by-nc-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
  ---
4
+ This model is a generation model trained via [semiparametric token-sequence co-supervision](https://github.com/kaistAI/Semiparametric_Token-Sequence_Co-Supervision) on top of Llama2-7B.
5
+ The embedding model which constructs the nonparametric sequence embedding spaces is in [here](https://huggingface.co/kaist-ai/cosupervision-emb_seq-Llama2_7b).
6
+ The models are trained on information-seeking datasets provided by [self-rag](https://selfrag.github.io/) with co-supervision from next token prediction (NTP) and next sequence prediction (NSP).
7
+ In the inference step, the model generates a response by retrieving relevant sequences.
8
+ See full descriptions in our paper.
9
+
10
+ ### Usage
11
+ Here, we show an easy way to quickly download our model from HuggingFace.
12
+ Make sure to install dependencies listed at requirements.txt.
13
+ To run our full inference pipeline with embedding model, please use our [code](https://github.com/kaistAI/Semiparametric_Token-Sequence_Co-Supervision).
14
+
15
+ ```
16
+ from transformers import AutoTokenizer, LlamaForCausalLM
17
+
18
+ model = LlamaForCausalLM.from_pretrained(
19
+ "kaist-ai/cosupervision-emb_seq-Llama2_7b",
20
+ load_in_8bit=True if train_config.quantization else None,
21
+ device_map="auto" if train_config.quantization else None,
22
+ )
23
+ ```