dpfried commited on
Commit
4b854d4
1 Parent(s): 00e3bf6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -3
README.md CHANGED
@@ -1,3 +1,34 @@
1
- ---
2
- license: cc-by-nc-4.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # InCoder 1B
2
+
3
+ A 1B parameter decoder-only Transformer model trained on code using a causal-masked objective, which allows inserting/infilling code as well as standard left-to-right generation.
4
+
5
+ The model was trained on public open-source repositories with a permissive, non-copyleft, license (Apache 2.0, MIT, BSD-2 or BSD-3) from GitHub and GitLab, as well as StackOverflow. Repositories were primarily on Python and JavaScript, but also include code from 28 languages, as well as StackOverflow.
6
+
7
+ For more information, see our:
8
+
9
+ - [Demo](https://huggingface.co/spaces/facebook/incoder-demo)
10
+ - [Project site](https://sites.google.com/view/incoder-code-models)
11
+ - [Examples](https://sites.google.com/view/incoder-code-models/home/examples)
12
+ - [Paper](https://github.com/dpfried/incoder/blob/main/paper/InCoder.pdf)
13
+
14
+ A larger, 6B, parameter model is also available at [facebook/incoder-6B](https://huggingface.co/facebook/incoder-6B).
15
+
16
+ ## Requirements
17
+
18
+ `pytorch`, `tokenizers`, and `transformers`. Our model requires HF's tokenizers >= 0.12.1, due to changes in the pretokenizer. This version is close to release, but in the meantime you can install directly from source via pip.
19
+
20
+ ```
21
+ pip install pytorch
22
+ pip install git+https://github.com/huggingface/tokenizers
23
+ pip install git+https://github.com/huggingface/transformers
24
+ ```
25
+
26
+ ## Usage
27
+
28
+ See [https://github.com/dpfried/incoder](https://github.com/dpfried/incoder) for example code.
29
+
30
+ ## Credits
31
+
32
+ The model was developed by Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer and Mike Lewis.
33
+
34
+ Thanks to Lucile Saulnier, Leandro von Werra, Nicolas Patry, Suraj Patil, Omar Sanseviero, and others at HuggingFace for help with the model release, and to Naman Goyal and Stephen Roller for the code our demo was based on!