Update README.md
Browse files
README.md
CHANGED
@@ -1,4 +1,3 @@
|
|
1 |
-
---
|
2 |
pipeline_tag: text-generation
|
3 |
license: apache-2.0
|
4 |
tags:
|
@@ -83,24 +82,14 @@ Search-based technology, AutoNAC.
|
|
83 |
|
84 |
## Model Architecture
|
85 |
|
86 |
-
| Parameters | Layers | Heads | Sequence Length | GQA num_key_value_heads |
|
87 |
-
|
88 |
-
| 6B | 32 | 32 | 4096 | Variable |
|
89 |
|
90 |
|
91 |
- **Decoder layer:** Variable Grouped Query Attention
|
92 |
- **Position Embeddings:** Rotary Position Embeddings [Su et al., 2021](https://arxiv.org/abs/2104.09864)
|
93 |
|
94 |
-
## Uses
|
95 |
-
|
96 |
-
The model is intended to perform single/multiline code completion from a
|
97 |
-
context window of up to 4096k tokens. It is *not* an instruction model
|
98 |
-
and commands like \"Write a function that computes the absolute value of
|
99 |
-
an integer,\" won't yield the desired results. A more effective approach
|
100 |
-
is to frame instructions in the style of source code comments (e.g. \#
|
101 |
-
this function calculates the absolute value of an integer) or to present
|
102 |
-
a function signature and docstring, enabling the model to complete the
|
103 |
-
function's body.
|
104 |
|
105 |
### How to Use
|
106 |
|
|
|
|
|
1 |
pipeline_tag: text-generation
|
2 |
license: apache-2.0
|
3 |
tags:
|
|
|
82 |
|
83 |
## Model Architecture
|
84 |
|
85 |
+
| Parameters | Layers | Heads | Sequence Length | GQA num_key_value_heads |
|
86 |
+
|:----------|:----------|:----------|:----------|:----------|
|
87 |
+
| 6B | 32 | 32 | 4096 | Variable |
|
88 |
|
89 |
|
90 |
- **Decoder layer:** Variable Grouped Query Attention
|
91 |
- **Position Embeddings:** Rotary Position Embeddings [Su et al., 2021](https://arxiv.org/abs/2104.09864)
|
92 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
93 |
|
94 |
### How to Use
|
95 |
|