Spaces:
Runtime error
Runtime error
Hector Salvador [Fisharp]
commited on
Commit
•
da23173
1
Parent(s):
4f9e345
Use of a proper .js file for click action scripts
Browse files(instead of the python file with js scripts in variables)
Including changes to fuctions for loading files
- README.md +48 -2
- app.py +20 -23
- settings.py +2 -0
- src/utils.py +11 -7
- src/share_btn.py → static/community-btn.js +2 -2
- static/{community_icon.svg → community-icon.svg} +0 -0
- static/formats.md +0 -47
- static/{loading_icon.svg → loading-icon.svg} +0 -0
- static/styles.css +7 -1
README.md
CHANGED
@@ -21,8 +21,6 @@ This is a demo playground to generate code with the power of ⭐[StarCoder](http
|
|
21 |
|
22 |
🗣️For instruction and chatting you can chat with a prompted version of the model directly at the [HuggingFace🤗Chat💬(hf.co/chat)](https://huggingface.co/chat/?model=starcoder)
|
23 |
|
24 |
-
![StarCoder](https://huggingface.co/datasets/bigcode/admin/resolve/main/StarCoderBanner.png)
|
25 |
-
|
26 |
---
|
27 |
|
28 |
**Intended Use**: this app and its [supporting model](https://huggingface.co/bigcode/starcoder) are provided for demonstration purposes only; not to serve as a replacement for human expertise. For more details on the model's limitations in terms of factuality and biases, please refer to the source [model card](hf.co/bigcode)
|
@@ -30,3 +28,51 @@ This is a demo playground to generate code with the power of ⭐[StarCoder](http
|
|
30 |
⚠️ Any use or sharing of this demo constitutes your acceptance of the BigCode [OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) License Agreement and the use restrictions included within.
|
31 |
|
32 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
21 |
|
22 |
🗣️For instruction and chatting you can chat with a prompted version of the model directly at the [HuggingFace🤗Chat💬(hf.co/chat)](https://huggingface.co/chat/?model=starcoder)
|
23 |
|
|
|
|
|
24 |
---
|
25 |
|
26 |
**Intended Use**: this app and its [supporting model](https://huggingface.co/bigcode/starcoder) are provided for demonstration purposes only; not to serve as a replacement for human expertise. For more details on the model's limitations in terms of factuality and biases, please refer to the source [model card](hf.co/bigcode)
|
|
|
28 |
⚠️ Any use or sharing of this demo constitutes your acceptance of the BigCode [OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) License Agreement and the use restrictions included within.
|
29 |
|
30 |
---
|
31 |
+
|
32 |
+
## Model Formats
|
33 |
+
|
34 |
+
The model is pretrained on code and is formatted with special tokens in addition to the pure code data,\
|
35 |
+
such as prefixes specifying the source of the file or tokens separating code from a commit message.\
|
36 |
+
Use these templates to explore the model's capacities:
|
37 |
+
|
38 |
+
### 1. Prefixes 🏷️
|
39 |
+
|
40 |
+
For pure code files, use any combination of the following prefixes:
|
41 |
+
|
42 |
+
```xml
|
43 |
+
<reponame>REPONAME<filename>FILENAME<gh_stars>STARS\ncode<|endoftext|>
|
44 |
+
```
|
45 |
+
|
46 |
+
STARS can be one of: 0, 1-10, 10-100, 100-1000, 1000+
|
47 |
+
|
48 |
+
### 2. Commits 💾
|
49 |
+
|
50 |
+
The commits data is formatted as follows:
|
51 |
+
|
52 |
+
```xml
|
53 |
+
<commit_before>code<commit_msg>text<commit_after>code<|endoftext|>
|
54 |
+
```
|
55 |
+
|
56 |
+
### 3. Jupyter Notebooks 📓
|
57 |
+
|
58 |
+
The model is trained on Jupyter notebooks as Python scripts and structured formats like:
|
59 |
+
|
60 |
+
```xml
|
61 |
+
<start_jupyter><jupyter_text>text<jupyter_code>code<jupyter_output>output<jupyter_text>
|
62 |
+
```
|
63 |
+
|
64 |
+
### 4. Issues 🐛
|
65 |
+
|
66 |
+
We also trained on GitHub issues using the following formatting:
|
67 |
+
|
68 |
+
```xml
|
69 |
+
<issue_start><issue_comment>text<issue_comment>...<issue_closed>
|
70 |
+
```
|
71 |
+
|
72 |
+
### 5. Fill-in-the-middle 🧩
|
73 |
+
|
74 |
+
Fill in the middle requires rearranging the model inputs. The playground handles this for you - all you need is to specify where to fill:
|
75 |
+
|
76 |
+
```xml
|
77 |
+
code before<FILL_HERE>code after
|
78 |
+
```
|
app.py
CHANGED
@@ -8,8 +8,6 @@ from gradio.themes.utils import sizes
|
|
8 |
from text_generation import Client
|
9 |
from src.request import StarCoderRequest, StarCoderRequestConfig
|
10 |
|
11 |
-
# todo: remove and replace by the actual js file instead
|
12 |
-
from src.share_btn import (share_js)
|
13 |
from src.utils import (
|
14 |
get_file_as_string,
|
15 |
get_sections,
|
@@ -50,21 +48,20 @@ preview("StarCoder Model URL", API_URL_STAR)
|
|
50 |
preview("StarCoderBase Model URL", API_URL_BASE)
|
51 |
preview("HF Token", HF_TOKEN, ofuscate=True)
|
52 |
|
53 |
-
|
54 |
-
|
55 |
-
|
56 |
-
|
57 |
-
|
58 |
-
|
59 |
-
|
60 |
-
|
61 |
-
|
62 |
-
|
63 |
-
|
64 |
-
|
65 |
-
|
66 |
-
|
67 |
-
manifest, description, disclaimer = readme_sections[:3]
|
68 |
|
69 |
theme = gr.themes.Monochrome(
|
70 |
primary_hue="indigo",
|
@@ -72,7 +69,7 @@ theme = gr.themes.Monochrome(
|
|
72 |
neutral_hue="slate",
|
73 |
radius_size=sizes.radius_sm,
|
74 |
font=[
|
75 |
-
gr.themes.GoogleFont("
|
76 |
"ui-sans-serif",
|
77 |
"system-ui",
|
78 |
"sans-serif",
|
@@ -159,7 +156,7 @@ examples = [
|
|
159 |
"def alternating(list1, list2):\n results = []\n for i in range(min(len(list1), len(list2))):\n results.append(list1[i])\n results.append(list2[i])\n if len(list1) > len(list2):\n <FILL_HERE>\n else:\n results.extend(list2[i+1:])\n return results",
|
160 |
]
|
161 |
|
162 |
-
with gr.Blocks(theme=theme, analytics_enabled=False, css=
|
163 |
with gr.Column():
|
164 |
gr.Markdown(description)
|
165 |
with gr.Row():
|
@@ -223,8 +220,8 @@ with gr.Blocks(theme=theme, analytics_enabled=False, css=CSS) as demo:
|
|
223 |
)
|
224 |
gr.Markdown(disclaimer)
|
225 |
with gr.Group(elem_id="share-btn-container"):
|
226 |
-
community_icon = gr.HTML(
|
227 |
-
loading_icon = gr.HTML(
|
228 |
share_button = gr.Button(
|
229 |
"Share to community", elem_id="share-btn", visible=True
|
230 |
)
|
@@ -235,7 +232,7 @@ with gr.Blocks(theme=theme, analytics_enabled=False, css=CSS) as demo:
|
|
235 |
fn=process_example,
|
236 |
outputs=[output],
|
237 |
)
|
238 |
-
gr.Markdown(
|
239 |
|
240 |
submit.click(
|
241 |
generate,
|
@@ -245,6 +242,6 @@ with gr.Blocks(theme=theme, analytics_enabled=False, css=CSS) as demo:
|
|
245 |
max_batch_size=8,
|
246 |
show_progress=True
|
247 |
)
|
248 |
-
share_button.click(None, [], [], _js=
|
249 |
|
250 |
demo.queue(concurrency_count=16).launch(debug=True, server_port=DEFAULT_PORT)
|
|
|
8 |
from text_generation import Client
|
9 |
from src.request import StarCoderRequest, StarCoderRequestConfig
|
10 |
|
|
|
|
|
11 |
from src.utils import (
|
12 |
get_file_as_string,
|
13 |
get_sections,
|
|
|
48 |
preview("StarCoderBase Model URL", API_URL_BASE)
|
49 |
preview("HF Token", HF_TOKEN, ofuscate=True)
|
50 |
|
51 |
+
_styles = get_file_as_string("styles.css")
|
52 |
+
_script = get_file_as_string("community-btn.js")
|
53 |
+
_sharing_icon_svg = get_file_as_string("community-icon.svg")
|
54 |
+
_loading_icon_svg = get_file_as_string("loading-icon.svg")
|
55 |
+
|
56 |
+
# Loads the whole content of the ./README.md file
|
57 |
+
# slicing/unpacking its different sections into their proper variables
|
58 |
+
readme_file_content = get_file_as_string("README.md", path='./')
|
59 |
+
(
|
60 |
+
manifest,
|
61 |
+
description,
|
62 |
+
disclaimer,
|
63 |
+
formats,
|
64 |
+
) = get_sections(readme_file_content, "---", up_to=4)
|
|
|
65 |
|
66 |
theme = gr.themes.Monochrome(
|
67 |
primary_hue="indigo",
|
|
|
69 |
neutral_hue="slate",
|
70 |
radius_size=sizes.radius_sm,
|
71 |
font=[
|
72 |
+
gr.themes.GoogleFont("IBM Plex Sans", [400, 600]),
|
73 |
"ui-sans-serif",
|
74 |
"system-ui",
|
75 |
"sans-serif",
|
|
|
156 |
"def alternating(list1, list2):\n results = []\n for i in range(min(len(list1), len(list2))):\n results.append(list1[i])\n results.append(list2[i])\n if len(list1) > len(list2):\n <FILL_HERE>\n else:\n results.extend(list2[i+1:])\n return results",
|
157 |
]
|
158 |
|
159 |
+
with gr.Blocks(theme=theme, analytics_enabled=False, css=_styles) as demo:
|
160 |
with gr.Column():
|
161 |
gr.Markdown(description)
|
162 |
with gr.Row():
|
|
|
220 |
)
|
221 |
gr.Markdown(disclaimer)
|
222 |
with gr.Group(elem_id="share-btn-container"):
|
223 |
+
community_icon = gr.HTML(_sharing_icon_svg, visible=True)
|
224 |
+
loading_icon = gr.HTML(_loading_icon_svg, visible=True)
|
225 |
share_button = gr.Button(
|
226 |
"Share to community", elem_id="share-btn", visible=True
|
227 |
)
|
|
|
232 |
fn=process_example,
|
233 |
outputs=[output],
|
234 |
)
|
235 |
+
gr.Markdown(formats)
|
236 |
|
237 |
submit.click(
|
238 |
generate,
|
|
|
242 |
max_batch_size=8,
|
243 |
show_progress=True
|
244 |
)
|
245 |
+
share_button.click(None, [], [], _js=_script)
|
246 |
|
247 |
demo.queue(concurrency_count=16).launch(debug=True, server_port=DEFAULT_PORT)
|
settings.py
CHANGED
@@ -5,6 +5,8 @@ DEFAULT_STARCODER_BASE_API_PATH = "bigcode/starcoderbase/"
|
|
5 |
FIM_INDICATOR = "<FILL_HERE>"
|
6 |
DEFAULT_PORT = 7860
|
7 |
|
|
|
|
|
8 |
DEFAULT_SETTINGS = dict(
|
9 |
temperature = 0.9,
|
10 |
max_new_tokens = 256,
|
|
|
5 |
FIM_INDICATOR = "<FILL_HERE>"
|
6 |
DEFAULT_PORT = 7860
|
7 |
|
8 |
+
STATIC_PATH = "static"
|
9 |
+
|
10 |
DEFAULT_SETTINGS = dict(
|
11 |
temperature = 0.9,
|
12 |
max_new_tokens = 256,
|
src/utils.py
CHANGED
@@ -2,8 +2,10 @@ import os
|
|
2 |
from typing import List
|
3 |
from urllib.parse import urljoin
|
4 |
|
5 |
-
from settings import
|
6 |
-
|
|
|
|
|
7 |
|
8 |
def masked(value: str, n_shown: int, length: int = None) -> str:
|
9 |
"""Returns a string with the first and last n_shown characters
|
@@ -61,11 +63,11 @@ def get_url_from_env_or_default_path(env_name: str, api_path: str) -> str:
|
|
61 |
DEFAULT_HUGGINGFACE_MODELS_API_BASE_URL, api_path
|
62 |
)
|
63 |
|
64 |
-
def get_file_as_string(file_name, path=
|
65 |
"""Loads the content of a file given its name
|
66 |
and returns all of its lines as a single string
|
67 |
if a file path is given, it will be used
|
68 |
-
instead of the
|
69 |
|
70 |
Args:
|
71 |
file_name (_type_): The name of the file to load.
|
@@ -78,16 +80,18 @@ def get_file_as_string(file_name, path='.') -> str:
|
|
78 |
return f.read()
|
79 |
|
80 |
|
81 |
-
def get_sections(string: str, delimiter: str) -> List[str]:
|
82 |
"""Splits a string into sections given a delimiter
|
83 |
|
84 |
Args:
|
85 |
string (str): The string to split
|
86 |
delimiter (str): The delimiter to use
|
|
|
|
|
87 |
|
88 |
Returns:
|
89 |
-
List[str]: The list of sections
|
90 |
"""
|
91 |
return [section.strip()
|
92 |
for section in string.split(delimiter)
|
93 |
-
if (section and not section.isspace())]
|
|
|
2 |
from typing import List
|
3 |
from urllib.parse import urljoin
|
4 |
|
5 |
+
from settings import (
|
6 |
+
DEFAULT_HUGGINGFACE_MODELS_API_BASE_URL,
|
7 |
+
STATIC_PATH,
|
8 |
+
)
|
9 |
|
10 |
def masked(value: str, n_shown: int, length: int = None) -> str:
|
11 |
"""Returns a string with the first and last n_shown characters
|
|
|
63 |
DEFAULT_HUGGINGFACE_MODELS_API_BASE_URL, api_path
|
64 |
)
|
65 |
|
66 |
+
def get_file_as_string(file_name, path=STATIC_PATH) -> str:
|
67 |
"""Loads the content of a file given its name
|
68 |
and returns all of its lines as a single string
|
69 |
if a file path is given, it will be used
|
70 |
+
instead of the default static path (from settings)
|
71 |
|
72 |
Args:
|
73 |
file_name (_type_): The name of the file to load.
|
|
|
80 |
return f.read()
|
81 |
|
82 |
|
83 |
+
def get_sections(string: str, delimiter: str, up_to: int = None) -> List[str]:
|
84 |
"""Splits a string into sections given a delimiter
|
85 |
|
86 |
Args:
|
87 |
string (str): The string to split
|
88 |
delimiter (str): The delimiter to use
|
89 |
+
up_to (int, optional): The maximum number of sections to return.
|
90 |
+
Defaults to None (which means all sections)
|
91 |
|
92 |
Returns:
|
93 |
+
List[str]: The list of sections (up to the given limit, if any provided)
|
94 |
"""
|
95 |
return [section.strip()
|
96 |
for section in string.split(delimiter)
|
97 |
+
if (section and not section.isspace())][:up_to]
|
src/share_btn.py → static/community-btn.js
RENAMED
@@ -1,4 +1,4 @@
|
|
1 |
-
|
2 |
async function uploadFile(file){
|
3 |
const UPLOAD_URL = 'https://huggingface.co/uploads';
|
4 |
const response = await fetch(UPLOAD_URL, {
|
@@ -72,4 +72,4 @@ ${outputTxt}`;
|
|
72 |
shareBtnEl.style.removeProperty('pointer-events');
|
73 |
shareIconEl.style.removeProperty('display');
|
74 |
loadingIconEl.style.display = 'none';
|
75 |
-
}
|
|
|
1 |
+
async () => {
|
2 |
async function uploadFile(file){
|
3 |
const UPLOAD_URL = 'https://huggingface.co/uploads';
|
4 |
const response = await fetch(UPLOAD_URL, {
|
|
|
72 |
shareBtnEl.style.removeProperty('pointer-events');
|
73 |
shareIconEl.style.removeProperty('display');
|
74 |
loadingIconEl.style.display = 'none';
|
75 |
+
}
|
static/{community_icon.svg → community-icon.svg}
RENAMED
File without changes
|
static/formats.md
DELETED
@@ -1,47 +0,0 @@
|
|
1 |
-
## Model Formats
|
2 |
-
|
3 |
-
The model is pretrained on code and is formatted with special tokens in addition to the pure code data,\
|
4 |
-
such as prefixes specifying the source of the file or tokens separating code from a commit message.\
|
5 |
-
Use these templates to explore the model's capacities:
|
6 |
-
|
7 |
-
### 1. Prefixes 🏷️
|
8 |
-
|
9 |
-
For pure code files, use any combination of the following prefixes:
|
10 |
-
|
11 |
-
```
|
12 |
-
<reponame>REPONAME<filename>FILENAME<gh_stars>STARS\ncode<|endoftext|>
|
13 |
-
```
|
14 |
-
|
15 |
-
STARS can be one of: 0, 1-10, 10-100, 100-1000, 1000+
|
16 |
-
|
17 |
-
### 2. Commits 💾
|
18 |
-
|
19 |
-
The commits data is formatted as follows:
|
20 |
-
|
21 |
-
```
|
22 |
-
<commit_before>code<commit_msg>text<commit_after>code<|endoftext|>
|
23 |
-
```
|
24 |
-
|
25 |
-
### 3. Jupyter Notebooks 📓
|
26 |
-
|
27 |
-
The model is trained on Jupyter notebooks as Python scripts and structured formats like:
|
28 |
-
|
29 |
-
```
|
30 |
-
<start_jupyter><jupyter_text>text<jupyter_code>code<jupyter_output>output<jupyter_text>
|
31 |
-
```
|
32 |
-
|
33 |
-
### 4. Issues 🐛
|
34 |
-
|
35 |
-
We also trained on GitHub issues using the following formatting:
|
36 |
-
|
37 |
-
```
|
38 |
-
<issue_start><issue_comment>text<issue_comment>...<issue_closed>
|
39 |
-
```
|
40 |
-
|
41 |
-
### 5. Fill-in-the-middle 🧩
|
42 |
-
|
43 |
-
Fill in the middle requires rearranging the model inputs. The playground handles this for you - all you need is to specify where to fill:
|
44 |
-
|
45 |
-
```
|
46 |
-
code before<FILL_HERE>code after
|
47 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
static/{loading_icon.svg → loading-icon.svg}
RENAMED
File without changes
|
static/styles.css
CHANGED
@@ -1,3 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
.generating {
|
2 |
visibility: hidden
|
3 |
}
|
@@ -44,7 +50,7 @@ a {
|
|
44 |
justify-content: center;
|
45 |
align-items: center;
|
46 |
border-radius: 9999px !important;
|
47 |
-
width:
|
48 |
}
|
49 |
|
50 |
#share-btn {
|
|
|
1 |
+
@import url('https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;600;700&display=swap');
|
2 |
+
|
3 |
+
h1, h2 {
|
4 |
+
font-family: 'IBM Plex Mono', sans-serif;
|
5 |
+
}
|
6 |
+
|
7 |
.generating {
|
8 |
visibility: hidden
|
9 |
}
|
|
|
50 |
justify-content: center;
|
51 |
align-items: center;
|
52 |
border-radius: 9999px !important;
|
53 |
+
width: 15rem;
|
54 |
}
|
55 |
|
56 |
#share-btn {
|