Update README.md
Browse files
README.md
CHANGED
@@ -12,58 +12,44 @@ pipeline_tag: text2text-generation
|
|
12 |
# Model Card for Model ID
|
13 |
|
14 |
<!-- Provide a quick summary of what the model is/does. -->
|
15 |
-
|
16 |
-
This modelcard aims to be a base template for new models.
|
17 |
|
18 |
## Model Details
|
19 |
|
20 |
### Model Description
|
21 |
|
22 |
<!-- Provide a longer summary of what this model is. -->
|
23 |
-
This model aims to provide the service of traslation between Chinese and English(Only Chinese to English available now), which based on
|
|
|
24 |
The model is good at grammar in translation betweeen zh-en, if you don't want to fork this repository, just try the API reference aside this page ^_^
|
25 |
|
26 |
|
27 |
-
- **Developed by:** Varine
|
28 |
-
- **
|
29 |
-
- **
|
30 |
-
- **
|
31 |
-
- **
|
32 |
-
- **
|
33 |
-
- **Finetuned from model [optional]:** [More Information Needed]
|
34 |
|
35 |
-
### Model Sources
|
36 |
|
37 |
<!-- Provide the basic links for the model. -->
|
38 |
|
39 |
-
- **Repository:**
|
40 |
-
- **Paper [optional]:** [More Information Needed]
|
41 |
-
- **Demo [optional]:** [More Information Needed]
|
42 |
|
43 |
## Uses
|
44 |
|
45 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
46 |
-
|
47 |
### Direct Use
|
48 |
|
49 |
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
50 |
-
|
51 |
-
[More Information Needed]
|
52 |
-
|
53 |
-
### Downstream Use [optional]
|
54 |
-
|
55 |
-
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
56 |
-
|
57 |
-
[More Information Needed]
|
58 |
-
|
59 |
-
### Out-of-Scope Use
|
60 |
-
|
61 |
-
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
62 |
-
|
63 |
-
[More Information Needed]
|
64 |
|
65 |
## Bias, Risks, and Limitations
|
66 |
-
|
|
|
67 |
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
68 |
|
69 |
[More Information Needed]
|
@@ -76,21 +62,31 @@ Users (both direct and downstream) should be made aware of the risks, biases and
|
|
76 |
|
77 |
## How to Get Started with the Model
|
78 |
|
79 |
-
Use the code below to get started with the model.
|
80 |
|
81 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
82 |
|
83 |
## Training Details
|
84 |
|
85 |
### Training Data
|
86 |
|
87 |
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
88 |
-
|
89 |
-
[More Information Needed]
|
90 |
|
91 |
### Training Procedure
|
92 |
|
93 |
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
|
|
|
|
94 |
|
95 |
#### Preprocessing [optional]
|
96 |
|
|
|
12 |
# Model Card for Model ID
|
13 |
|
14 |
<!-- Provide a quick summary of what the model is/does. -->
|
15 |
+
**This is a Chinese-English translation model based on Transformer architecture(beta version 0.0.1).**
|
16 |
+
This modelcard aims to be a base template for new models.
|
17 |
|
18 |
## Model Details
|
19 |
|
20 |
### Model Description
|
21 |
|
22 |
<!-- Provide a longer summary of what this model is. -->
|
23 |
+
This model aims to provide the service of traslation between Chinese and English(Only Chinese to English available now), which based on wmt19 dataset.
|
24 |
+
|
25 |
The model is good at grammar in translation betweeen zh-en, if you don't want to fork this repository, just try the API reference aside this page ^_^
|
26 |
|
27 |
|
28 |
+
- **Developed by:** **Varine**
|
29 |
+
- **Shared by:** **TianQi Xu**
|
30 |
+
- **Model type:** **Tranformer**
|
31 |
+
- **Language(s) (NLP):** **Chinese, English**
|
32 |
+
- **License:** **MIT**
|
33 |
+
- **Finetuned from model:** **opus-mt-zh-en-fintuned-zhen-checkpoints**
|
|
|
34 |
|
35 |
+
### Model Sources
|
36 |
|
37 |
<!-- Provide the basic links for the model. -->
|
38 |
|
39 |
+
- **Repository:** <https://huggingface.co/Varine/opus-mt-zh-en-model>
|
|
|
|
|
40 |
|
41 |
## Uses
|
42 |
|
43 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
44 |
+
This model can be used in tranlation missions between Chinese and English.
|
45 |
### Direct Use
|
46 |
|
47 |
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
48 |
+
As it's a traditional translation model, it can be used in many circumstances, including translation between some academical papers, news, and **even some of the literary works(as the excellent performance the model is in grammar and multi-context cases)**.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
49 |
|
50 |
## Bias, Risks, and Limitations
|
51 |
+
**1.Remember this is a beta version of this translation model,thus we add the limitation on the scale of input tokens, so plz make sure the scale of your input text won't overflow the limit.**
|
52 |
+
**2.DO NOT APPLY THIS MODEL FOR ILLEGAL USES.**
|
53 |
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
54 |
|
55 |
[More Information Needed]
|
|
|
62 |
|
63 |
## How to Get Started with the Model
|
64 |
|
65 |
+
Before we enjoy this model, Plz follow the possible directions to make sure your environment is appropriate.Use the code below to get started with the model.
|
66 |
|
67 |
+
1.Use git tools to fork this repository(If you don't wan't to torture youself in configuring environment, just feel free to use API!):
|
68 |
+
```
|
69 |
+
git clone "https://huggingface.co/Varine/opus-mt-zh-en-model"
|
70 |
+
```
|
71 |
+
2.After forking, plz make sure you have installed the modules below in your Jupiter Notebook or other IDEs:
|
72 |
+
```
|
73 |
+
! pip install transformers datasets numpy
|
74 |
+
```
|
75 |
+
3.After checking the packages, plz run the code in [translation.ipynb](https://huggingface.co/Varine/opus-mt-zh-en-model/translation.ipynb), and a Jupiter Notebook environment is recommended in this step.
|
76 |
+
4.Ultimately you can enjoy the whole model my loading the model by code and pipelining translation strategy, and don't forget to input your text!
|
77 |
|
78 |
## Training Details
|
79 |
|
80 |
### Training Data
|
81 |
|
82 |
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
83 |
+
- wmt/wmt19
|
|
|
84 |
|
85 |
### Training Procedure
|
86 |
|
87 |
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
88 |
+
As the dataset we choose in training is tremendous in scale, so after analyzing, we decided to use the only 4% among the whole dataset to train, and we divided the 4% data in 10 epoch to evaluate the training loss and and validation loss in every part of the epoch.
|
89 |
+
Moreover, we need to claim that, the data form that we used in our training progress is Chinese-English sentence pairs(to better embedding and compare them in higher-dimension space in Transformer architecture).
|
90 |
|
91 |
#### Preprocessing [optional]
|
92 |
|