Add Usage and datasets

README.md

@@ -1,17 +1,106 @@
 ---
 datasets:
 - google-research-datasets/C4_200M-synthetic-dataset-for-grammatical-error-correction
+- common-crawl
 
 language:
 - en
+
 license: mit
+
 widget:
-- text: "Do you even know why I always need changed our checking account number"
-  example_title: "Example 1"
-- text: "Ironman and Captain America is going out"
-  example_title: "Example 2"
-- text: "We all eat fish and then made dessert"
-  example_title: "Example 3"
-- text: "We have our Dinner yesterday"
-  example_title: "Example 4"
----
+- text: Do you even know why I always need changed our checking account number
+  example_title: Example 1
+- text: Ironman and Captain America is going out
+  example_title: Example 2
+- text: We all eat fish and then made dessert
+  example_title: Example 3
+- text: We have our Dinner yesterday
+  example_title: Example 4
+
+tags:
+- context-correction
+- error-correction
+---
+
+# T5 Context Corrector (base-sized)
+t5-context-corrector model is a fine-tuned [T5 model](https://huggingface.co/t5-base) on the [Synthetic GEC](https://github.com/google-research-datasets/C4_200M-synthetic-dataset-for-grammatical-error-correction) and filtered CommonCrawl data in English Language.
+The Base Model(T5) is Pre-trained on C4(Colossal Clean Crawled Corpus) dataset and works with numerous downstrem tasks. <br>
+Our Model specifically is fine-tuned on a single downstream task of context correction that's trained on the above mentioned two datasets.
+
+## Model description
+This Model has the same architecture as of its base model, thus having 220 Million Parameters while consisting of 12 encoder blocks and 12 decoder blocks with an input embedding size of 32128. Please refer to this [link](https://arxiv.org/pdf/1910.10683.pdf) to know more about the model details.
+
+## Intended Use & Limitations
+As the model is intented to correct the context of the given sentence, all you have to do is pass the non-contextually correct sentence and get the corrected response back.<br>
+Based on Multiple experiments performed as part of the training, we observe that the model works best when the the total number of tokens in the input is less than 256.<br>
+So, if you have a long paragraph that needs to be context corrected, we suggest to first sentence tokenize the paragraph and run the context corrector for each sentence separately to obtain best results.
+
+Note that the model is primarily trained on general publicly available corpus, so it maynot work well for Medical Contexts.
+
+## Usage
+
+You can use this model directly with a pipeline for Text to Text Generation:
+
+```python
+from transformers import pipeline
+
+
+ctx_corr = pipeline("text-generation", model='DeathReaper0965/t5-context-corrector')
+ctx_corr("Do you even know why I always need changed our checking account number")
+
+###########OUTPUT###########
+
+# [{'generated_text': 'Do you even know why I always need to change our checking account number?'}]
+```
+
+Or you can also use the model to get the features for a given text:
+
+```python
+from nltk import sent_tokenize
+
+from transformers import T5ForConditionalGeneration, T5Tokenizer
+
+
+# Load model and tokenizer
+cc_tokenizer = T5Tokenizer.from_pretrained("DeathReaper0965/t5-context-corrector")
+cc_model = T5ForConditionalGeneration.from_pretrained("DeathReaper0965/t5-context-corrector")
+
+# Utility function to correct context
+def correct_context(input_text, temperature=0.5):
+    # tokenize
+    batch = cc_tokenizer(input_text,
+                         truncation=True,
+                         padding='max_length',
+                         max_length=256, 
+                         return_tensors="pt")
+
+    # forward pass
+    results = cc_model.generate(**batch,
+                                max_length=256,
+                                num_beams=3,
+                                no_repeat_ngram_size=2,
+                                repetition_penalty=2.5,
+                                temperature=temperature,
+                                do_sample=True)
+    
+    return results
+
+# Utility function to split the paragraph to multiple sentences
+def split_and_correct_context(sent):
+    sents = sent_tokenize(sent)
+    
+    final_sents = cc_tokenizer.batch_decode(correct_context(sents), 
+                                            clean_up_tokenization_spaces=True, 
+                                            skip_special_tokens=True)
+    
+    final_sents = " ".join([final_sents[i].strip() for i in range(len(final_sents))])
+    
+    return final_sents
+
+
+split_and_correct_context("Do you even know why I always need changed our checking account number. If not let me know")
+
+###########OUTPUT###########
+'Do you even know why I always need to change our checking account number? If not, let me know.'
+```