Lohith9923 commited on
Commit
53967c1
β€’
1 Parent(s): 35e6f3b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -91
README.md CHANGED
@@ -1,92 +1,13 @@
1
- # English to Hindi Text Translation using Transformers
2
-
3
- This project showcases a simple text translation model that translates English text to Hindi using the Hugging Face Transformers library. The model utilizes pre-trained sequence-to-sequence architecture for accurate and efficient translation.
4
-
5
- ## Table of Contents
6
-
7
- - [Project Overview](#project-overview)
8
- - [Installation](#installation)
9
- - [Usage](#usage)
10
- - [Model Training and Dataset](#model-training-and-dataset)
11
- - [Model Testing and Deployment](#model-testing-and-deployment)
12
- - [User Interface](#user-interface)
13
- - [Challenges Faced](#challenges-faced)
14
- - [Contributions](#contributions)
15
-
16
- ## Project Overview
17
-
18
- Text translation is an essential task in natural language processing, and this project aims to provide a practical example of building and deploying a translation model. The project covers the following aspects:
19
-
20
- - Data preprocessing: Tokenization and dataset preparation.
21
- - Model training: Training a sequence-to-sequence model for English-to-Hindi translation.
22
- - Model testing: Translating text using the trained model.
23
- - User interface: Creating a user-friendly interface for text translation.
24
-
25
- ## Installation
26
-
27
- To run this project, you'll need the following dependencies:
28
-
29
- - Python 3.x
30
- - TensorFlow
31
- - Hugging Face Transformers
32
- - Datasets library
33
- - Gradio
34
-
35
- You can install the required libraries using the following shell command:
36
-
37
- ```shell
38
- pip install datasets transformers[sentencepiece] tensorflow gradio -q
39
- ```
40
-
41
- ## Usage
42
- Download the folder from here and the run the following command
43
-
44
- ```shell
45
- python3 app.py
46
- ```
47
- After running this command
48
- ## Model Training and Dataset
49
- For training the text translation model.
50
- You can checkout the pre-trained model from [here](https://colab.research.google.com/corgiredirector?site=https%3A%2F%2Fhuggingface.co%2FHelsinki-NLP%2Fopus-mt-en-hi) and Dataset from [here](https://huggingface.co/datasets/cfilt/iitb-english-hindi/viewer/cfilt--iitb-english-hindi).
51
- - First Download the pre-trained model using **transformers** library in python.
52
- - Load the Dataset **cfilt/iitb-english-hindi** using **Datasets** library in python.
53
- - Initialized the model, tokenizer, and preprocessing function.
54
- - Tokenized the dataset and prepared the training and validation data.
55
- - Compiled the model with the optimizer(**Adam**) with required parameters.
56
- - Trained the model for the desired number of epochs.
57
-
58
- ## Model Testing and Deployment
59
- To test the trained model and deploy a user interface:
60
-
61
- - Saved the trained model at a preferred location.
62
- - Loaded the model from the location and tokenizer for testing.
63
- - Translated sample input text using the model.
64
- - Deployed a Gradio interface for user-friendly translation.
65
-
66
- ## User Interface
67
-
68
- The Gradio interface provides an interactive way to translate English text to Hindi. To use the interface:
69
-
70
- - Run the project and navigate to the specified URL.
71
- - Enter English text in the input box.
72
- - Checkout the translated Hindi text in the output box.
73
-
74
- ## Challenges Faced
75
-
76
- - Surfed through lot of resources in google and other platforms for best dataset for my project.
77
- - Spent a lot of time gathering the correct resources for understanding about transformers, LLM's and gradio.
78
-
79
- ## Contributions
80
- Contributions to this project are welcome! Here are some ways you can contribute:
81
-
82
- - Improve the model's translation quality and performance.
83
- - Enhance the user interface for a better user experience.
84
- - Add support for more languages and translation directions.
85
-
86
- To contribute, follow these steps:
87
-
88
- - Fork this repository.
89
- - Create a new branch for your feature or bug fix.
90
- - Commit your changes and push them to your fork.
91
- - Open a pull request with a detailed description of your changes.
92
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
 
2
+ ---
3
+ title: Text Translation
4
+ emoji: πŸ‘€
5
+ colorFrom: purple
6
+ colorTo: pink
7
+ sdk: gradio
8
+ sdk_version: 3.40.1
9
+ app_file: app.py
10
+ pinned: false
11
+ ---
12
+
13
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference