G.Hemanth Sai commited on
Commit
6b40170
·
1 Parent(s): 32d9382

readme changes

Browse files
Files changed (1) hide show
  1. README.md +9 -84
README.md CHANGED
@@ -1,85 +1,10 @@
1
- # Internship-IVIS-labs
2
-
3
- - The *Intelligent Question Generator* app is an easy-to-use interface built in Streamlit which uses [KeyBERT](https://github.com/MaartenGr/KeyBERT), [Sense2vec](https://github.com/explosion/sense2vec), [T5](https://huggingface.co/ramsrigouthamg/t5_paraphraser)
4
- - It uses a minimal keyword extraction technique that leverages multiple NLP embeddings and relies on [Transformers](https://huggingface.co/transformers/) 🤗 to create keywords/keyphrases that are most similar to a document.
5
- - [sense2vec](https://github.com/explosion/sense2vec) (Trask et. al, 2015) is a nice twist on word2vec that lets you learn more interesting and detailed word vectors.
6
-
7
- ## Repository Breakdown
8
- ### src Directory
9
  ---
10
- - `src/Pipeline/QAhaystack.py`: This file contains the code of question answering using [haystack](https://haystack.deepset.ai/overview/intro).
11
- - `src/Pipeline/QuestGen.py`: This file contains the code of question generation.
12
- - `src/Pipeline/Reader.py`: This file contains the code of reading the document.
13
- - `src/Pipeline/TextSummariztion.py`: This file contains the code of text summarization.
14
- - `src/PreviousVersionCode/context.py`: This file contains the finding the context of the paragraph.
15
- - `src/PreviousVersionCode/QuestionGenerator.py`: This file contains the code of first attempt of question generation.
16
-
17
- ## Installation
18
- ```shell
19
- $ git clone https://github.com/HemanthSai7/Internship-IVIS-labs.git
20
- ```
21
- ```shell
22
- $ cd Internship-IVIS-labs
23
- ```
24
- ```python
25
- pip install -r requirements.txt
26
- ```
27
- - For the running the app for the first time locally, you need to uncomment the the lines in `src/Pipeline/QuestGen.py` to download the models to the models directory.
28
-
29
- ```python
30
- streamlit run app.py
31
- ```
32
- - Once the app is running, you can access it at http://localhost:8501
33
- ```shell
34
- You can now view your Streamlit app in your browser.
35
-
36
- Local URL: http://localhost:8501
37
- Network URL: http://192.168.0.103:8501
38
- ```
39
-
40
- ## Tech Stack Used
41
- ![image](https://img.shields.io/badge/Sense2vec-EF546D?style=for-the-badge&logo=Explosion.ai&logoColor=white)
42
- ![image](https://img.shields.io/badge/Spacy-09A3D5?style=for-the-badge&logo=spaCy&logoColor=white)
43
- ![image](https://img.shields.io/badge/Haystack-03AF9D?style=for-the-badge&logo=Haystackh&logoColor=white)
44
- ![image](https://img.shields.io/badge/Python-3776AB?style=for-the-badge&logo=python&logoColor=white)
45
- ![image](https://img.shields.io/badge/PyTorch-D04139?style=for-the-badge&logo=pytorch&logoColor=white)
46
- ![image](https://img.shields.io/badge/Numpy-013243?style=for-the-badge&logo=numpy&logoColor=white)
47
- ![image](https://img.shields.io/badge/Pandas-130654?style=for-the-badge&logo=pandas&logoColor=white)
48
- ![image](https://img.shields.io/badge/matplotlib-b2feb0?style=for-the-badge&logo=matplotlib&logoColor=white)
49
- ![image](https://img.shields.io/badge/scikit_learn-F7931E?style=for-the-badge&logo=scikit-learn&logoColor=white)
50
- ![image](https://img.shields.io/badge/Streamlit-EA6566?style=for-the-badge&logo=streamlit&logoColor=white)
51
-
52
- ## Timeline
53
- ### Week 1-2:
54
- #### Tasks
55
- - [x] Understanding and brushing up the concepts of NLP.
56
- - [x] Extracting images and text from a pdf file and storing it in a texty file.
57
- - [x] Exploring various open source tools for generating questions from a given text.
58
- - [x] Read papers related to the project (Bert,T5,RoBERTa etc).
59
- - [x] Summarizing the extracted text using T5 base pre-trained model from the pdf file.
60
-
61
- ### Week 3-4:
62
- #### Tasks
63
- - [x] Understanding the concept of QA systems.
64
- - [x] Created a basic script for generating questions from the text.
65
- - [x] Created a basic script for finding the context of the paragraph.
66
-
67
- ### Week 5-6:
68
- #### Tasks
69
-
70
- - [x] Understanding how Transformers models work for NLP tasks Question answering and generation
71
- - [x] Understanding how to use the Haystack library for QA systems.
72
- - [x] Understanding how to use the Haystack library for Question generation.
73
- - [x] PreProcessed the document for Haystack QA for better results .
74
-
75
- ### Week 7-8:
76
- #### Tasks
77
- - [x] Understanding how to generate questions intelligently.
78
- - [x] Explored wordnet to find synonyms
79
- - [x] Used BertWSD for disambiguating the sentence provided.
80
- - [x] Used KeyBERT for finding the keywords in the document.
81
- - [x] Used sense2vec for finding better words with high relatedness for the keywords generated.
82
-
83
- ### Week 9-10:
84
- #### Tasks
85
- - [x] Create a streamlit app to demonstrate the project.
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: {{title}}
3
+ emoji: {{emoji}}
4
+ colorFrom: {{colorFrom}}
5
+ colorTo: {{colorTo}}
6
+ sdk: {{streamlit}}
7
+ sdk_version: {{1.13.0}}
8
+ app_file: app.py
9
+ pinned: false
10
+ ---