adding blog post

app.py

@@ -23,6 +23,8 @@ def load_data():
     """
 
     st.header('📂 Load Data')
+    read_md('markdown/load_data.md')
+
     uploaded_file = st.file_uploader("Choose a CSV file",
                                      help='Upload a CSV file with the following columns: Title, Abstract')
 
@@ -49,7 +51,7 @@ def load_data():
     st.write(f'Number of rows: {len(data)}')
     if len(data) > 500:
         data = data.sample(500, random_state=0)
-        st.write(f'Only random 500 rows will be analyzed')
+        st.write('Only random 500 rows will be analyzed')
 
     data = data.reset_index(drop=True)
 
@@ -71,6 +73,8 @@ def topic_modeling(data):
     """
 
     st.header('🔥 Topic Modeling')
+    read_md('markdown/topic_modeling.md')
+
     cols = st.columns(3)
     with cols[0]:
         min_topic_size = st.slider('Minimum topic size', key='min_topic_size', min_value=2,
@@ -118,6 +122,7 @@ def strip_network(data, topic_data, topics):
     """
 
     st.header('🚀 STriP Network')
+    read_md('markdown/stripnet.md')
 
     with st.spinner('Cosine Similarity Calculation'):
         cosine_sim_matrix = helpers.cosine_sim(data)
@@ -175,6 +180,7 @@ def network_centrality(nx_net, topic_data):
     """
 
     st.header('🏅 Most Important Papers')
+    read_md('markdown/centrality.md')
 
     centrality_mapping = {
         'Betweenness Centrality': nx.betweenness_centrality,
@@ -199,26 +205,18 @@ def network_centrality(nx_net, topic_data):
             st.plotly_chart(fig, use_container_width=True)
 
 
-def about_me():
-    st.markdown(
-        """
-        💡🔥🚀 STriP v1.0 🚀🔥💡
-
-        If you like this work, please consider ❤️ this HugginFace Space and ⭐ this Github repo [Link](https://github.com/stephenleo/stripnet)
-
-        👨‍🔬 Author: Marie Stephen Leo 
-        
-        💻 Github: [stephenleo](https://github.com/stephenleo)
-
-        👔 Linkedin: [Marie Stephen Leo](https://www.linkedin.com/in/marie-stephen-leo/)
+def read_md(file_path):
+    """Reads a markdown file and returns the contents as a streamlit markdown component.
+    """
+    with open(file_path, 'r') as f:
+        content = st.markdown(f.read())
 
-        📝 Medium: [@stephen-leo](https://stephen-leo.medium.com/)
-        """
-    )
+    return content
 
 
 def main():
-    st.title('STriP (S3P): Semantic Similarity of Scientific Papers!')
+    st.title('STriPNet: Semantic Similarity of Scientific Papers!')
+    read_md('markdown/about_stripnet.md')
 
     logger.info('========== Step1: Loading data ==========')
     data, selected_cols = load_data()
@@ -233,7 +231,7 @@ def main():
         logger.info('========== Step4: Network Centrality ==========')
         network_centrality(nx_net, topic_data)
 
-    about_me()
+    read_md('markdown/about_me.md')
 
 
 if __name__ == '__main__':

helpers.py

@@ -149,7 +149,7 @@ def network_plot(topic_data, topics, neighbors):
             {
                 'group': row.Topic,
                 'label': row.Index,
-                'title': row.Text,
+                'title': text_processing(row.Text),
                 'size': 20, 'font': {'size': 20, 'color': 'white'}
             }
         )

markdown/about_me.md

@@ -0,0 +1,15 @@
+---
+
+💡🔥🚀 STriPNet v1.0 🚀🔥💡
+
+If you like this work, please consider ❤️ this HugginFace Space and 🌟 the STriPNet Github repo [Link](https://github.com/stephenleo/stripnet)
+
+👨‍🔬 Author: Marie Stephen Leo 
+
+💻 Github: [stephenleo](https://github.com/stephenleo)
+
+👔 Linkedin: [Marie Stephen Leo](https://www.linkedin.com/in/marie-stephen-leo/)
+
+📝 Medium: [@stephen-leo](https://stephen-leo.medium.com/)
+
+---

markdown/about_stripnet.md

@@ -0,0 +1,4 @@
+- Do you read a lot of Scientific Papers? 
+- Have you ever wondered what are the overarching themes in the papers that you've read and how all the papers are semantically connected to one another? 
+- **Look no further!!!**
+- Leverage the power of NLP Topic Modeling, Semantic Similarity, and Network analysis to study the themes and semantic relations within a corpus of research papers.

markdown/centrality.md

@@ -0,0 +1,10 @@
+- Now that we've contructed our semantic similarity network, we can use Graph Theory on this network to compute the centrality of each node using [NetworkX](https://networkx.org/).
+- The nodes with the highest centrality will be the most important nodes (papers) in our semantic network. [Centrality Wiki](https://en.wikipedia.org/wiki/Centrality)
+- There are several types of centrality depending on the problem statement [Link](https://neo4j.com/developer/graph-data-science/centrality-graph-algorithms/).
+- Since our goal is to find the most important papers that connect all the other papers in our corpus, we'll use [Betweenness Centrality](https://networkx.org/documentation/stable/reference/algorithms/generated/networkx.algorithms.centrality.betweenness_centrality.html#networkx.algorithms.centrality.betweenness_centrality) by default. Feel free to test out other centrality algorithms using the `Select Centrality Measure` dropdown.
+- The plot is generated using [Plotly](https://plotly.com/python/) and is fully interactive!
+    - Bars are ordered in descending order, so the most important paper in your corpus is the top bar.
+    - Colored by the topic the paper belongs to.
+    - Hover over any bar to view the title and truncated abstract of the paper
+    - Zoom in and out by clicking and dragging over the chart. Double click to zoom out.
+- Click on the `Download plot as png` button on the top right hand side corner to download a .png file of your plot.

markdown/load_data.md

@@ -0,0 +1,5 @@
+- STriPNet only accepts a `csv` file. 
+- Though STriPNet can work with any text column, it works best when two columns `Title` and `Abstract` (case insensitive) are provided. 
+- STriPNet automatically selects any column called `title` and `abstract` (case insensitive) that are present in your uploaded csv file. If your file doesnt contain any such column, you will need to manually select the text column(s) on which to run the analysis. 
+- If you select multiple columns, STriPNet internally concats the columns with a `[SEP]` keyword.
+- Finally, to keep resource usage down, this 🤗 Huggingface Spaces app will internally restrict to analysing the first 500 rows in the data that you upload. If you want to analyse more data, please check out (and 🌟 😄) the [STriPNet Github repo](https://github.com/stephenleo/stripnet) to run it on your local! You can also directly download this HF Spaces app and run on your local.

markdown/stripnet.md

@@ -0,0 +1,14 @@
+- STriP Network plots the network of the semantically similar papers and how they are connected to each other.
+- A network is a graph with nodes and edges. Each node here is one paper and the edges show the similarity between papers.
+- Similarity is calculated by the cosine similarity between the [SPECTER](https://github.com/allenai/specter) embeddings of the two papers. The cosine similarity is a measure of the angle between two vectors.
+- The `Cosine Similarity Threshold` parameter controls how similar two papers need to be, to get connected by an edge. Hover over the tooltip to get more information about it.
+- The default value of the Cosine Similarity Threshold is heuristically calculated by STriPNet to give decent results. Feel free to play around with it until you are satisfied.
+- The plot is generated using [PyVis](https://github.com/WestHealth/pyvis) with some customizations to [VisJS](https://visjs.github.io/vis-network/docs/network/) and is fully interactive! 
+    - Hover on a node to see the paper's title and truncated abstract. 
+    - Click on a node to see the edges that connect that node. 
+    - Zoom in and out by scrolling.
+    - Click on any empty space and drag the plot to move and recenter it.
+    - Click on nodes and move them around to view them better.
+    - Click on the legend boxes and move them around. I like to place the legend boxes over the cluster of nodes of the same color!
+    - The number on the node is the row number of the input csv file on which this paper was located. Remember that python row numbers start from 0.
+- Once you are happy with how your STriP Network plot looks, right click and save image to your local.

markdown/topic_modeling.md

@@ -0,0 +1,6 @@
+- The first step in STriPNet is to run Topic Modeling using the [BERTopic](https://github.com/MaartenGr/BERTopic) library.
+- BERTopic internally uses [Sentence Transformer](https://www.sbert.net/) models to convert text to embeddings, clusters them and extracts keywords from each cluster. 
+- Specifically, since STriPNet is intended to be used with scientific papers, we're using the [SPECTER](https://github.com/allenai/specter) pretrained sentence transformers model by Allen AI.
+- The `Minimum topic size` and `N-gram range` parameters control the clustering and keyword extraction of BERTopic respectively. Hover over the tooltip of each parameter to get more information about them. STriPNet internally chooses some heuristically tuned parameters depending on the data you've uploaded. Feel free to play around with the parameters until you get good topics.
+- You can visualize the quality of the topic modeling in various ways provided by the dropdown menu `Select Topic Modeling Visualization`. 
+- Finally, please take note that BERTopic results change with every run so the topics extracted might change everytime you run STriPNet even on the same data with the same settings. If your topics look weird, a simple page refresh or a $+1$ increase to `Minimum topic size` might fix it!