Update README.md

README.md

@@ -72,8 +72,61 @@ This model gets the following results on WebArena and WebVoyager:
 
 
 ## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-TODO
+
+### **Bias**
+As with all ML models, **Llama8b-NNetNav-WA** inherits biases from its training data. Since the dataset is collected via unsupervised exploration on self-hosted WebArena websites, it will reflect biases present in website structures, navigation flows, and content representations.
+
+- **Selection Bias:** The model is trained on Self-hosted websites that mimic reddit, github, google maps, simple e-commerce websites and CMS websites. This model is likely to struggle with websites with modern layouts seen on live websites.   
+- **Demographic Bias:** WebArena self-hosted websites over-represent Western English-speaking users, and the model may perform worse on non-English or culturally distinct websites.  
+  - Example: A model trained mostly on U.S. e-commerce sites may navigate amazon.com effectively but may struggle with Flipkart (India) or Rakuten (Japan).
+
+If you are interested in training a NNetNav based agent for your own domain, please check out our [codebase](https://github.com/MurtyShikhar/NNetnav). Or if you're interested in a model that has been shown to work well on a variety of live websites, please check out [LLama8b-NNetNav-Live](https://huggingface.co/stanfordnlp/llama8b-nnetnav-live)
+
+### **Risks**
+#### **1. Unintended Actions**
+The model operates by executing web actions based on textual observation spaces, which may lead to unintended consequences when dealing with ambiguous or poorly structured websites.
+
+- If instructed to "delete all spam messages in my inbox," but the website has unusual button placement in the AXTree, the model might mistakenly delete important emails instead.
+- If asked to "buy the cheapest laptop on Amazon," the model might select an accessory instead of an actual laptop if the AXTree of the listing page has misleading layout
+
+#### **2. Security & Privacy Risks**
+Since the model interacts with external web content, there are significant risks related to unintentional data exposure, credential leaks, and interaction with harmful content.
+
+- If asked to "log into my Gmail and check unread emails," the model may type and submit credentials without realizing it, potentially exposing passwords.
+- A user asking the model to "search for free software downloads" might inadvertently lead to interactions with phishing or malware-hosting sites.
+
+#### **3. Adversarial Manipulation**
+Malicious websites can deceive the model by using **dark patterns**—UI/UX tricks that mislead users (or bots).
+
+- A fraudulent website may create **fake "Close" buttons** in the AXTree that actually trigger **downloads or pop-ups**. The model, thinking it's closing a window, may instead **click a malicious link**.
+- If asked to "unsubscribe from a newsletter," but the page uses **misleading button labels** in the AXTree (e.g., "Unsubscribe" actually means "Resubscribe"), the model could perform the opposite action.
+
+#### **4. Legal & Ethical Considerations**
+Web navigation often involves handling user-generated content, news, and e-commerce transactions, all of which pose ethical and legal challenges.
+
+- If instructed to "find the latest election results," the model might click on a misleading news source, potentially spreading misinformation.
+- If asked to "find the cheapest flight ticket," it could unintentionally violate terms of service by scraping restricted airline data.
+
+### **Limitations**
+#### **1. Generalization to Unseen Websites**
+This model is trained via interaction on 5 self-hosted WebArena websites, and is known to struggle on real, live websites. Please check out [LLama8b-NNetNav-Live](https://huggingface.co/stanfordnlp/llama8b-nnetnav-live) for a model that performs better on live websites.
+
+#### **2. Instruction Sensitivity**
+Vague instructions can lead to unintended actions.
+
+- "Find me the best laptop for gaming" is **subjective**, and the model might select a **random option** instead of following some criteria (e.g., GPU, refresh rate).
+
+#### **3. Performance on Long-Horizon Tasks**
+The model may struggle when tasks require **deep memory retention, complex multi-step planning, or backtracking**.
+
+- *Example:* When booking a hotel on a travel website, the model might navigate **through multiple filters and options** but forget previous selections when reaching the checkout page.
+
+#### **4. Token Limitations**
+The model's **maximum sequence length of 20k tokens** limits its ability to handle long, continuous web interactions.
+
+- *Example:* When filling a very long multi-step form, the model might forget earlier responses, leading to errors.
+
+
 
 ## How to Get Started with the Model