metadata

language:
  - en
license: other
library_name: transformers
tags:
  - dark-pattern
  - dark-pattern-classification
  - BERT
  - dark-pattern-detection
metrics:
  - accuracy
pipeline_tag: text-classification

Model Card for Model ID

This modelcard aims to be a base template for new models. It has been generated using this raw template.

Model Details

Model Description

Developed by: [Adarsh Maurya]
Model type: [Safetensors-F32]
License: [Other]
Finetuned from model: [google-bert/bert-base-uncased]

Model Sources [optional]

Repository: [https://github.com/4darsh-Dev/CogniGaurd]
Paper [optional]: [More Information Needed]
Demo: [https://huggingface.co/spaces/4darsh-Dev/dark_pattern_detector_app]

Uses

For Detection of Text Based Dark Patterns.
It has been to classify dark patterns in 7 Categories( Urgency, Scarcity, Misdirection, Social-Proof, Obstruction, Sneaking, Forced Action) + Not Dark Pattern.

Direct Use

Usage

This model can be loaded and used with the Transformers library:

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = "your-username/your-model-name"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Example usage
text = "Only 2 items left in stock!"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
predictions = outputs.logits.argmax(-1)

How to Get Started with the Model

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

class DarkPatternDetector:
    def __init__(self, model_name):
        self.label_dict = {
            0: "Urgency", 1: "Not Dark Pattern", 2: "Scarcity", 3: "Misdirection",
            4: "Social Proof", 5: "Obstruction", 6: "Sneaking", 7: "Forced Action"
        }
        self.device = 'cuda' if torch.cuda.is_available() else 'cpu'
        print(f"Using device: {self.device}")

        self.model = AutoModelForSequenceClassification.from_pretrained(model_name).to(self.device)
        self.tokenizer = AutoTokenizer.from_pretrained(model_name)

    def predict(self, text):
        inputs = self.tokenizer(text, return_tensors="pt", truncation=True, padding=True).to(self.device)
        
        with torch.no_grad():
            outputs = self.model(**inputs)
            probabilities = torch.nn.functional.softmax(outputs.logits, dim=1)
            predicted_label = torch.argmax(probabilities, dim=1).item()

        return self.label_dict[predicted_label]

# Usage
if __name__ == "__main__":
    # Replace with your Hugging Face model name
    model_name = "your-username/your-model-name"
    detector = DarkPatternDetector(model_name)

    # Example usage
    texts_to_predict = [
        "Only 2 items left in stock!",
        "This offer ends in 10 minutes!",
        "Join now and get 50% off!",
        "By clicking 'Accept', you agree to our terms and conditions."
    ]

    for text in texts_to_predict:
        result = detector.predict(text)
        print(f"Text: '{text}'\nPredicted Dark Pattern: {result}\n")

Training Details

Training Data

[More Information Needed]

Training Process

The model was fine-tuned for 5 epochs on a dataset of 5,000 examples.
We used the AdamW optimizer with a learning rate of 2e-5.
The maximum sequence length was set to 256 tokens.
Training was performed using mixed precision (FP16) for efficiency.

-->

[More Information Needed]

Testing Data, Factors & Metrics

Testing Data

[More Information Needed]

Metrics

Our model's performance is evaluated using the following metrics:

Accuracy: The proportion of correct predictions among the total number of cases examined.
Precision: The ratio of correctly predicted positive observations to the total predicted positive observations.
Recall: The ratio of correctly predicted positive observations to all observations in the actual class.
F1-Score: The harmonic mean of Precision and Recall, providing a single score that balances both metrics.

These metrics were chosen to provide a comprehensive view of the model's performance across different aspects of classification accuracy.

Results

Metric	Score
Accuracy	0.811881
Precision	0.808871
Recall	0.811881
F1-Score	0.796837

Our model demonstrates strong performance across all metrics:

An accuracy of 81.19% indicates that the model correctly classifies a high proportion of samples.
The precision of 80.89% shows that when the model predicts a specific dark pattern, it is correct about 81% of the time.
The recall of 81.19% indicates that the model successfully identifies about 81% of the actual dark patterns in the dataset.
An F1-Score of 79.68% represents a good balance between precision and recall.

Summary

These results suggest that the model is effective at detecting and classifying dark patterns, with a good balance between identifying true positives and avoiding false positives.

Model Architecture and Objective

Compute Infrastructure

Hardware

GPU: NVIDIA Tesla P100 (16GB VRAM)
Platform: Kaggle Notebooks

Software

Python 3.10
PyTorch 1.13.1
Transformers library 4.29.2
CUDA 11.6

Model Card Authors

This model card was authored by:

Adarsh Maurya (CS Student, Keshav Mahavidyala[UOD])

Model Card Contact

For questions, comments, or feedback about this model, please contact:

Email: adarsh@onionreads.com
GitHub: https://github.com/4darsh-Dev/CogniGaurd
Twitter: @4darsh_Dev

For urgent inquiries, don't hesitate to get in touch with the lead researcher: Mr. Adarsh Maurya Email: adarsh230427@keshav.du.ac.in