File size: 5,467 Bytes
30f2c6a bd92217 30f2c6a bd92217 30f2c6a de3f434 30f2c6a de3f434 30f2c6a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 |
---
license: apache-2.0
datasets:
- Zakia/drugscom_reviews
language:
- en
metrics:
- training loss
library_name: transformers
pipeline_tag: text-generation
tags:
- health
- medicine
- patient reviews
- drug reviews
- depression
- text generation
widget:
- text: After starting this new treatment, I felt
example_title: Example 1
- text: I was apprehensive about the side effects of
example_title: Example 2
- text: This medication has changed my life for the better
example_title: Example 3
- text: I've had a terrible experience with this medication
example_title: Example 4
- text: Since I began taking L-methylfolate, my experience has been
example_title: Example 5
---
# Model Card for Zakia/gpt2-drugscom_depression_reviews
This model is a GPT-2-based language model fine-tuned on drug reviews for the depression medical condition from Drugs.com.
The dataset used for fine-tuning is the [Zakia/drugscom_reviews](https://huggingface.co/datasets/Zakia/drugscom_reviews) dataset, which is filtered for the condition 'Depression'.
The base model for fine-tuning was the [gpt2](https://huggingface.co/gpt2).
## Model Details
### Model Description
- Developed by: [Zakia](https://huggingface.co/Zakia)
- Model type: Text Generation
- Language(s) (NLP): English
- License: Apache 2.0
- Finetuned from model: gpt2
## Uses
### Direct Use
This model is intended to generate text that mimics patient reviews of depression medications, useful for understanding patient sentiments and experiences.
### Out-of-Scope Use
This model is not designed to diagnose or treat depression or to replace professional medical advice.
## Bias, Risks, and Limitations
The model may inherit biases present in the dataset and should be used with caution in decision-making processes.
### Recommendations
Use the model as a tool for generating synthetic patient reviews and for NLP research.
## How to Get Started with the Model
Use the code below to generate synthetic reviews with the model.
```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer
import torch
model_name = "Zakia/gpt2-drugscom_depression_reviews"
model = GPT2LMHeadModel.from_pretrained(model_name)
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
# Function to generate text
def generate_review(prompt, model, tokenizer):
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs)
return tokenizer.decode(outputs[0], skip_special_tokens=True)
# Example usage for various scenarios
prompts = [
"After starting this new treatment, I felt",
"I was apprehensive about the side effects of",
"This medication has changed my life for the better",
"I've had a terrible experience with this medication",
"Since I began taking L-methylfolate, my experience has been"
]
for prompt in prompts:
print(f"Prompt: {prompt}")
print(generate_review(prompt, model, tokenizer))
print()
```
## Training Details
### Training Data
The model was fine-tuned on patient reviews related to depression, filtered from Drugs.com.
This dataset is accessible from [Zakia/drugscom_reviews](https://huggingface.co/datasets/Zakia/drugscom_reviews) on Hugging Face datasets (condition = 'Depression') for 'train'.
Number of records in train dataset: 9069 rows.
### Training Procedure
#### Preprocessing
The reviews were cleaned and preprocessed to remove quotes, HTML tags and decode HTML entities.
#### Training Hyperparameters
- Batch Size: 2
- Epochs: 5
## Evaluation
- Training Loss
#### Metrics
The model's performance was evaluated based on Training Loss.
### Results
The fine-tuning process yielded the following results:
| Epoch | Training Loss | Training Runtime | Training Samples | Training Samples per Second | Training Steps per Second |
|-------|---------------|------------------|------------------|-----------------------------|---------------------------|
| 5.0 | 0.5944 | 2:15:40.11 | 4308 | 2.646 | 1.323 |
The fine-tuning process achieved a final training loss of 0.5944 after 5 epochs, with the model processing
approximately 2.646 samples per second and completing 1.323 training steps per second over a training runtime
of 2 hours, 15 minutes, and 40 seconds.
## Technical Specifications
### Model Architecture and Objective
GPT-2 model architecture was used, with the objective of generating coherent and contextually relevant text based on patient reviews.
### Compute Infrastructure
The model was trained using a T4 GPU on Google Colab.
#### Hardware
T4 GPU via Google Colab.
## Citation
If you use this model, please cite the original GPT-2 paper:
**BibTeX:**
```bibtex
@article{radford2019language,
title={Language Models are Unsupervised Multitask Learners},
author={Radford, Alec and others},
year={2019}
}
```
**APA:**
Radford, A., et al. (2019). Language Models are Unsupervised Multitask Learners.
## More Information
For further queries or issues with the model, please use the [discussions section on this model's Hugging Face page](https://huggingface.co/Zakia/gpt2-drugscom_depression_reviews/discussions).
## Model Card Authors
- [Zakia](https://huggingface.co/Zakia)
## Model Card Contact
For more information or inquiries regarding this model, please use the [discussions section on this model's Hugging Face page](https://huggingface.co/Zakia/gpt2-drugscom_depression_reviews/discussions). |