|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
metrics: |
|
- accuracy |
|
pipeline_tag: text2text-generation |
|
tags: |
|
- health |
|
- FHIR |
|
--- |
|
|
|
# bart-large |
|
|
|
This model is a fine-tuned version of [bart-large](https://huggingface.co/facebook/bart-large) on a manually created dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.40 |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 64 |
|
- eval_batch_size: 64 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- num_epochs: 3 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-----:|:----:|:---------------:| |
|
| - | 1.0 | 47 | 4.5156 |
|
... |
|
| - | 10 | 490 | 0.4086 |
|
|
|
|
|
## How to use |
|
|
|
```python |
|
def generate_text(input_text): |
|
# Tokenize the input text |
|
input_tokens = tokenizer(input_text, return_tensors='pt') |
|
|
|
# Move the input tokens to the same device as the model |
|
input_tokens = input_tokens.to(model.device) |
|
|
|
# Generate text using the fine-tuned model |
|
output_tokens = model.generate(**input_tokens) |
|
|
|
# Decode the generated tokens to text |
|
output_text = tokenizer.decode(output_tokens[0], skip_special_tokens=True) |
|
|
|
return output_text |
|
|
|
from transformers import BartForConditionalGeneration |
|
|
|
# Load the pre-trained BART model from the Hugging Face model hub |
|
model = BartForConditionalGeneration.from_pretrained('rasta/BART-FHIR-question') |
|
|
|
input_text = "List all procedures with reason reference to resource with ID 24680135." |
|
output_text = generate_text(input_text) |
|
print(output_text) |
|
``` |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.18.0 |
|
- Pytorch 1.11.0+cu113 |
|
- Datasets 2.1.0 |
|
- Tokenizers 0.12.1 |
|
|
|
|
