|
---
|
|
language:
|
|
- en
|
|
tags:
|
|
- question-answering
|
|
- transformers
|
|
- bert
|
|
- squad
|
|
license: apache-2.0
|
|
datasets:
|
|
- squad
|
|
model-name: bert-base-uncased-finetuned-squad
|
|
library_name: transformers
|
|
---
|
|
|
|
# BERT-Base Uncased Fine-Tuned on SQuAD
|
|
|
|
## Overview
|
|
|
|
This repository contains a **BERT-Base Uncased** model fine-tuned on the **SQuAD (Stanford Question Answering Dataset)** for **Question Answering (QA) tasks**. The model has been fine-tuned for **2 epochs**, making it suitable for extracting answers from given contexts by predicting start and end token positions.
|
|
|
|
## The Model predicts 2 probabilities among all the tokens in the vocab , One indicating the start token and the other indicating the end token, Then the answer between both these tokens are extracted.
|
|
|
|
## Model Details
|
|
|
|
- **Model Type**: BERT-Base Uncased
|
|
- **Fine-Tuning Dataset**: SQuAD (Stanford Question Answering Dataset)
|
|
- **Number of Epochs**: 2
|
|
- **Task**: Question Answering
|
|
- **Base Model**: [BERT-Base Uncased](https://huggingface.co/bert-base-uncased)
|
|
|
|
---
|
|
|
|
## Usage
|
|
|
|
### How to Load the Model
|
|
|
|
You can load the model using the `transformers` library from Hugging Face:
|
|
|
|
```python
|
|
from transformers import BertForQuestionAnswering, BertTokenizer
|
|
|
|
# Load the tokenizer and model
|
|
tokenizer = BertTokenizer.from_pretrained("Abdo36/Bert-SquAD-QA")
|
|
model = BertForQuestionAnswering.from_pretrained("Abdo36/Bert-SquAD-QA")
|
|
|
|
context = "BERT is a method of pre-training language representations."
|
|
question = "What is BERT?"
|
|
|
|
inputs = tokenizer.encode_plus(question, context, return_tensors="pt")
|
|
|
|
# Perform inference
|
|
outputs = model(**inputs)
|
|
start_scores = outputs.start_logits
|
|
end_scores = outputs.end_logits
|
|
|
|
# Extract answer
|
|
start_index = start_scores.argmax()
|
|
end_index = end_scores.argmax()
|
|
answer = tokenizer.decode(inputs["input_ids"][0][start_index:end_index + 1])
|
|
|
|
print("Answer:", answer)
|
|
```
|
|
|
|
## Citation
|
|
|
|
If you use this model in your research, please cite the original BERT paper:
|
|
|
|
```bibtex
|
|
@article{devlin2018bert,
|
|
title={BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding},
|
|
author={Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
|
|
journal={arXiv preprint arXiv:1810.04805},
|
|
year={2018}
|
|
}
|
|
```
|
|
|
