--- license: apache-2.0 tags: - generated_from_keras_callback model-index: - name: multilingual_sentiment_newspaper_headlines results: [] --- # multilingual_sentiment_newspaper_headlines This model is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) on a dataset of 30k newspaper headlines in German, Polish, English, Dutch and Spanish. The dataset contains 6k headlines in each of the five languages. The newspapers used are as follows: + Polish: *Fakt, Rzeczpospolita, Gazeta Wyborcza* + English: *The Times, The Guardian, The Sun* + Dutch: *De Telegraaf, NRC, Volkskrant* + Spanish: *El Mundo, El Pais, ABC* + German: *Suddeutsche Zeitung, De Welt, Bild* It achieves the following results on the evaluation set: - Train Loss: 0.2886 - Train Sparse Categorical Accuracy: 0.8688 - Validation Loss: 1.0107 - Validation Sparse Categorical Accuracy: 0.6434 - Epoch: 4 ``` {python} from transformers import AutoTokenizer, AutoModelForSequenceClassification, TextClassificationPipeline tokenizer = tokenizer = AutoTokenizer.from_pretrained("z-dickson/multilingual_sentiment_newspaper_headlines") m1 = TFAutoModelForSequenceClassification.from_pretrained("z-dickson/multilingual_sentiment_newspaper_headlines") sentiment_classifier = TextClassificationPipeline(tokenizer=tokenizer, model=m1) sentiment_classifier('Brazylia: Bolsonaro wci±ż nie uznał porażki. Jego zwolennicy blokuj± autostrady') [{'label': 'negative, 0', 'score': 0.9989686012268066}] ``` ### Training hyperparameters The following hyperparameters were used during training: - optimizer: {'name': 'Adam', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False} - training_precision: float32 ### Training results | Train Loss | Train Sparse Categorical Accuracy | Validation Loss | Validation Sparse Categorical Accuracy | Epoch | |:----------:|:---------------------------------:|:---------------:|:--------------------------------------:|:-----:| | 0.8008 | 0.6130 | 0.7099 | 0.6558 | 0 | | 0.6148 | 0.6973 | 0.7559 | 0.6200 | 1 | | 0.4626 | 0.7690 | 0.8233 | 0.6368 | 2 | | 0.3632 | 0.8229 | 0.9609 | 0.6454 | 3 | | 0.2886 | 0.8688 | 1.0107 | 0.6434 | 4 | ### Framework versions - Transformers 4.26.0 - TensorFlow 2.9.2 - Tokenizers 0.13.2