language:
- en
pipeline_tag: text2text-generation
library_name: transformers
tags:
- style-transfer
- formality-transfer
Text Style Transfer using CycleGANs
This repository contains the models from the paper "Self-supervised Text Style Transfer using Cycle-Consistent Adversarial Networks" (ACM TIST 2024).
The work introduces a novel approach to Text Style Transfer using CycleGANs with sequence-level supervision and Transformer architectures.
Available Models
Formality transfer
GYAFC dataset (Family & Relationships)
| model | checkpoint |
|---|---|
| BART base | informal-to-formal, formal-to-informal |
| BART large | informal-to-formal, formal-to-informal |
| T5 small | informal-to-formal, formal-to-informal |
| T5 base | informal-to-formal, formal-to-informal |
| T5 large | informal-to-formal, formal-to-informal |
| BERT base | style classifier |
GYAFC dataset (Entertainment & Music)
| model | checkpoint |
|---|---|
| BART base | informal-to-formal, formal-to-informal |
| BART large | informal-to-formal, formal-to-informal |
| T5 small | informal-to-formal, formal-to-informal |
| T5 base | informal-to-formal, formal-to-informal |
| T5 large | informal-to-formal, formal-to-informal |
| BERT base | style classifier |
Sentiment transfer
Yelp dataset
| model | checkpoint |
|---|---|
| BART base | negative-to-positive, positive-to-negative |
| BART large | negative-to-positive, positive-to-negative |
| T5 small | negative-to-positive, positive-to-negative |
| T5 base | negative-to-positive, positive-to-negative |
| T5 large | negative-to-positive, positive-to-negative |
| BERT base | style classifier |
Model Description
The models implement a CycleGAN architecture for Text Style Transfer that:
- Applies self-supervision directly at sequence level
- Maintains content while transferring style attributes
- Employs pre-trained style classifiers to guide generation
- Uses Transformer-based generators and discriminators
The models achieve state-of-the-art results on both formality and sentiment transfer tasks.
Usage
Both generators and style classifiers can be used with the Hugging Face 🤗 transformers library:
Each generator model can be loaded as:
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
model = AutoModelForSeq2SeqLM.from_pretrained("[GENERATOR_MODEL]")
tokenizer = AutoTokenizer.from_pretrained("[GENERATOR_MODEL]")
The style classifiers can be loaded as:
from transformers import AutoModelForSequenceClassification, AutoTokenizer
classifier = AutoModelForSequenceClassification.from_pretrained("[CLASSIFIER_MODEL]")
tokenizer = AutoTokenizer.from_pretrained("[CLASSIFIER_MODEL]")
Citation
For more details, you can refer to the paper.
@article{10.1145/3678179,
author = {La Quatra, Moreno and Gallipoli, Giuseppe and Cagliero, Luca},
title = {Self-supervised Text Style Transfer Using Cycle-Consistent Adversarial Networks},
year = {2024},
issue_date = {October 2024},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {15},
number = {5},
issn = {2157-6904},
url = {https://doi.org/10.1145/3678179},
doi = {10.1145/3678179},
journal = {ACM Trans. Intell. Syst. Technol.},
month = nov,
articleno = {110},
numpages = {38},
keywords = {Text Style Transfer, Sentiment transfer, Formality transfer, Cycle-consistent Generative Adversarial Networks, Transformers}
}
Code
The full implementation is available at: https://github.com/gallipoligiuseppe/TST-CycleGAN.
License
This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.