initial commit -m

LICENSE.txt

@@ -0,0 +1,84 @@
+# Coqui Public Model License 1.0.0
+https://coqui.ai/cpml.txt
+
+
+This license allows only non-commercial use of a machine learning model and its outputs.
+
+
+## Acceptance
+
+
+In order to get any license under these terms, you must agree to them as both strict obligations and conditions to all your licenses.
+
+
+## Licenses
+
+
+The licensor grants you a copyright license to do everything you might do with the model that would otherwise infringe the licensor's copyright in it, for any non-commercial purpose. The licensor grants you a patent license that covers patent claims the licensor can license, or becomes able to license, that you would infringe by using the model in the form provided by 
+the licensor, for any non-commercial purpose.
+
+
+## Non-commercial Purpose
+
+
+Non-commercial purposes include any of the following uses of the model or its output, but only so far as you do not receive any direct or indirect payment arising from the use of the model or its output.
+
+
+### Personal use for research, experiment, and testing for the benefit of public knowledge, personal study, private entertainment, hobby projects, amateur pursuits, or religious
+observance. 
+
+
+### Use by commercial or for-profit entities for testing, evaluation, or non-commercial research and development. Use of the model to train other models for commercial use is not a non-commercial purpose. 
+
+
+### Use by any charitable organization for charitable purposes, or for testing or evaluation. Use for revenue-generating activity, including projects directly funded by government grants, is not a non-commercial purpose.
+
+
+## Notices
+
+
+You must ensure that anyone who gets a copy of any part of the model, or any modification of the model, or their output, from you also gets a copy of these terms or the URL for them above.
+
+
+## No Other Rights
+
+
+These terms do not allow you to sublicense or transfer any of your licenses to anyone else, or prevent the licensor from granting licenses to anyone else. These terms do not imply
+any other licenses.
+
+
+## Patent Defense
+
+
+If you make any written claim that the model infringes or contributes to infringement of any patent, your licenses for the model granted under these terms ends immediately. If your company makes such a claim, your patent license ends immediately for work on behalf of your company.
+
+
+## Violations
+
+
+The first time you are notified in writing that you have violated any of these terms, or done anything with the model or its output that is not covered by your licenses, your licenses can nonetheless continue if you come into full compliance with these terms, and take practical steps to correct past violations, within 30 days of receiving notice.  Otherwise, all your licenses
+end immediately.
+
+
+## No Liability
+
+
+***As far as the law allows, the model and its output come as is, without any warranty or condition, and the licensor will not be liable to you for any damages arising out of these terms or the use or nature of the model or its output, under any kind of legal claim. If this provision is not enforceable in your jurisdiction, your licenses are void.***
+
+
+## Definitions
+
+
+The **licensor** is the individual or entity offering these terms, and the **model** is the model the licensor makes available under these terms, including any documentation or similar information about the model.
+
+
+**You** refers to the individual or entity agreeing to these terms.
+
+
+**Your company** is any legal entity, sole proprietorship, or other kind of organization that you work for, plus all organizations that have control over, are under the control of, or are under common control with that organization.  **Control** means ownership of substantially all the assets of an entity, or the power to direct its management and policies by vote, contract, or otherwise.  Control can be direct or indirect.
+
+
+**Your licenses** are all the licenses granted to you under these terms.
+
+
+**Use** means anything you do with the model or its output requiring one of your licenses.

README.md

@@ -1,3 +1,86 @@
 ---
-license: cc-by-nc-sa-4.0
+license: other
+license_name: coqui-public-model-license
+license_link: https://coqui.ai/cpml
+library_name: coqui
+pipeline_tag: text-to-speech
+widget:
+  - text: "Abraham said today is a good day to sound like an African"
 ---
+
+# Afro-TTS
+Afro-TTS is the first pan-African accented English speech synthesis system capable of generating speech in 86 African accents. It includes 1000 personas representing the rich phonological diversity across the continent for applications in Education, Public Health, and Automated Content Creation. Afro-TTS lets you clone voices into different African accents by using just a quick 6-second audio clip. 
+The model was adapted from the XTTS model which was developed by [Coqui Studio](https://coqui.ai/).
+
+Read more about this model in our paper: https://arxiv.org/abs/2406.11727
+
+### Features
+- Supports 86 unique African accents
+- Voice cloning with just a 6-second audio clip
+- Emotion and style transfer by cloning
+- Multi-accent English speech generation
+- 24kHz sampling rate for high-quality audio
+
+## Performance
+
+Afro-TTS achieves near ground truth Mean Opinion Scores (MOS) for naturalness and accentedness. Objective and subjective evaluations demonstrated that the model generates natural-sounding accented speech, bridging the current gap in the representation of African voices in speech synthesis.
+
+
+### Languages
+Afro-TTS supports only English languages for now.
+Stay tuned as we continue to add support for more languages. If you have any language requests, feel free to reach out!
+
+### Code
+The code-base for the paper of this model can be found [here](https://github.com/intron-innovation/AfriSpeech-TTS)
+
+
+### License
+This model is licensed under [Coqui Public Model License](https://coqui.ai/cpml). There's a lot that goes into a license for generative models, and you can read more of [the origin story of CPML here](https://coqui.ai/blog/tts/cpml).
+
+### Contact
+Come and join in our Bioramp Community. We're active on [Masakhane Slack Server](https://join.slack.com/t/masakhane-nlp/shared_invite/zt-1zgnxx911-YWvICNas~mpeKDNqiO3r3g) and our [website](https://bioramp.org/).
+You can also mail the authors at sewade.ogun@inria.fr, tobi@intron.io
+
+#### Using Afro-TTS:
+
+Install the Coqui TTS package:
+
+```bash
+pip install TTS
+```
+Run the following code:
+```python
+from scipy.io.wavfile import write
+from TTS.tts.configs.xtts_config import XttsConfig
+from TTS.tts.models.xtts import Xtts
+
+config = XttsConfig()
+config.load_json("intronhealth/afro-tts/config.json")
+model = Xtts.init_from_config(config)
+model.load_checkpoint(config, checkpoint_dir="intronhealth/afro-tts/", eval=True)
+model.cuda()
+
+outputs = model.synthesize(
+    "Abraham said today is a good day to sound like an African.,
+    config,
+    speaker_wav="audios/reference_accent.wav",
+    gpt_cond_len=3,
+    language="en",
+)
+
+write("audios/output.wav", 24000, outputs['wav'])
+
+
+```
+
+### BibTeX entry and citation info.
+```
+@misc{ogun20241000,
+      title={1000 African Voices: Advancing inclusive multi-speaker multi-accent speech synthesis}, 
+      author={Sewade Ogun and Abraham T. Owodunni and Tobi Olatunji and Eniola Alese and Babatunde Oladimeji and Tejumade Afonja and Kayode Olaleye and Naome A. Etori and Tosin Adewumi},
+      year={2024},
+      eprint={2406.11727},
+      archivePrefix={arXiv},
+      primaryClass={id='eess.AS' full_name='Audio and Speech Processing' is_active=True alt_name=None in_archive='eess' is_general=False description='Theory and methods for processing signals representing audio, speech, and language, and their applications. This includes analysis, synthesis, enhancement, transformation, classification and interpretation of such signals as well as the design, development, and evaluation of associated signal processing systems. Machine learning and pattern analysis applied to any of the above areas is also welcome.  Specific topics of interest include: auditory modeling and hearing aids; acoustic beamforming and source localization; classification of acoustic scenes; speaker separation; active noise control and echo cancellation; enhancement; de-reverberation; bioacoustics; music signals analysis, synthesis and modification; music information retrieval;  audio for multimedia and joint audio-video processing; spoken and written language modeling, segmentation, tagging, parsing, understanding, and translation; text mining; speech production, perception, and psychoacoustics; speech analysis, synthesis, and perceptual modeling and coding; robust speech recognition; speaker recognition and characterization; deep learning, online learning, and graphical models applied to speech, audio, and language signals; and implementation aspects ranging from system architecture to fast algorithms.'}
+}
+```

config.json

@@ -0,0 +1,205 @@
+{
+    "output_path": "/srv/storage/talc2@talc-data2.nancy/multispeech/calcul/users/sogun/coqui-ai-TTS/recipes/ljspeech/xtts_v2/run/training",
+    "logger_uri": null,
+    "run_name": "GPT_XTTS_v2.0_AfroTTS_FT",
+    "project_name": "XTTS_trainer",
+    "run_description": "\n            GPT XTTS training\n            ",
+    "print_step": 5000,
+    "plot_step": 5000,
+    "model_param_stats": false,
+    "wandb_entity": null,
+    "dashboard_logger": "tensorboard",
+    "save_on_interrupt": true,
+    "log_model_step": 1000,
+    "save_step": 5000,
+    "save_n_checkpoints": 3,
+    "save_checkpoints": true,
+    "save_all_best": false,
+    "save_best_after": 0,
+    "target_loss": null,
+    "print_eval": true,
+    "test_delay_epochs": 0,
+    "run_eval": true,
+    "run_eval_steps": null,
+    "distributed_backend": "nccl",
+    "distributed_url": "tcp://localhost:54321",
+    "mixed_precision": false,
+    "precision": "fp16",
+    "epochs": 1000,
+    "batch_size": 2,
+    "eval_batch_size": 2,
+    "grad_clip": 0.0,
+    "scheduler_after_epoch": true,
+    "lr": 5e-06,
+    "optimizer": "AdamW",
+    "optimizer_params": {
+        "betas": [
+            0.9,
+            0.96
+        ],
+        "eps": 1e-08,
+        "weight_decay": 0.01
+    },
+    "lr_scheduler": "MultiStepLR",
+    "lr_scheduler_params": {
+        "milestones": [
+            900000,
+            2700000,
+            5400000
+        ],
+        "gamma": 0.5,
+        "last_epoch": -1
+    },
+    "use_grad_scaler": false,
+    "allow_tf32": false,
+    "cudnn_enable": true,
+    "cudnn_deterministic": false,
+    "cudnn_benchmark": false,
+    "training_seed": 1,
+    "model": "xtts",
+    "num_loader_workers": 8,
+    "num_eval_loader_workers": 0,
+    "use_noise_augment": false,
+    "audio": {
+        "sample_rate": 22050,
+        "output_sample_rate": 24000,
+        "dvae_sample_rate": 22050
+    },
+    "use_phonemes": false,
+    "phonemizer": null,
+    "phoneme_language": null,
+    "compute_input_seq_cache": false,
+    "text_cleaner": null,
+    "enable_eos_bos_chars": false,
+    "test_sentences_file": "",
+    "phoneme_cache_path": null,
+    "characters": null,
+    "add_blank": false,
+    "batch_group_size": 64,
+    "loss_masking": null,
+    "min_audio_len": 1,
+    "max_audio_len": Infinity,
+    "min_text_len": 1,
+    "max_text_len": Infinity,
+    "compute_f0": false,
+    "compute_energy": false,
+    "compute_linear_spec": false,
+    "precompute_num_workers": 0,
+    "start_by_longest": false,
+    "shuffle": false,
+    "drop_last": false,
+    "datasets": [
+        {
+            "formatter": "",
+            "dataset_name": "",
+            "path": "",
+            "meta_file_train": "",
+            "ignored_speakers": null,
+            "language": "",
+            "phonemizer": "",
+            "meta_file_val": "",
+            "meta_file_attn_mask": ""
+        }
+    ],
+    "test_sentences": [
+        {
+            "text": "It took me quite a long time to develop a voice, and now that I have it I'm not going to be silent.",
+            "speaker_wav": [
+                "/srv/storage/talc2@talc-data2.nancy/multispeech/calcul/users/sogun/AfriSpeech-TTS/afrispeech_16k_trimmed/AfriSpeech-TTS/train/defc5e03-926c-4e0b-a639-c821e5e7db89/14f64f13c57f9a64a2a1521253934a0b_KYA8MaKS.wav"
+            ],
+            "language": "en"
+        },
+        {
+            "text": "This cake is great. It's so delicious and moist.",
+            "speaker_wav": [
+                "/srv/storage/talc2@talc-data2.nancy/multispeech/calcul/users/sogun/AfriSpeech-TTS/afrispeech_16k_trimmed/AfriSpeech-TTS/train/defc5e03-926c-4e0b-a639-c821e5e7db89/14f64f13c57f9a64a2a1521253934a0b_KYA8MaKS.wav"
+            ],
+            "language": "en"
+        }
+    ],
+    "eval_split_max_size": 256,
+    "eval_split_size": 0.01,
+    "use_speaker_weighted_sampler": false,
+    "speaker_weighted_sampler_alpha": 1.0,
+    "use_language_weighted_sampler": false,
+    "language_weighted_sampler_alpha": 1.0,
+    "use_length_weighted_sampler": false,
+    "length_weighted_sampler_alpha": 1.0,
+    "model_args": {
+        "gpt_batch_size": 1,
+        "enable_redaction": false,
+        "kv_cache": true,
+        "gpt_checkpoint": "",
+        "clvp_checkpoint": null,
+        "decoder_checkpoint": null,
+        "num_chars": 255,
+        "tokenizer_file": "/srv/storage/talc2@talc-data2.nancy/multispeech/calcul/users/sogun/coqui-ai-TTS/recipes/ljspeech/xtts_v2/run/training/XTTS_v2.0_original_model_files/vocab.json",
+        "gpt_max_audio_tokens": 605,
+        "gpt_max_text_tokens": 402,
+        "gpt_max_prompt_tokens": 70,
+        "gpt_layers": 30,
+        "gpt_n_model_channels": 1024,
+        "gpt_n_heads": 16,
+        "gpt_number_text_tokens": 6681,
+        "gpt_start_text_token": 261,
+        "gpt_stop_text_token": 0,
+        "gpt_num_audio_tokens": 1026,
+        "gpt_start_audio_token": 1024,
+        "gpt_stop_audio_token": 1025,
+        "gpt_code_stride_len": 1024,
+        "gpt_use_masking_gt_prompt_approach": true,
+        "gpt_use_perceiver_resampler": true,
+        "input_sample_rate": 22050,
+        "output_sample_rate": 24000,
+        "output_hop_length": 256,
+        "decoder_input_dim": 1024,
+        "d_vector_dim": 512,
+        "cond_d_vector_in_each_upsampling_layer": true,
+        "duration_const": 102400,
+        "min_conditioning_length": 66150,
+        "max_conditioning_length": 132300,
+        "gpt_loss_text_ce_weight": 0.01,
+        "gpt_loss_mel_ce_weight": 1.0,
+        "debug_loading_failures": false,
+        "max_wav_length": 255995,
+        "max_text_length": 300,
+        "mel_norm_file": "/srv/storage/talc2@talc-data2.nancy/multispeech/calcul/users/sogun/coqui-ai-TTS/recipes/ljspeech/xtts_v2/run/training/XTTS_v2.0_original_model_files/mel_stats.pth",
+        "dvae_checkpoint": "/srv/storage/talc2@talc-data2.nancy/multispeech/calcul/users/sogun/coqui-ai-TTS/recipes/ljspeech/xtts_v2/run/training/XTTS_v2.0_original_model_files/dvae.pth",
+        "xtts_checkpoint": "/srv/storage/talc2@talc-data2.nancy/multispeech/calcul/users/sogun/coqui-ai-TTS/recipes/ljspeech/xtts_v2/run/training/GPT_XTTS_v2.0_AfroTTS_FT-March-06-2024_12+32AM-581cf506/checkpoint_30000.pth",
+        "vocoder": ""
+    },
+    "model_dir": null,
+    "languages": [
+        "en",
+        "es",
+        "fr",
+        "de",
+        "it",
+        "pt",
+        "pl",
+        "tr",
+        "ru",
+        "nl",
+        "cs",
+        "ar",
+        "zh-cn",
+        "hu",
+        "ko",
+        "ja",
+        "hi"
+    ],
+    "temperature": 0.85,
+    "length_penalty": 1.0,
+    "repetition_penalty": 2.0,
+    "top_k": 50,
+    "top_p": 0.85,
+    "num_gpt_outputs": 1,
+    "gpt_cond_len": 12,
+    "gpt_cond_chunk_len": 4,
+    "max_ref_len": 10,
+    "sound_norm_refs": false,
+    "optimizer_wd_only_on_weights": true,
+    "weighted_loss_attrs": {},
+    "weighted_loss_multipliers": {},
+    "github_branch": "* dev"
+}

dvae.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b29bc227d410d4991e0a8c09b858f77415013eeb9fba9650258e96095557d97a
+size 210514388

mel_stats.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f69422a8a8f344c4fca2f0c6b8d41d2151d6615b7321e48e6bb15ae949b119c
+size 1067

model.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e90d054911b2fe01f85194ff7c4c5bfb63fd2104bc8995afe04ba392e0038415
+size 5607927893