GAIR/Anole-7b-v0.1

Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation

Anole is the first open-source, autoregressive, and natively trained large multimodal model capable of interleaved image-text generation (without using stable diffusion). While it builds upon the strengths of Chameleon, Anole excels at the complex task of generating coherent sequences of alternating text and images. Through an innovative fine-tuning process using a carefully curated dataset of approximately 6,000 images, Anole achieves remarkable image generation and understanding capabilities with minimal additional training. This efficient approach, combined with its open-source nature, positions Anole as a catalyst for accelerated research and development in multimodal AI. Preliminary tests demonstrate Anole's exceptional ability to follow nuanced instructions, producing high-quality images and interleaved text-image content that closely aligns with user prompts.

The major functionalities of Anole are listed below:

• Text-to-Image Generation
• Interleaved Text-Image Generation
• Text Generation
• MultiModal Understanding

where Bold represents newly added capabilities on the basis of Chameleon.

Please refer to our github repo and paper for examples generated by Anole!

Citation

@article{chern2024anole,
  title={ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation},
  author={Chern, Ethan and Su, Jiadi and Ma, Yan and Liu, Pengfei},
  journal={arXiv preprint arXiv:2407.06135},
  year={2024}
}