metadata

title: README
emoji: 🦀
colorFrom: red
colorTo: yellow
pinned: false

rustformers - Large Language Models in Rust

rustformers is a group that wants to make it easy for Rust developers to access the power of large language models (LLMs). As of June 2023, the focus is on keeping pace with the fast-moving GGML ecosystem - a C library that uses quantization to make it possible to run LLMs on the CPU. In the future, there will be support for other backends that allow running LLMs on GPUs, in Wasm environments, and more.

This 🤗 space will host models vetted by the rustformers team.

If you're a Rust developer and are looking to build with LLMs, keep reading to learn about llm - a Rust crate that is the entry point to the rustformers ecosystem.

`llm`

The llm crate exposes an interface for working with LLMs and provides implementations for many popular model architectures, including BLOOM, GPT-2, GPT-J, GPT-NeoX, LLaMA, and MPT. The llm project includes a simple CLI for interacting with LLMs, as well as examples of how to use llm in a Rust project. To learn more about llm, visit its GitHub repository or check out the official documentation for released versions.

Get in Touch!

We're building a community of enthusiastic developers and would love for you to join! If you're looking to chat, check out our Discord. We also actively monitor GitHub Issues and Discussions.