Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model:
|
| 4 |
+
- Qwen/Qwen2.5-14B
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
<div align="center">
|
| 8 |
+
<img src="https://i.ibb.co/pXD6Bcv/SW2-U-g-QQLSH1-ZAbxhs-Iu-A.webp" alt="Virtuoso-Small" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
|
| 9 |
+
</div>
|
| 10 |
+
|
| 11 |
+
# Virtuoso-Small
|
| 12 |
+
|
| 13 |
+
Virtuoso-Small is the debut public release of the Virtuoso series of models by Arcee.ai, designed to bring cutting-edge generative AI capabilities to organizations and developers in a compact, efficient form. With 14 billion parameters, Virtuoso-Small is an accessible entry point for high-quality instruction-following, complex reasoning, and business-oriented generative AI tasks. Its larger siblings, Virtuoso-Forte and Virtuoso-Prime, offer even greater capabilities and are available via API at [models.arcee.ai](https://models.arcee.ai).
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
## Performance Benchmarks
|
| 17 |
+
|
| 18 |
+
| **Groups** | **Metric** | ↑ | **Value** | ± | **Stderr** |
|
| 19 |
+
|---------------------------|--------------------------|---|----------:|----|-----------:|
|
| 20 |
+
| **Leaderboard** | **Accuracy** | ↑ | 0.5194 | ± | 0.0046 |
|
| 21 |
+
| | Normalized Accuracy | ↑ | 0.5814 | ± | 0.0051 |
|
| 22 |
+
| | Exact Match | ↑ | 0.3006 | ± | 0.0117 |
|
| 23 |
+
| | Instruction-Level Loose Accuracy | ↑ | 0.8489 | ± | N/A |
|
| 24 |
+
| | Instruction-Level Strict Accuracy | ↑ | 0.8249 | ± | N/A |
|
| 25 |
+
| | Prompt-Level Loose Accuracy | ↑ | 0.7856 | ± | 0.0177 |
|
| 26 |
+
| | Prompt-Level Strict Accuracy | ↑ | 0.7523 | ± | 0.0186 |
|
| 27 |
+
| **Leaderboard-BBH** | Normalized Accuracy | ↑ | 0.6516 | ± | 0.0058 |
|
| 28 |
+
| **Leaderboard-GPQA** | Normalized Accuracy | ↑ | 0.3389 | ± | 0.0137 |
|
| 29 |
+
| **Leaderboard-Math-Hard** | Exact Match | ↑ | 0.3006 | ± | 0.0117 |
|
| 30 |
+
| **Leaderboard-MuSR** | Normalized Accuracy | ↑ | 0.4286 | ± | 0.0175 |
|
| 31 |
+
|
| 32 |
+
---
|
| 33 |
+
|
| 34 |
+
## Key Features
|
| 35 |
+
|
| 36 |
+
- **Compact and Efficient**: With 14 billion parameters, Virtuoso-Small provides a high-performance solution optimized for smaller hardware configurations without sacrificing quality.
|
| 37 |
+
- **Business-Oriented**: Tailored for use cases such as customer support, content creation, and technical assistance, Virtuoso-Small meets the demands of modern enterprises.
|
| 38 |
+
- **Scalable Ecosystem**: Part of the Virtuoso series, Virtuoso-Small is fully interoperable with its larger siblings, Forte and Prime, enabling seamless scaling as your needs grow.
|
| 39 |
+
|
| 40 |
+
---
|
| 41 |
+
|
| 42 |
+
## Deployment Options
|
| 43 |
+
|
| 44 |
+
Virtuoso-Small is available under the Apache-2.0 license and can be deployed locally or accessed through an API at [models.arcee.ai](https://models.arcee.ai). For larger-scale or more demanding applications, consider Virtuoso-Forte or Virtuoso-Prime.
|
| 45 |
+
|