meditsolutions/Llama-3.2-SUN-HDIC-1B-Instruct
Text Generation
•
Updated
•
189
MedIT Solutions
company
Verified
AI & ML interests
None defined yet.
Recent Activity
View all activity
meditsolutions's activity
mkurman
updated
a
model
21 days ago
Post
288
How Do I Contribute (HDIC)
Exciting times to come? We are working on a layer self-esteem technique to score their contribution to the final prediction. For now, it unlocks a lot of knowledge already stored in weights we couldn't force the model to extract by further fine-tuning!
Exciting times to come? We are working on a layer self-esteem technique to score their contribution to the final prediction. For now, it unlocks a lot of knowledge already stored in weights we couldn't force the model to extract by further fine-tuning!
Post
429
What AI-enhanced research tools would you recommend for searching and analyzing scientific papers?
-
- 5 replies
Adding Evaluation Results
#1 opened 30 days ago
by
leaderboard-pr-bot
Post
1177
We built a new small language model SmolLM2-MedIT-Upscale-2B, based on SmolLM2-1.7B-Instruct from Hugging Face. The premise was simple - increasing the vector in attention layers would positively impact the model's capabilities.
What did we prove?
In total, not much really, since we don't have the original trained under the same conditions as our upscale. However...
1. We scaled up the model without losing its quality
2. We confirmed that the method we devised works
3. After extremely short fine-tuning, the model achieved much better results in IFEval compared to the original (53.68 vs 64.29) and a higher overall average score in Open LLM Leaderboard (14.75 vs 15.17)
I consider this a big success 😇, since surpassing the original in metrics is often very time-consuming, generates high costs, and doesn't always work out.
Meanwhile, we're moving forward, training SmolLM2 400M Instruct as an upscale of 136M.
We're curious about how increasing the base and intermediate vectors will affect the model's quality. We'll compare it to the original and the 360M Instruct version released by Hugging Face.
License: Apache 2.0
meditsolutions/SmolLM2-MedIT-Upscale-2B
What did we prove?
In total, not much really, since we don't have the original trained under the same conditions as our upscale. However...
1. We scaled up the model without losing its quality
2. We confirmed that the method we devised works
3. After extremely short fine-tuning, the model achieved much better results in IFEval compared to the original (53.68 vs 64.29) and a higher overall average score in Open LLM Leaderboard (14.75 vs 15.17)
I consider this a big success 😇, since surpassing the original in metrics is often very time-consuming, generates high costs, and doesn't always work out.
Meanwhile, we're moving forward, training SmolLM2 400M Instruct as an upscale of 136M.
We're curious about how increasing the base and intermediate vectors will affect the model's quality. We'll compare it to the original and the 360M Instruct version released by Hugging Face.
License: Apache 2.0
meditsolutions/SmolLM2-MedIT-Upscale-2B
mkurman
updated
a
model
about 1 month ago
Adding Evaluation Results
#1 opened about 1 month ago
by
mkurman
mkurman
updated
a
model
about 1 month ago
mkurman
updated
a
collection
about 1 month ago
mkurman
updated
a
collection
about 2 months ago
mkurman
updated
2
models
about 2 months ago
Adding Evaluation Results
#1 opened about 2 months ago
by
leaderboard-pr-bot
mkurman
updated
a
model
about 2 months ago
Post
715
We are happy to introduce MedIT SUN 1B, a downscaled version of the MedIT SUN 2.5B Llama 3.2 variant.
Give it a try!
meditsolutions/Llama-3.2-SUN-1B-chat
Give it a try!
meditsolutions/Llama-3.2-SUN-1B-chat
mkurman
updated
a
collection
about 2 months ago
mkurman
updated
a
model
about 2 months ago
mkurman
updated
3
models
2 months ago