is this the llama-3-8b model clone?

#1
by malhajar - opened

No, this is false llama3

Unsloth AI org

:D

Yes, it is the clone but it is specifically designed for Unsloth users to train it 2xfaster with 60% less memory etc

Unsloth AI org

No, this is false llama3

Sorry what is this supposed to mean? :)

Unsloth AI org

Yes it's a clone :) But no gated access, and works seamlessly for Unsloth users

Am just throw a false alarm in case author didn't response.

Performance seems worse than llama2-7b

This is base model

Unsloth AI org

This is base model

correct it is the base model!

Unsloth AI org

Performance seems worse than llama2-7b

Really? Do you happen to see in which areas?

The performance of 8B Instruct models (unsloth) are much better than their llama2 counterpath, however logical reasoning is still not so good, for example asking which is heavier 1 kg feather or 2 kg feather yields wrong output.

The hash code of the files between https://hf-mirror.com/unsloth/llama-3-8b/ and https://huggingface.co/meta-llama/Meta-Llama-3-8B/ is different,
so if you sure that it's a clone?
@shimmyshimmer

Unsloth AI org
β€’
edited May 23

@upupbug Oh sorry actually there is a difference - our base model trained the <eot> and <start_header> tokens since it was untrained in llama-3 base. We only editted the lm_head and embed_tokens for these 2 tokens

Unsloth AI org

@upupbug I reuploaded it and removed our changes! Instead I might manually edit the tokens in a future release

Sign up or log in to comment