Hugging Face's logo

microsoft
/
Phi-3-small-8k-instruct

Text Generation
Transformers
Safetensors
multilingual
phi3small
nlp
code
conversational
custom_code
Model card Files Community
31

Ensure the query_states and key_states remain in bf16

#21
by mrm196 - opened
base: refs/heads/main
←
from: refs/pr/21
Discussion Files changed
+2
-2
mrm196

Applying the fix suggested from this thread.
Basically, making sure that after applying rotary embedding the data type does not change.

Ensure the query_states and key_states remain in bf1655b8e963
bapatra
 Microsoft org

LGTM. Thank you !

bapatra changed pull request status to merged

Sign up or log in to comment