metadata
library_name: keras-nlp
pipeline_tag: text-generation
This is a Phi3 model uploaded using the KerasNLP library and can be used with JAX, TensorFlow, and PyTorch backends.
This model is related to a CausalLM task.
Model config:
- name: phi3_backbone_1
- trainable: True
- vocabulary_size: 32064
- num_layers: 32
- num_query_heads: 32
- hidden_dim: 3072
- intermediate_dim: 8192
- num_key_value_heads: 32
- layer_norm_epsilon: 1e-05
- dropout: 0.0
- max_sequence_length: 4096
- pretraining_sequence_length: 4096
- rope_max_wavelength: 10000.0
- rope_scaling_type: None
- rope_scaling_short_factor: None
- rope_scaling_long_factor: None
This model card has been generated automatically and should be completed by the model author. See Model Cards documentation for more information.