SpydazWeb_AI_HumanAI_006 Beta

• Developed by: LeroyDyer
• License: apache-2.0
• Finetuned from model : LeroyDyer/SpydazWeb_AI_HumanAI_006 ___ CONTEXT EXTENDED MODEL ___ - Training Step !

This model as traied with extended context ! ( instead of traiing with the normal 2048 - 4096 ) This is largely due to the extended prompting used i traiing , so this time the context was extended to allow for the model generation to be more indepth ! The model was previoulsy merged wirh 512 models with extended context ad has managed to serve log contexts despite dome repeating ! By extending the model again i training although the unsloth manages your maq seq for you , allowing for you to train on less cotext or more context as qell as retainn its original settigns, so here the model was also trained with very high rank and lora alpha pushung as much setting to also acomodate the new sequencne:

this has returned a model wich was not easy to merge and export !!! memeory requirements : but eventually it came !

I will also try to dop the next sets of traiing with these settings despite the slowness !

This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.