--- base_model: - starnet/19star03 - lionking927/s11-m1-0303 - starnet/15star03 library_name: transformers tags: - mergekit - merge --- # output This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the [TIES](https://arxiv.org/abs/2306.01708) merge method using [starnet/15star03](https://huggingface.co/starnet/15star03) as a base. ### Models Merged The following models were included in the merge: * [starnet/19star03](https://huggingface.co/starnet/19star03) * [lionking927/s11-m1-0303](https://huggingface.co/lionking927/s11-m1-0303) ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: starnet/15star03 parameters: density: [1, 0.7, 0.1] # density gradient weight: 1.0 - model: starnet/19star03 parameters: density: 0.5 weight: [0, 0.3, 0.7, 1] # weight gradient - model: lionking927/s11-m1-0303 parameters: density: 0.33 weight: - filter: mlp value: 0.5 - value: 0 merge_method: ties base_model: starnet/15star03 parameters: normalize: true int8_mask: true dtype: float16 ```