Redwood0's picture
Update README.md
6ca73cb
metadata
license: cc-by-nc-4.0

Calibration dataset: VMware Open Instruct, 4096

Dumb assistant, 🔥 author.

4-bit Examples with Alpaca

!!NSFW!! - 🔥Erotica Writing Example🔥 - !!NSFW!!

Thanks to Charles Goddard for the recipe.

The idea here is to "move" Iambe from being based on vanilla L2 to being based on sequelbox/DynamicFactor instead.

Because task_arithmetic uses the raw deltas, this should be similar to if the SFT had been done over DynamicFactor.

Recipe

merge_method: task_arithmetic

base_model: athirdpath/BigLlama-20b-v1.1 # Base model you want to "move out" from

models:

  • model: athirdpath/Iambe-20b-DARE-v2 # SFTd model you want to transfer

  • model: athirdpath/DoubleFactor-20b # Base model you want to "move in" to

parameters:

  • weight: 1.0

dtype: bfloat16