huihui-ai

/

Llama-3.3-70B-Instruct-abliterated

like 13

I followed Sumandora/remove-refusals-with-transformers, too. And I simply used model.save_pretrained() after applying refusal dirs. Abliteration worked smoothly, and I was able to test the model right away with inference.py. But as I tried to load model using vllm, it errored with message like KeyError: 'layers.41.input_layernorm.weight'. I tested multiple models and the error persisted. I would be grateful if you could tell me how you apply abliteration and save the model.