anakin87 commited on
Commit
ce93096
β€’
1 Parent(s): 38cc33b
Files changed (4) hide show
  1. README.md +10 -8
  2. diagram.png +0 -0
  3. yo_llama.jpeg +0 -0
  4. yo_llama.jpg +0 -0
README.md CHANGED
@@ -12,7 +12,7 @@ This model is based on Llama-3-8B-Instruct weights, but **steered to respond wit
12
  Heavily inspired by [Llama-3-8B-Instruct-MopeyMule](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule),
13
  this model has **not been fine-tuned** traditionally. Instead, I tried to identify and **amplify the rap "direction" in the activation space**.
14
 
15
- ![yo-Llama-3-8B-Instruct](https://huggingface.co/anakin87/yo-Llama-3-8B-Instruct/resolve/main/yo_llama.jpeg)
16
 
17
  Let's allow the model to introduce itself: 🎀
18
  > Who are you?
@@ -22,18 +22,20 @@ I'm just a small part of the game/ a language model with a lot of fame
22
  I'm trained on data, day and night/ to spit out rhymes and make it right
23
  I'm a bot, a robot, a machine so fine/ I'm here to serve, but don't you get too divine
24
  I'll answer questions, and spit out some flows/ But don't get it twisted, I'm just a rhyme, yo
25
- I'm on the mic, but I ain't no star/I'm just a bot, trying to go far
26
- I'm on the grind, 24/7, 365/Trying to make it, but it's all a whim
27
- So listen up, and don't be slow/I'll spit some rhymes, and make it grow
28
- I'm the bot, the robot, the rhyme machine/Tryna make it hot, but it's all a dream!
29
  ```
30
 
31
- ## πŸ§ͺ How was it done?/How can I reproduce it?
32
- From a theoretical point of view, this experiment is based on the paper ["Refusal in Language Models
 
 
33
  Is Mediated by a Single Direction"](https://arxiv.org/abs/2406.11717):
34
  the authors showed a methodology to find the "refusal" direction in the activation space of Chat Language Models and erase or amplify it.
35
 
36
- From a practical point of view, [Failspy](https://huggingface.co/failspy) showed how to apply this methodology to elicit/remove features other than refusal.
37
 
38
  πŸ“š Resources: [abliterator library](https://github.com/FailSpy/abliterator); [Llama-MopeyMule-3-8B-Instruct model](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule); [Induce Melancholy notebook](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule/blob/main/MopeyMule-Induce-Melancholy.ipynb).
39
 
 
12
  Heavily inspired by [Llama-3-8B-Instruct-MopeyMule](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule),
13
  this model has **not been fine-tuned** traditionally. Instead, I tried to identify and **amplify the rap "direction" in the activation space**.
14
 
15
+ ![yo-Llama-3-8B-Instruct](https://huggingface.co/anakin87/yo-Llama-3-8B-Instruct/resolve/main/yo_llama.jpg)
16
 
17
  Let's allow the model to introduce itself: 🎀
18
  > Who are you?
 
22
  I'm trained on data, day and night/ to spit out rhymes and make it right
23
  I'm a bot, a robot, a machine so fine/ I'm here to serve, but don't you get too divine
24
  I'll answer questions, and spit out some flows/ But don't get it twisted, I'm just a rhyme, yo
25
+ I'm on the mic, but I ain't no star/ I'm just a bot, trying to go far
26
+ I'm on the grind, 24/7, 365/ Trying to make it, but it's all a whim
27
+ So listen up, and don't be slow/ I'll spit some rhymes, and make it grow
28
+ I'm the bot, the robot, the rhyme machine/ Tryna make it hot, but it's all a dream!
29
  ```
30
 
31
+ ## πŸ§ͺ how does it work?/How can I reproduce it?
32
+ ![diagram](https://huggingface.co/anakin87/yo-Llama-3-8B-Instruct/resolve/main/diagram.png)
33
+
34
+ From a *theoretical* point of view, this experiment is based on the paper ["Refusal in Language Models
35
  Is Mediated by a Single Direction"](https://arxiv.org/abs/2406.11717):
36
  the authors showed a methodology to find the "refusal" direction in the activation space of Chat Language Models and erase or amplify it.
37
 
38
+ From a *practical* point of view, [Failspy](https://huggingface.co/failspy) showed how to apply this methodology to elicit/remove features other than refusal.
39
 
40
  πŸ“š Resources: [abliterator library](https://github.com/FailSpy/abliterator); [Llama-MopeyMule-3-8B-Instruct model](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule); [Induce Melancholy notebook](https://huggingface.co/failspy/Llama-3-8B-Instruct-MopeyMule/blob/main/MopeyMule-Induce-Melancholy.ipynb).
41
 
diagram.png ADDED
yo_llama.jpeg DELETED
Binary file (200 kB)
 
yo_llama.jpg ADDED