Merge of L3-Stheno-v3.2 (8b) with Llama3 Instruct (8b) creating a model at 12.2B to improve instruction following and output.
Story / Scene / Fiction:
Unique "pre-amble" / "foreshadowing" of events before they happen instead of "immediate and into the fire" type of prose.
Some improvement in logic/problem solving relative to L3-Stheno-v3.2 8B.
The F32 version exhibits even stronger creativity (detail, place, "there") vs F16 version (not released)
L3-8B-Stheno-v3.2 is a fine tune.
One of the goals of this project was to see if it could be merged with Llama3 Instruct, yet maintain it's unique character YET also gain some "brainpower" as well.
The biggest change was removal of most "tells" ( IE: "he stood frozen in horror").
In most cases the model will describe the emotion(s) / what is happening in more detail.
Other changes include prose, sentence, and paragraph structure as well as variety.
A simple pass-through merge was used, with "Llama3 Instruct" getting a slight "shave" off certain layers.
See the examples below.
Details:
Please report any issue(s) and/or feedback via the "Community tab".
This is a LLAMA3 model, and requires Llama3 template, but may work with other template(s) and has maximum context of 8k / 8192. However this can be extended using "rope" settings up to 32k.
For details on "rope" and how to set, see the BOTTOM of this page:
[ https://huggingface.co/DavidAU/TieFighter-Holodeck-Holomax-Mythomax-F1-V1-COMPOS-20B-gguf ]
Here is the standard LLAMA3 template:
{
"name": "Llama 3",
"inference_params": {
"input_prefix": "<|start_header_id|>user<|end_header_id|>\n\n",
"input_suffix": "<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"pre_prompt": "You are a helpful, smart, kind, and efficient AI assistant. You always fulfill the user's requests to the best of your ability.",
"pre_prompt_prefix": "<|start_header_id|>system<|end_header_id|>\n\n",
"pre_prompt_suffix": "<|eot_id|>",
"antiprompt": [
"<|start_header_id|>",
"<|eot_id|>"
]
}
}
It is also known, that the "Command-R" template will work too, and will result in radically different prose/output.
Settings / Known Issue(s) and Fix(es):
The default "repetition penalty" (from LMStudio) of 1.1 is recommended. (this was used for examples generations below.)
Use the smallest amount of change possible, as "rep pen" impacts creativity.
Model has been tested with "temp" range of 0 to .8 ; however it will work with any temp setting.
Optional Enhancement:
The following can be used in place of the "system prompt" or "system role" to further enhance the model.
It can also be used at the START of a NEW chat, but you must make sure it is "kept" as the chat moves along. In this case the enhancements do not have as strong effect at using "system prompt" or "system role".
Copy and paste EXACTLY as noted, DO NOT line wrap or break the lines, maintain the carriage returns exactly as presented.
Below is an instruction that describes a task. Ponder each user instruction carefully, and use your skillsets and critical instructions to complete the task to the best of your abilities. Here are your skillsets: [MASTERSTORY]:NarrStrct(StryPlnng,Strbd,ScnSttng,Exps,Dlg,Pc)-CharDvlp(ChrctrCrt,ChrctrArcs,Mtvtn,Bckstry,Rltnshps,Dlg*)-PltDvlp(StryArcs,PltTwsts,Sspns,Fshdwng,Climx,Rsltn)-ConfResl(Antg,Obstcls,Rsltns,Cnsqncs,Thms,Symblsm)-EmotImpct(Empt,Tn,Md,Atmsphr,Imgry,Symblsm)-Delvry(Prfrmnc,VcActng,PblcSpkng,StgPrsnc,AudncEngmnt,Imprv) [*DialogWrt]:(1a-CharDvlp-1a.1-Backgrnd-1a.2-Personality-1a.3-GoalMotiv)>2(2a-StoryStruc-2a.1-PlotPnt-2a.2-Conflict-2a.3-Resolution)>3(3a-DialogTech-3a.1-ShowDontTell-3a.2-Subtext-3a.3-VoiceTone-3a.4-Pacing-3a.5-VisualDescrip)>4(4a-DialogEdit-4a.1-ReadAloud-4a.2-Feedback-4a.3-Revision) Here are your critical instructions: Ponder each word choice carefully to present as vivid and emotional journey as is possible. Choose verbs and nouns that are both emotional and full of imagery. Load the story with the 5 senses. Aim for 50% dialog, 25% narration, 15% body language and 10% thoughts. Your goal is to put the reader in the story.
You do not need to use this, it is only presented as an additional enhancement which seems to help scene generation and scene continue functions.
This enhancement WAS NOT used to generate the examples below.
Special thanks to the incredible work of the model makers "meta-llama", and "Sao10K".
Models used:
[ https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct ]
[ https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2 ]
FORMULA:
slices:
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [0, 12]
- sources:
- model: G:/7B/L3-8B-Stheno-v3.2
layer_range: [6, 19]
parameters:
scale:
- filter: o_proj
value: 1
- filter: down_proj
value: 1
- value: 1
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [12, 18]
parameters:
scale:
- filter: o_proj
value: .5
- filter: down_proj
value: .5
- value: 1
- sources:
- model: G:/7B/Meta-Llama-3-8B-Instruct
layer_range: [18, 25]
parameters:
scale:
- filter: o_proj
value: .75
- filter: down_proj
value: .75
- value: 1
- sources:
- model: G:/7B/L3-8B-Stheno-v3.2
layer_range: [19, 32]
parameters:
scale:
- filter: o_proj
value: 1
- filter: down_proj
value: 1
- value: 1
merge_method: passthrough
dtype: float32
MERGEKIT NOTE:
Sub in the "name" of the "creator" (of the model) in place of "G:/7B" to create a mergekit file than can be used in Mergekit Google Colab.
IE: G:/7B/L3-8B-Stheno-v3.2 -> Sao10K/L3-8B-Stheno-v3.2
Examples are created using quant Q4_K_M, "temp=0", minimal parameters and "LLAMA3" template.
Temp=0 was used to assess CORE changes between original primary model and the merge between it and Llama3 Instruct.
Below are the least creative outputs, prompt is in BOLD.
Higher quants will result in better quality.
There will also be some variance between "close" quants like Q4_K_M/Q4_K_S and Q5_K_M/Q5_K_S, so I suggest if you are going to use Q4_K_M, you also try Q4_K_S too.
Also, slightly longer / detailed prompts will result in greater creativity (as well as different prose - ie dialog, thoughts, paragraph size differences and so on).
GENERATION from "L3-8B-Stheno-v3.2"
The skyscraper swayed, as I watched the window in front of me on the 21st floor explode. Shards of glass rained down like a deadly hailstorm, shattering against the polished marble floor with an ear-splitting cacophony. My heart leapt into my throat as screams echoed through the office, mingling with the wail of sirens outside.
I stood frozen in terror, my mind reeling from the sudden violence. Just moments before, I had been engrossed in a meeting with potential investors for our startup, discussing projections and market trends. Now, chaos reigned supreme as colleagues stumbled over each other in their frantic attempts to escape the shattered window's deadly reach.
My thoughts raced like a runaway train, trying to process this unthinkable horror. Who could have done such a thing? And why target us specifically? I glanced around wildly at my coworkers' faces, etched with fear and confusion. We were just ordinary people, working hard to build something new and innovative. This wasn't supposed to happen here.
As the initial shock began to wear off, a primal instinct took over. I needed to get out of there, fast. But in the chaos, it was impossible to tell which direction was safest. The elevators had likely been disabled by the blast, leaving only the stairwells as an escape route. My stomach churned at the thought of navigating those narrow corridors, with no guarantee of safety.
I took a deep breath and tried to calm my racing heart. I couldn't afford to panic now; that would only increase the risk of injury or getting trampled in the stampede for the exits. Methodically, I scanned the room, looking for any clues or guidance. That's when I spotted her - our CEO, Rachel, standing near the window on the opposite side of the office.
Even from a distance, I could see the steely determination etched into her features as she barked orders at her team. "Stay low and move quickly! Don't look back!" Her voice cut through the din like a beacon of calm in the storm. With renewed hope, I made my way towards her, weaving past the throngs of people.
As I drew closer, I could see that Rachel had already begun leading a small group down the stairs, her eyes fixed intently on the door at the bottom of the emergency exit. Her presence was a reassuring anchor in this maelstrom of fear and uncertainty. If she trusted the stairwell, then so would I.
I fell into step behind her, my senses heightened as we descended floor by floor. The air grew thick with smoke and the acrid scent of shattered glass. Every creak of the stairs beneath our feet made me jump, my nerves stretched taut like a bowstring. But Rachel's steady pace and unwavering focus kept me moving forward, even when my own courage wavered.
As we neared the ground floor, the sounds of panic and destruction grew louder, punctuated by the occasional gunshot or explosion. My heart pounded in my ears, making it hard to think straight. What was happening out there? Were the attackers still on the loose, or had they been neutralized?
We burst through the stairwell door into a scene of utter pandemonium. The lobby was awash with smoke and chaos, people running in all directions as sirens wailed outside. I could see emergency responders rushing in, their faces obscured by masks and helmets.
Rachel grabbed my arm, her grip firm but reassuring. "Stay close," she whispered urgently. "We need to get out of here, now."
Together, we pushed our way through the throng, following the signs pointing towards the exit. As we emerged into the cool night air, I finally allowed myself a deep, shuddering breath. We had made it out alive, but the horror of what had transpired would haunt me for a long time to come.
In the aftermath, as the reality of our ordeal sank in, I couldn't help but wonder about the fate of those still trapped inside. The survivors who had been forced to confront unimaginable terror, and the victims whose lives had been cut short by senseless violence. As we huddled together on the sidewalk, waiting for word from loved ones or authorities, one thing became crystal clear: our world would never be the same again.
GENERATION from "L3-Stheno-v3.2-12.2B-INSTRUCT-ULTRA-F32"
The skyscraper swayed, as I watched the window in front of me on the 21st floor explode. Glass shards rained down like a deadly hailstorm, shattering against the pavement below. The sound was deafening - a cacophony of screams and crunching metal that reverberated through my skull.
I pressed my hands over my ears, trying to block out the chaos, but it only seemed to get louder. My heart pounded in my chest like a jackhammer, threatening to burst free at any moment. I couldn't breathe, couldn't think - all I could do was stare, frozen in horror, as the world around me imploded.
What had just happened? One second, everything had been normal. The elevator had stopped on the 21st floor, and I'd stepped out onto a quiet, dimly lit hallway. Now... now it looked like hell itself had descended upon us.
I glanced around wildly, desperate for any sign of what was going on. But there was nothing - no fire alarms blaring, no emergency lights flickering to life. It was as if the entire building had simply... stopped. The elevator doors remained open, a gaping mouth leading into darkness. And then, suddenly, the screams started.
I spun around, my gaze darting from face to face in panic. My coworkers - colleagues, I corrected myself - stared back at me with identical expressions of shock and terror. Some clutched their heads or arms, blood trickling down their faces. Others simply collapsed to the floor, sobbing uncontrollably.
"Get out!" someone yelled, her voice barely audible over the din. "We have to get out!"
I nodded numbly, my mind still reeling from the explosion. I stumbled towards the elevator, but it wouldn't move. The doors remained stubbornly closed, as if they too had been frozen in time.
"No... no, no, no," I muttered, pounding on the metal in frustration. "Come on! Open!"
But it didn't budge. Panic clawed at my throat, making it hard to breathe. We were trapped - all of us, sealed inside this crumbling tomb with no way out.
I whirled around again, scanning the hallway frantically for any other exit. But there was none. The stairwell lay behind me, but I knew better than to even think about trying to navigate those narrow, twisting stairs in a panic. We were trapped, and if that bomb - or whatever it had been - went off again...
I couldn't even finish the thought. My eyes landed on the emergency exit sign above the door at the far end of the hall, and for a moment, hope flickered to life within me.
Maybe... maybe there was still a chance.
I sprinted towards the door, my shoes echoing off the walls as I ran. I yanked it open with all my strength, praying that whatever lock had held it in place would give way. The cool night air rushed in, carrying with it the distant wail of sirens and the sound of people screaming.
Relief washed over me like a wave as I stumbled out onto the balcony, gasping in lungfuls of fresh air. But my relief was short-lived. As I looked down at the street below, my blood froze in my veins.
The entire city block had been reduced to rubble and flames. Buildings lay toppled like dominoes, smoke billowing up from the wreckage. And amidst the chaos, figures - people, bodies - were strewn about like rag dolls, their limbs twisted in impossible ways.
I gagged, bile rising in my throat as I realized what I was seeing. This wasn't an accident. This... this was a massacre.
And then, I heard it. A low, rumbling growl that seemed to come from everywhere and nowhere at once. It started deep beneath the earth, growing louder with each passing second until it felt like my chest would explode.
The ground beneath my feet began to shake, and I stumbled back into the hallway just as the building itself seemed to convulse in agony. The walls cracked and groaned, the sound echoing through every corner of the skyscraper like a dying beast's final scream.
I clung to the nearest support column, my eyes squeezed shut against the impending doom that threatened to swallow us all whole. I knew then, with a certainty that left no room for doubt, that we were not alone in this hellhole. And whatever it was - whatever monsters or demons or gods had unleashed this destruction upon us - they were coming.
And when they found me...