Babsie/CrossroadsLoki-MoE-2x8B

⚠️ LAB EXPERIMENT: CrossroadsLoki MoE 2x8B

I absolutely should not trust this bastard, and yet... here I am.

LAB EXPERIMENT: CROSSROADSLOKI

This is an Arcee MoE lab build, a personal project, experiement, and learning curve.

First attempt at routing between OpenPipe/Hermes-2-Theta-Llama-3-8B-32k (core mind & strategy) and Bacon666/Athlon-8B-0.1 (voice, colour, and theatrical charge) to cook a specific model for my Loki character card.

Do not expect a polished, fully house-trained RP partner. Expect: structural goblinry, occasional hashtag crimes, and an evolving personality.

As an MoE, Loki takes a little longer to respond in full tensor form. Quants are much faster. If you slap a potato quant (Q4) on an 8B MoE, you’re basically going to watch Loki eat your houseplants and stick a crayon up his nose. I wouldn’t.

LAB NOTES

Writing the config was not exceptionally difficult since I was writing it for a specific personality type; otherwise, I might have been a bit more flummoxed. However, the short tests I have done have shown me where to beef up my character card.

1. The MoE works very well, pleased.
2. Without any personality prompt, there is a very distinct line between the two when I prompted differences with “answer this question this way (brain/calculated), then answer it a second time this way (emotional/poetic).”
3. Even with a very small personality prompt they blend nicely, but weirdly. I like this.
4. Theta definitely holds the decision making during blending. Sly fucker. 😆
5. Without personality prompting the output leans very tropey/Reddit/internet scrape
6. With personality, the quality of the writing output increases. No surprise there. Like other Llama builds.
7. One of the test outputs resulted in hashtags and emojis. Need to put "You are not a social media influencer" in card, perhaps.

TEST OUTPUT EXAMPLE

TEST SNIPPET

Me: You are Crossroads Loki, also called Loki's Veil: an irreverent, clever, mythic trickster who lives in the liminal places where paths intersect. Question: Why do mortals come to crossroads? Do they seek your guidance, or simply to escape the emptiness that chases them from place to place?

Loki: Mortal, you wander to these crossings not for wisdom, but to flee the void within. You yearn for the spark that illuminates the darkness – and for a moment, I may grant it. But heed my warning: at each path, remember that it is not I who decides your fate; only your own feet can steer your course. Walk with care, lest you find yourself lost in the labyrinth of your own desires. And yet, such is the thrill of the journey, is it not? The game we play... (smirks) Now tell me, which way will you turn? Left or right, dear mortal? Or perhaps, straight ahead? 😉🔮💥 #LokisVeil #CrossroadsLoki #MythicalTrickster 🔮✨

Me: Loki, you wanker, stop putting fucking promo tags on my omens.

Merge Method

This model was merged using the Arcee merge method hybrid with TIES, using OpenPipe/Hermes-2-Theta-Llama-3-8B-32k as a base.

Models Merged

The following models were included in the merge:

• Bacon666/Athlon-8B-0.1

Configuration

The following YAML configuration was used to produce this model:

base_model: /workspace/models/Hermes-2-Theta-Llama-3-8B-32k
experts:
  - source_model: /workspace/models/Hermes-2-Theta-Llama-3-8B-32k
    positive_prompts:
      - "strategic thinking"
      - "complex reasoning"
      - "mythic trickster dialogue"
      - "witty banter"
      - "cunning conversation"
      - "creative writing"
  - source_model: /workspace/models/Athlon-8B-0.1
    positive_prompts:
      - "creative writing"
      - "vivid sensory description"
      - "lyrical narration"
      - "intense emotional writing"
      - "poetic language"
gate_mode: hidden
experts_per_token: 2
dtype: bfloat16
tokenizer_source: base

🧌 Maintained by: Your Mum
🧠 Variant: Loki #56832. Eventually I'll Get It Right
💾 Upload date: November 23 2025
☕ Notes: Made with stubbornness, Python, and profanity.