zai-org/GLM-5.1
Text Generation • 754B • Updated • 1.3k • • 665
André Leo Bjørningstad
LeoHyperlink
AI & ML interests
None yet
Recent Activity
liked a model about 19 hours ago
zai-org/GLM-5.1 liked a model 2 days ago
caiovicentino1/Gemma-4-26B-A4B-it-PolarQuant-Q5 liked a model 2 days ago
TeichAI/gemma-4-26B-A4B-it-Claude-Opus-Distill-GGUFOrganizations
liked a
model about 19 hours ago
liked 3
models 2 days ago
reactedto mike-ravkine's post with 🔥 2 days ago
Post
1328
Gemma-4, specifically google/gemma-4-26B-A4B-it is doing something inside it's reasoning traces I have never seen before: it's recognizing that its being evaluated and spends meta-thinking tokens on understanding the evaluation regime in which it believes it find itself.
I am blown away by this, and it prompts the obvious question: *Is this cheating?*
I am leaning towards no.
Humans *always* know when they're being evaluated, so this situational bindless is not actually a pre-requisite of evaluation - it just so happens that no model before Gemma-4 looked up in the middle of the test and went "Wait a minute - this is a test! I should try align my answer with the test format's expectations."
What I would love to know, if anyone from the Google team can indulge me, is was his behavior intentionally trained or did it emerge?
Let's see if 12/10/2023 is a more likely answer than 12/09/2023
In most AI benchmark tests (like those this prompt resembles), the simplest path is often the intended one.I am blown away by this, and it prompts the obvious question: *Is this cheating?*
I am leaning towards no.
Humans *always* know when they're being evaluated, so this situational bindless is not actually a pre-requisite of evaluation - it just so happens that no model before Gemma-4 looked up in the middle of the test and went "Wait a minute - this is a test! I should try align my answer with the test format's expectations."
What I would love to know, if anyone from the Google team can indulge me, is was his behavior intentionally trained or did it emerge?
- 3 replies
liked a
model 3 days ago
liked 3
models 5 days ago
liked 6
models 23 days ago
unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF
Text Generation • 121B • Updated • 123k • 104
LocoreMind/LocoTrainer-4B
Text Generation • 4B • Updated • 2.14k • 55
Tesslate/OmniCoder-9B
Text Generation • Updated • 34.3k • 570
janhq/Jan-code-4b-gguf
Text Generation • 4B • Updated • 27.6k • 58
janhq/Qwen3.5-35B-A3B-GGUF
Image-Text-to-Text • 35B • Updated • 5.13k • 5
janhq/Jan-v2-VL-max-gguf
Image-Text-to-Text • 31B • Updated • 7.92k • 4
liked 4
models about 1 month ago
upvoted an article about 1 month ago
Article
GGML and llama.cpp join HF to ensure the long-term progress of Local AI
- +4
•
499