Qwen/Qwen3.5-397B-A17B
Image-Text-to-Text • 403B • Updated
• 303k • • 951
Van os
Vanos007
AI & ML interests
None yet
Recent Activity
liked
a model 6 days ago
Qwen/Qwen3.5-397B-A17B liked
a model 6 days ago
MiniMaxAI/MiniMax-M2.5 liked
a Space 7 months ago
Vanos007/jenkins_ci_cd Organizations
liked 2 models 6 days ago
liked a Space 7 months ago
published a Space 7 months ago
liked a model 7 months ago
liked a model 9 months ago
upvoted a collection 11 months ago
reacted to lianghsun's post with 👍 11 months ago
Post
3712
With the arrival of Twinkle April — Twinkle AI’s annual open-source celebration held every April — our community is excited to unveil its very first project:
📊 Twinkle Eval (https://github.com/ai-twinkle/Eval), a next-generation evaluation tool led by our contributor @tedslin .
Unlike traditional evaluation tools like iKala’s ievals (https://github.com/ikala-ai/ievals), which can only evaluate language models (LMs) one sample at a time, Twinkle Eval is designed with Large Reasoning Models (LRMs) in mind. As reasoning time increases with more complex models, traditional tools become increasingly inefficient 😲 — for example, evaluating LRMs on the ikala/tmmluplus benchmark could take *
half a day without finishing.
One question we were especially curious about:
Does shuffling multiple-choice answer order impact model accuracy? 🤔
→ See: "Change Answer Order Can Decrease MMLU Accuracy" – arXiv:2406.19470v1
To address these challenges, Twinkle Eval brings three key innovations to the table:
1️⃣ Parallelized evaluation of samples
2️⃣ Multi-round testing for stability
3️⃣ Randomized answer order to test robustness
After running experiments, we observed that Twinkle Eval can speed up evaluation by up to 15× 🚀🚀. Interestingly, most models scored slightly lower under the 2️⃣3️⃣ test settings compared to their claimed performance — suggesting further benchmarking is needed.
This framework also comes with additional tunable parameters and detailed logging of LM behavior per question — perfect for those who want to dive deeper. 😆
If you find Twinkle Eval useful, please ⭐ the project and help spread the word 🤗
- 5 replies
liked a dataset 11 months ago
liked a Space 11 months ago
liked 3 datasets 12 months ago
liked a Space 12 months ago
liked 4 models 12 months ago
reacted to prithivMLmods's post with 😎 about 1 year ago
Post
4312
QwQ Edge Gets a Small Update..! 💬
GitHub : https://github.com/PRITHIVSAKTHIUR/QwQ-Edge
🚀Now, you can use the following commands for different tasks:
🖼️ @image 'prompt...' → Generates an image
🔉@tts1 'prompt...' → Generates speech in a female voice
🔉 @tts2 'prompt...' → Generates speech in a male voice
🅰️@text 'prompt...' → Enables textual conversation (If not specified, text-to-text generation is the default mode)
💬Multimodality Support : prithivMLmods/Qwen2-VL-OCR-2B-Instruct
💬For text generation, the FastThink-0.5B model ensures quick and efficient responses, prithivMLmods/FastThink-0.5B-Tiny
💬Image Generation: sdxl lightning model, SG161222/RealVisXL_V4.0_Lightning
Github: https://github.com/PRITHIVSAKTHIUR/QwQ-Edge
GitHub : https://github.com/PRITHIVSAKTHIUR/QwQ-Edge
🚀Now, you can use the following commands for different tasks:
🖼️ @image 'prompt...' → Generates an image
🔉@tts1 'prompt...' → Generates speech in a female voice
🔉 @tts2 'prompt...' → Generates speech in a male voice
🅰️@text 'prompt...' → Enables textual conversation (If not specified, text-to-text generation is the default mode)
💬Multimodality Support : prithivMLmods/Qwen2-VL-OCR-2B-Instruct
💬For text generation, the FastThink-0.5B model ensures quick and efficient responses, prithivMLmods/FastThink-0.5B-Tiny
💬Image Generation: sdxl lightning model, SG161222/RealVisXL_V4.0_Lightning
Github: https://github.com/PRITHIVSAKTHIUR/QwQ-Edge
graph TD
A[User Interface] --> B[Chat Logic]
B --> C{Command Type}
C -->|Text| D[FastThink-0.5B]
C -->|Image| E[Qwen2-VL-OCR-2B]
C -->|@image| F[Stable Diffusion XL]
C -->|@tts| G[Edge TTS]
D --> H[Response]
E --> H
F --> H
G --> Hliked a model about 1 year ago
