OpenSourceHealth's picture

In a Training Loop 🔄

4 3 4

OpenSourceHealth

Khyatimirani

·

AI & ML interests

Turning clinical knowledge into open AI systems for women’s health.

Recent Activity

updated a dataset 1 day ago

Khyatimirani/pcos-nutrition-natural-compound

reacted to OzTianlu's post with 🤗 1 day ago

O(1) inference is the foundational design of Spartacus-1B-Instruct 🛡️ ! https://huggingface.co/NoesisLab/Spartacus-1B-Instruct We have successfully replaced the KV-cache bottleneck inherent in Softmax Attention with Causal Monoid State Compression. By defining the causal history as a monoid recurrence, , the entire prefix is lossily compressed into a fixed-size state matrix per head. The technical core of this architecture relies on the associativity of the monoid operator: Training: parallel prefix scan using Triton-accelerated JIT kernels to compute all prefix states simultaneously. Inference: True sequential updates. Memory and time complexity per token are decoupled from sequence length. Explicit Causality: We discard RoPE and attention masks. Causality is a first-class citizen, explicitly modeled through learned, content-dependent decay gates. Current zero-shot benchmarks demonstrate that Spartacus-1B-Instruct (1.3B) is already outperforming established sub-quadratic models like Mamba-1.4B and RWKV-6-1.6B on ARC-Challenge (0.3063). Recent integration of structured Chain-of-Thought (CoT) data has further pushed reasoning accuracy to 75%. The "Spartacus" era is about scaling intelligence, not the memory wall ♾️.

new activity 1 day ago

Khyatimirani/pcos-nutrition-natural-compound:[bot] Conversion to Parquet

View all activity

Organizations

Khyatimirani 's models 2

Khyatimirani/pcos-fertility-llama3-8b

Text Generation • 3B • Updated 4 days ago • 58

Khyatimirani/pcos-fertility-llama3-8b-lora

Question Answering • Updated 16 days ago