AI & ML interests
Researching and building foundation models with improved generalization and reasoning. LAION & friends spin-off for open-sourcing foundation models with strong generalization and reasoning , including datasets necessary for their creation, to serve as common open, reproducible grounds for further research experiments.
-
open-sci/open-sci-ref-v0.01-0.13b-fineweb-edu-1.4t-300B-4096
0.1B • Updated • 73 -
open-sci/open-sci-ref-v0.01-0.4b-fineweb-edu-1.4t-300B-4096
0.4B • Updated • 10 -
open-sci/open-sci-ref-v0.01-1.3b-fineweb-edu-1.4t-300B-4096
1B • Updated • 84 -
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-1T-4096
2B • Updated • 105
Research baseline models trained on various open reference datasets
Open-sci-ref: reference baselines releases
-
open-sci/open-sci-ref-v0.01-0.13b-commoncorpus-300B-4096
0.1B • Updated • 58 -
open-sci/open-sci-ref-v0.01-0.4b-commoncorpus-300B-4096
0.4B • Updated • 8 -
open-sci/open-sci-ref-v0.01-1.3b-commoncorpus-300B-4096
1B • Updated • 9 -
open-sci/open-sci-ref-v0.01-1.7b-commoncorpus-300B-4096
2B • Updated • 76 • 1
-
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000-lr0.006-2
0.1B • Updated • 80 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000
0.4B • Updated • 48 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000-lr0.004-2
0.4B • Updated • 73 -
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000
0.1B • Updated • 8
-
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096-rope_theta-100k
2B • Updated • 14 -
open-sci/open-sci-ref-v0.01-0.13b-nemotron-hq-300B-4096
0.1B • Updated • 77 -
open-sci/open-sci-ref-v0.01-0.4b-nemotron-hq-300B-4096
0.4B • Updated • 11 -
open-sci/open-sci-ref-v0.01-1.3b-nemotron-hq-300B-4096
1B • Updated • 83
openMammut models trained on various datasets (Re-LAION, DataComp, DFN)
-
laion/openMaMMUT-ViT-L-14-DataComp-1.4B-s12.8B-b180K
Zero-Shot Image Classification • Updated • 24 • 5 -
laion/openMaMMUT-ViT-B-32-512x512-pt_DFN2B-ft_DFN512x512-s293M-b73k
Zero-Shot Image Classification • Updated • 29 -
laion/openMaMMUT-ViT-B-16-512x512-pt_DFN2B-ft_DFN512x512-s293M-b73k
Zero-Shot Image Classification • Updated • 10
Materials related to OpenThoughts and OpenThinker releases
-
open-sci/open-sci-ref-v0.01-0.13b-commoncorpus-300B-4096
0.1B • Updated • 58 -
open-sci/open-sci-ref-v0.01-0.4b-commoncorpus-300B-4096
0.4B • Updated • 8 -
open-sci/open-sci-ref-v0.01-1.3b-commoncorpus-300B-4096
1B • Updated • 9 -
open-sci/open-sci-ref-v0.01-1.7b-commoncorpus-300B-4096
2B • Updated • 76 • 1
-
open-sci/open-sci-ref-v0.01-0.13b-fineweb-edu-1.4t-300B-4096
0.1B • Updated • 73 -
open-sci/open-sci-ref-v0.01-0.4b-fineweb-edu-1.4t-300B-4096
0.4B • Updated • 10 -
open-sci/open-sci-ref-v0.01-1.3b-fineweb-edu-1.4t-300B-4096
1B • Updated • 84 -
open-sci/open-sci-ref-v0.01-1.7b-fineweb-edu-1.4t-1T-4096
2B • Updated • 105
-
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000-lr0.006-2
0.1B • Updated • 80 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000
0.4B • Updated • 48 -
open-sci/open-sci-ref-v0.01-0.4b-c4-300B-4096-warmup25000-lr0.004-2
0.4B • Updated • 73 -
open-sci/open-sci-ref-v0.01-0.13b-c4-300B-4096-warmup25000
0.1B • Updated • 8
-
open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096-rope_theta-100k
2B • Updated • 14 -
open-sci/open-sci-ref-v0.01-0.13b-nemotron-hq-300B-4096
0.1B • Updated • 77 -
open-sci/open-sci-ref-v0.01-0.4b-nemotron-hq-300B-4096
0.4B • Updated • 11 -
open-sci/open-sci-ref-v0.01-1.3b-nemotron-hq-300B-4096
1B • Updated • 83
Research baseline models trained on various open reference datasets
openMammut models trained on various datasets (Re-LAION, DataComp, DFN)
-
laion/openMaMMUT-ViT-L-14-DataComp-1.4B-s12.8B-b180K
Zero-Shot Image Classification • Updated • 24 • 5 -
laion/openMaMMUT-ViT-B-32-512x512-pt_DFN2B-ft_DFN512x512-s293M-b73k
Zero-Shot Image Classification • Updated • 29 -
laion/openMaMMUT-ViT-B-16-512x512-pt_DFN2B-ft_DFN512x512-s293M-b73k
Zero-Shot Image Classification • Updated • 10
Open-sci-ref: reference baselines releases
Materials related to OpenThoughts and OpenThinker releases