Update README.md
Browse files
README.md
CHANGED
|
@@ -10,7 +10,7 @@ language:
|
|
| 10 |
---
|
| 11 |
a third experiment to train only on synthetic messages!
|
| 12 |
|
| 13 |
-
- parameters: 40.6M (13.11 mlp, 10.49 embed, 10.49 head,
|
| 14 |
- tokens seen: 975.2M
|
| 15 |
- num_layers: 16
|
| 16 |
- num_heads: 8
|
|
|
|
| 10 |
---
|
| 11 |
a third experiment to train only on synthetic messages!
|
| 12 |
|
| 13 |
+
- parameters: 40.6M (13.11 mlp, 10.49 embed, 10.49 head, 6.55 attn)
|
| 14 |
- tokens seen: 975.2M
|
| 15 |
- num_layers: 16
|
| 16 |
- num_heads: 8
|