hazyresearch
/

M2-BERT-128-Retrieval-Encoder-V1

Model card Files Files and versions

jonsaadfalcon commited on Feb 10, 2024

Commit

1761a9d

·

verified ·

1 Parent(s): 90f1c7e

Update config.json

Files changed (1) hide show

config.json +2 -2

config.json CHANGED Viewed

@@ -28,8 +28,8 @@
     "intermediate_size": 3072,
     "layer_norm_eps": 1e-12,
     "long_conv_kernel_learning_rate": 0.001,
-    "long_conv_l_max": 2048,
-    "max_position_embeddings": 2048,
     "model_type": "bert",
     "monarch_mlp_nblocks": 4,
     "num_attention_heads": 12,

     "intermediate_size": 3072,
     "layer_norm_eps": 1e-12,
     "long_conv_kernel_learning_rate": 0.001,
+    "long_conv_l_max": 128,
+    "max_position_embeddings": 128,
     "model_type": "bert",
     "monarch_mlp_nblocks": 4,
     "num_attention_heads": 12,