Hugging Face's logo

flexitok
/
bpe_script_SEAS_16000

Indonesian
Vietnamese
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
bpe_script_SEAS_16000 / script_1

Commit History

Upload script_1/flexitok--bpe_script_SEAS_16000_overlap.json with huggingface_hub
5bb2748
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_SEAS_16000_vocab.json with huggingface_hub
e61849a
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_SEAS_16000_super_mapping.json with huggingface_hub
e01a238
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_SEAS_16000_overlap.json with huggingface_hub
5efb5f1
verified

gsaltintas commited on

Upload script_1/participating_tokenizers.json with huggingface_hub
61a4274
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_SEAS_16000.yaml with huggingface_hub
3779e6b
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_SEAS_16000_info.json with huggingface_hub
e7c0617
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_SEAS_16000_vocab.json with huggingface_hub
083d4d3
verified

gsaltintas commited on

Upload script_1/flexitok--bpe_script_SEAS_16000_super_mapping.json with huggingface_hub
103ad38
verified

gsaltintas commited on