[tokenizer] AlternateTokenizer - a minpeter Collection

minpeter 's Collections

[Dataset] K-Corpus

[Dataset] FineWeb2 Edu Korean

[Model] Very, very small things

[Dataset] Pretrain-corpus

[Model] en-ko trans

[Dataset] Candidate datasets to translate

[Study] NN MNIST

[Model] FLUX.1 Full Finetuned & Merged

[🛠️] Huggingface Utility

[Dataset] unified standard function calling

[tokenizer] AlternateTokenizer

[Dataset] Function Calling

[tokenizer] AlternateTokenizer

updated Mar 15, 2025

A series of modified tokenizers for tuning chatml inspired by teknium and hermes.

minpeter/Llama-3.x-AlternateTokenizer

Updated Feb 11, 2025 • 1

Note An alternative tokenizer for the final form of the Llama 3.x series.
teknium/Llama-3.1-AlternateTokenizer

Text Generation • 8B • Updated Jun 18, 2025 • 22 • • 5
minpeter/Llama-3.2-1B-AlternateTokenizer-chatml

Text Generation • 1B • Updated Feb 9, 2025 • 5
minpeter/Llama-3.2-1B-AlternateTokenizer-tool-chatml

Text Generation • 1B • Updated Feb 10, 2025 • 9
philschmid/gemma-tokenizer-chatml

Updated Feb 29, 2024 • 32