Fine-tuning launchpad
Small open base models worth fine-tuning. Tokenizer, license, ecosystem, and recipes that work.
-
Tokenizer: Llama-3 BPE (128K vocab)
License: Llama 3.1 Community License. Commercial use allowed up to 700M MAU. AUP applies; legal should read it once.
Ecosystem: Highest in class. First-class support across HuggingFace TRL, Axolotl, Unsloth, LLaMA-Factory, llama.cpp, vLLM, MLX.
-
Tokenizer: Qwen tiktoken (151K vocab, multilingual-leaning)
License: Apache 2.0. No AUP, no MAU caps. Cleanest license in the small-model space.
Ecosystem: Strong and growing. First-class in TRL, Axolotl, Unsloth, llama.cpp, vLLM, SGLang.
-
Tokenizer: Gemma SentencePiece (256K vocab, multimodal-aware)
License: Apache 2.0. Cleaner than the older Gemma terms - same effective freedoms as Qwen and Mistral.
Ecosystem: Newer (April 2026). LoRA support via TRL is solid; community fine-tunes are catching up. Best for on-device or multimodal fine-tunes.