arxivcs.CLcs.LGq-bio.BM2026-07-06
Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES
Every chemical language model reading SMILES begins with a tokenizer, yet the field has inherited byte-pair encoding (BPE) from natural language with little scrutiny. In natural language, BPE's principal alternative, Unigram-LM, is known to build structurally different vocabulari…