Token
The unit of text an LLM reads and generates — roughly 4 characters or 0.75 words.
LLMs don't operate on characters or words — they operate on tokens, sub-word units learned during training. 'GlobalChatbot' might be 3 tokens; 'chatbot' might be 1.
Tokens matter because they're the unit of cost (you pay per token, input and output) and the unit of the context window. Optimizing prompts often means tokenizer-aware compression.
Different models tokenize differently. GPT and Claude both use byte-pair encoding (BPE) variants — same idea, slightly different vocabularies.
Esempio in GlobalChatbot
GlobalChatbot's analytics show per-conversation token usage so you can spot expensive workflows and optimize the system prompt or retrieval strategy.
Vedi in azione.
GlobalChatbot — agente AI per aziende serie. Configurazione in 5 minuti, 45 lingue, senza carta richiesta.
14 days · no card · cancel anytime