text normalization
Cleaning written input
Text normalization is the process of converting text into a more consistent, standard form.
In computing and linguistics, this typically means reducing variations—lowercasing, removing punctuation, expanding contractions, standardizing spellings—so that equivalent expressions match: Text normalization treats "don't" and "do not" as identical.
The term appears frequently in natural language processing, search engines, and speech recognition systems.
- tokenizationConverting input into discrete units
- preprocessingInitial data preparation steps
- cleaningRemoving dirt and mess
- noise removalEliminating unwanted signal artifacts
- whitespaceUnprintable gap characters
lowercase · stemming · punctuation · encoding · parsing · regex · scrubbing · stripping · standardization · conversion · formatting · spelling · spell correction · abbreviation · expansion · accent · diacritics · segmentation
- NLPtokenization, natural language processing, NLP, parsing, stemming
- text encodingUTF-8, ASCII, encoding, character set, charset
- voice synthesistext-to-speech, TTS, speech synthesis, voice synthesis, synthetic voice