org.apache.lucene.analysis

Class LowerCaseTokenizer

public final class LowerCaseTokenizer extends LetterTokenizer

LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. It divides text at non-letters and converts them to lower case. While it is functionally equivalent to the combination of LetterTokenizer and LowerCaseFilter, there is a performance advantage to doing the two tasks at once, hence this (redundant) implementation.

Note: this does a decent job for most European languages, but does a terrible job for some Asian languages, where words are not separated by spaces.

Constructor Summary
LowerCaseTokenizer(Reader in)
Construct a new LowerCaseTokenizer.
Method Summary
protected charnormalize(char c)
Collects only characters which satisfy {@link Character#isLetter(char)}.

Constructor Detail

LowerCaseTokenizer

public LowerCaseTokenizer(Reader in)
Construct a new LowerCaseTokenizer.

Method Detail

normalize

protected char normalize(char c)
Collects only characters which satisfy {@link Character#isLetter(char)}.
Copyright © 2000-2007 Apache Software Foundation. All Rights Reserved.