Package | Description |
---|---|
org.apache.lucene.analysis.standard |
Fast, general-purpose grammar-based tokenizer
StandardTokenizer
implements the Word Break rules from the Unicode Text Segmentation algorithm, as specified in
Unicode Standard Annex #29. |
Modifier and Type | Field and Description |
---|---|
private UAX29URLEmailTokenizerImpl |
UAX29URLEmailTokenizer.scanner
A private instance of the JFlex-constructed scanner
|
Modifier and Type | Method and Description |
---|---|
private UAX29URLEmailTokenizerImpl |
UAX29URLEmailTokenizer.getScanner() |