public class WhitespaceTokenizerFactory extends TokenizerFactory
WhitespaceTokenizer
.
<fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100"> <analyzer> <tokenizer class="solr.WhitespaceTokenizerFactory" rule="unicode" maxTokenLen="256"/> </analyzer> </fieldType>Options:
WhitespaceTokenizer
or "unicode" for UnicodeWhitespaceTokenizer
CharTokenizer
::DEFAULT_MAX_TOKEN_LENModifier and Type | Field and Description |
---|---|
private int |
maxTokenLen |
private java.lang.String |
rule |
static java.lang.String |
RULE_JAVA |
private static java.util.Collection<java.lang.String> |
RULE_NAMES |
static java.lang.String |
RULE_UNICODE |
LUCENE_MATCH_VERSION_PARAM, luceneMatchVersion
Constructor and Description |
---|
WhitespaceTokenizerFactory(java.util.Map<java.lang.String,java.lang.String> args)
Creates a new WhitespaceTokenizerFactory
|
Modifier and Type | Method and Description |
---|---|
Tokenizer |
create(AttributeFactory factory)
Creates a TokenStream of the specified input using the given AttributeFactory
|
availableTokenizers, create, forName, lookupClass, reloadTokenizers
get, get, get, get, get, getBoolean, getChar, getClassArg, getFloat, getInt, getLines, getLuceneMatchVersion, getOriginalArgs, getPattern, getSet, getSnowballWordSet, getWordSet, isExplicitLuceneMatchVersion, require, require, require, requireBoolean, requireChar, requireFloat, requireInt, setExplicitLuceneMatchVersion, splitAt, splitFileNames
public static final java.lang.String RULE_JAVA
public static final java.lang.String RULE_UNICODE
private static final java.util.Collection<java.lang.String> RULE_NAMES
private final java.lang.String rule
private final int maxTokenLen
public WhitespaceTokenizerFactory(java.util.Map<java.lang.String,java.lang.String> args)
public Tokenizer create(AttributeFactory factory)
TokenizerFactory
create
in class TokenizerFactory