org.exist.storage.analysis
Class SimpleTokenizer
java.lang.Object
org.exist.storage.analysis.SimpleTokenizer
- All Implemented Interfaces:
- Tokenizer
public class SimpleTokenizer
- extends java.lang.Object
- implements Tokenizer
This is the default class used by the fulltext indexer for
tokenizing a string into words. Known token types are defined
by class Token.
- Author:
- Wolfgang Meier
Methods inherited from class java.lang.Object |
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
SimpleTokenizer
public SimpleTokenizer()
SimpleTokenizer
public SimpleTokenizer(boolean stem)
setStemming
public void setStemming(boolean stem)
- Specified by:
setStemming
in interface Tokenizer
getLength
public int getLength()
getText
public java.lang.String getText()
nextToken
public TextToken nextToken()
- Specified by:
nextToken
in interface Tokenizer
nextToken
public TextToken nextToken(boolean wildcards)
- Specified by:
nextToken
in interface Tokenizer
setText
public void setText(java.lang.CharSequence text)
- Specified by:
setText
in interface Tokenizer
main
public static void main(java.lang.String[] args)
Copyright (C) Wolfgang Meier. All rights reserved.