org.apache.lucene.analysis.util

## Class TokenizerFactory

- java.lang.Object
- - org.apache.lucene.analysis.util.AbstractAnalysisFactory
  - - org.apache.lucene.analysis.util.TokenizerFactory

Direct Known Subclasses:
  ClassicTokenizerFactory, EdgeNGramTokenizerFactory, KeywordTokenizerFactory, LetterTokenizerFactory, NGramTokenizerFactory, PathHierarchyTokenizerFactory, PatternTokenizerFactory, SimplePatternSplitTokenizerFactory, SimplePatternTokenizerFactory, StandardTokenizerFactory, ThaiTokenizerFactory, UAX29URLEmailTokenizerFactory, WhitespaceTokenizerFactory, WikipediaTokenizerFactory

    public abstract class TokenizerFactory
    extends AbstractAnalysisFactory

Abstract parent class for analysis factories that create `Tokenizer` instances.

Since:
  3.1

### Field Summary

### Fields inherited from class org.apache.lucene.analysis.util.AbstractAnalysisFactory

`LUCENE_MATCH_VERSION_PARAM, luceneMatchVersion`

### Constructor Summary

Constructors
`protected` `TokenizerFactory(Map<String,String> args)` Initialize this factory via a set of key-value pairs.

### Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
`static Set<String>` `availableTokenizers()` returns a list of all available tokenizer names from context classpath
`Tokenizer` `create()` Creates a TokenStream of the specified input using the default attribute factory.
`abstract Tokenizer` `create(AttributeFactory factory)` Creates a TokenStream of the specified input using the given AttributeFactory
`static String` `findSPIName(Class<? extends TokenizerFactory> serviceClass)` looks up a SPI name for the specified tokenizer factory
`static TokenizerFactory` `forName(String name, Map<String,String> args)` looks up a tokenizer by name from context classpath
`static Class<? extends TokenizerFactory>` `lookupClass(String name)` looks up a tokenizer class by name from context classpath
`static void` `reloadTokenizers(ClassLoader classloader)` Reloads the factory list from the given `ClassLoader`.

### Methods inherited from class org.apache.lucene.analysis.util.AbstractAnalysisFactory

`get, get, get, get, get, getBoolean, getChar, getClassArg, getFloat, getInt, getLines, getLuceneMatchVersion, getOriginalArgs, getPattern, getSet, getSnowballWordSet, getWordSet, isExplicitLuceneMatchVersion, require, require, require, requireBoolean, requireChar, requireFloat, requireInt, setExplicitLuceneMatchVersion, splitAt, splitFileNames`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### TokenizerFactory

    protected TokenizerFactory(Map<String,String> args)

Initialize this factory via a set of key-value pairs.

### Method Detail

#### forName

    public static TokenizerFactory forName(String name,
                                           Map<String,String> args)

looks up a tokenizer by name from context classpath

#### lookupClass

    public static Class<? extends TokenizerFactory> lookupClass(String name)

looks up a tokenizer class by name from context classpath

#### availableTokenizers

    public static Set<String> availableTokenizers()

returns a list of all available tokenizer names from context classpath

#### findSPIName

    public static String findSPIName(Class<? extends TokenizerFactory> serviceClass)

looks up a SPI name for the specified tokenizer factory

#### reloadTokenizers

    public static void reloadTokenizers(ClassLoader classloader)

Reloads the factory list from the given `ClassLoader`. Changes to the factories are visible after the method ends, all iterators (`availableTokenizers()`,...) stay consistent.

NOTE: Only new factories are added, existing ones are never removed or replaced.

This method is expensive and should only be called for discovery of new factories on the given classpath/classloader!

#### create

    public final Tokenizer create()

Creates a TokenStream of the specified input using the default attribute factory.

#### create

    public abstract Tokenizer create(AttributeFactory factory)

Creates a TokenStream of the specified input using the given AttributeFactory