## Uses of Class
org.apache.lucene.analysis.TokenStream

Packages that use TokenStream
org.apache.lucene.analysis: Text analysis.
org.apache.lucene.analysis.standard: Fast, general-purpose grammar-based tokenizer `StandardTokenizer` implements the Word Break rules from the Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29.
org.apache.lucene.codecs: Codecs API: API for customization of the encoding and structure of the index.
org.apache.lucene.document: The logical representation of a `Document` for indexing and searching.
org.apache.lucene.index: Code to maintain and access indices.
org.apache.lucene.util: Some utility classes.
org.apache.lucene.util.graph: Utility classes for working with token streams as graphs.

### Uses of TokenStream in org.apache.lucene.analysis

Subclasses of TokenStream in org.apache.lucene.analysis
`class` `CachingTokenFilter` This class can be used if the token attributes of a TokenStream are intended to be consumed more than once.
`class` `FilteringTokenFilter` Abstract base class for TokenFilters that may remove tokens.
`class` `GraphTokenFilter` An abstract TokenFilter that exposes its input stream as a graph Call `GraphTokenFilter.incrementBaseToken()` to move the root of the graph to the next position in the TokenStream, `GraphTokenFilter.incrementGraphToken()` to move along the current graph, and `GraphTokenFilter.incrementGraph()` to reset to the next graph based at the current root.
`class` `LowerCaseFilter` Normalizes token text to lower case.
`class` `StopFilter` Removes stop words from a token stream.
`class` `TokenFilter` A TokenFilter is a TokenStream whose input is another TokenStream.
`class` `Tokenizer` A Tokenizer is a TokenStream whose input is a Reader.

Fields in org.apache.lucene.analysis declared as TokenStream
`protected TokenStream` TokenFilter.`input` The source of tokens for this filter.
`protected TokenStream` Analyzer.TokenStreamComponents.`sink` Sink tokenstream, such as the outer tokenfilter decorating the chain.

Methods in org.apache.lucene.analysis that return TokenStream
`TokenStream` Analyzer.TokenStreamComponents.`getTokenStream()` Returns the sink `TokenStream`
`protected TokenStream` AnalyzerWrapper.`normalize(String fieldName, TokenStream in)`
`protected TokenStream` Analyzer.`normalize(String fieldName, TokenStream in)` Wrap the given `TokenStream` in order to apply normalization filters.
`TokenStream` Analyzer.`tokenStream(String fieldName, Reader reader)` Returns a TokenStream suitable for `fieldName`, tokenizing the contents of `reader`.
`TokenStream` Analyzer.`tokenStream(String fieldName, String text)` Returns a TokenStream suitable for `fieldName`, tokenizing the contents of `text`.
`protected TokenStream` AnalyzerWrapper.`wrapTokenStreamForNormalization(String fieldName, TokenStream in)` Wraps / alters the given TokenStream for normalization purposes, taken from the wrapped Analyzer, to form new components.
`protected TokenStream` DelegatingAnalyzerWrapper.`wrapTokenStreamForNormalization(String fieldName, TokenStream in)`

Methods in org.apache.lucene.analysis with parameters of type TokenStream
`protected TokenStream` AnalyzerWrapper.`normalize(String fieldName, TokenStream in)`
`protected TokenStream` Analyzer.`normalize(String fieldName, TokenStream in)` Wrap the given `TokenStream` in order to apply normalization filters.
`Automaton` TokenStreamToAutomaton.`toAutomaton(TokenStream in)` Pulls the graph (including `PositionLengthAttribute`) from the provided `TokenStream`, and creates the corresponding automaton where arcs are bytes (or Unicode code points if unicodeArcs = true) from each term.
`protected TokenStream` AnalyzerWrapper.`wrapTokenStreamForNormalization(String fieldName, TokenStream in)` Wraps / alters the given TokenStream for normalization purposes, taken from the wrapped Analyzer, to form new components.
`protected TokenStream` DelegatingAnalyzerWrapper.`wrapTokenStreamForNormalization(String fieldName, TokenStream in)`

Constructors in org.apache.lucene.analysis with parameters of type TokenStream
Constructor and Description
`CachingTokenFilter(TokenStream input)` Create a new CachingTokenFilter around `input`.
`FilteringTokenFilter(TokenStream in)` Create a new `FilteringTokenFilter`.
`GraphTokenFilter(TokenStream input)` Create a new GraphTokenFilter
`LowerCaseFilter(TokenStream in)` Create a new LowerCaseFilter, that normalizes token text to lower case.
`StopFilter(TokenStream in, CharArraySet stopWords)` Constructs a filter which removes words from the input TokenStream that are named in the Set.
`TokenFilter(TokenStream input)` Construct a token stream filtering the given input.
`TokenStreamComponents(Consumer<Reader> source, TokenStream result)` Creates a new `Analyzer.TokenStreamComponents` instance.
`TokenStreamComponents(Tokenizer tokenizer, TokenStream result)` Creates a new `Analyzer.TokenStreamComponents` instance

### Uses of TokenStream in org.apache.lucene.analysis.standard

Subclasses of TokenStream in org.apache.lucene.analysis.standard
`class` `StandardTokenizer` A grammar-based tokenizer constructed with JFlex.

Methods in org.apache.lucene.analysis.standard that return TokenStream
`protected TokenStream` StandardAnalyzer.`normalize(String fieldName, TokenStream in)`

Methods in org.apache.lucene.analysis.standard with parameters of type TokenStream
`protected TokenStream` StandardAnalyzer.`normalize(String fieldName, TokenStream in)`

### Uses of TokenStream in org.apache.lucene.codecs

Methods in org.apache.lucene.codecs that return TokenStream
`TokenStream` StoredFieldsWriter.MergeVisitor.`tokenStream(Analyzer analyzer, TokenStream reuse)`

Methods in org.apache.lucene.codecs with parameters of type TokenStream
`TokenStream` StoredFieldsWriter.MergeVisitor.`tokenStream(Analyzer analyzer, TokenStream reuse)`

### Uses of TokenStream in org.apache.lucene.document

Fields in org.apache.lucene.document declared as TokenStream
`protected TokenStream` Field.`tokenStream` Pre-analyzed tokenStream for indexed fields; this is separate from fieldsData because you are allowed to have both; eg maybe field has a String value but you customize how it's tokenized

Methods in org.apache.lucene.document that return TokenStream
`TokenStream` Field.`tokenStream(Analyzer analyzer, TokenStream reuse)`
`TokenStream` FeatureField.`tokenStream(Analyzer analyzer, TokenStream reuse)`
`TokenStream` Field.`tokenStreamValue()` The TokenStream for this field to be used when indexing, or null.

Methods in org.apache.lucene.document with parameters of type TokenStream
`void` Field.`setTokenStream(TokenStream tokenStream)` Expert: sets the token stream to be used for indexing and causes isIndexed() and isTokenized() to return true.
`TokenStream` Field.`tokenStream(Analyzer analyzer, TokenStream reuse)`
`TokenStream` FeatureField.`tokenStream(Analyzer analyzer, TokenStream reuse)`

Constructors in org.apache.lucene.document with parameters of type TokenStream
Constructor and Description
`Field(String name, TokenStream tokenStream, IndexableFieldType type)` Create field with TokenStream value.
`TextField(String name, TokenStream stream)` Creates a new un-stored TextField with TokenStream value.

### Uses of TokenStream in org.apache.lucene.index

Methods in org.apache.lucene.index that return TokenStream
`TokenStream` IndexableField.`tokenStream(Analyzer analyzer, TokenStream reuse)` Creates the TokenStream used for indexing this field.

Methods in org.apache.lucene.index with parameters of type TokenStream
`TokenStream` IndexableField.`tokenStream(Analyzer analyzer, TokenStream reuse)` Creates the TokenStream used for indexing this field.

### Uses of TokenStream in org.apache.lucene.util

Methods in org.apache.lucene.util with parameters of type TokenStream
`protected Query` QueryBuilder.`analyzeBoolean(String field, TokenStream stream)` Creates simple boolean query from the cached tokenstream contents
`protected Query` QueryBuilder.`analyzeGraphBoolean(String field, TokenStream source, BooleanClause.Occur operator)` Creates a boolean query from a graph token stream.
`protected Query` QueryBuilder.`analyzeGraphPhrase(TokenStream source, String field, int phraseSlop)` Creates graph phrase query from the tokenstream contents
`protected Query` QueryBuilder.`analyzeMultiBoolean(String field, TokenStream stream, BooleanClause.Occur operator)` Creates complex boolean query from the cached tokenstream contents
`protected Query` QueryBuilder.`analyzeMultiPhrase(String field, TokenStream stream, int slop)` Creates complex phrase query from the cached tokenstream contents
`protected Query` QueryBuilder.`analyzePhrase(String field, TokenStream stream, int slop)` Creates simple phrase query from the cached tokenstream contents
`protected Query` QueryBuilder.`analyzeTerm(String field, TokenStream stream)` Creates simple term query from the cached tokenstream contents
`protected Query` QueryBuilder.`createFieldQuery(TokenStream source, BooleanClause.Occur operator, String field, boolean quoted, int phraseSlop)` Creates a query from a token stream.
`protected SpanQuery` QueryBuilder.`createSpanQuery(TokenStream in, String field)` Creates a span query from the tokenstream.

### Uses of TokenStream in org.apache.lucene.util.graph

Methods in org.apache.lucene.util.graph that return types with arguments of type TokenStream
`Iterator<TokenStream>` GraphTokenStreamFiniteStrings.`getFiniteStrings()` Get all finite strings from the automaton.
`Iterator<TokenStream>` GraphTokenStreamFiniteStrings.`getFiniteStrings(int startState, int endState)` Get all finite strings that start at `startState` and end at `endState`.

Constructors in org.apache.lucene.util.graph with parameters of type TokenStream
Constructor and Description
`GraphTokenStreamFiniteStrings(TokenStream in)`