# Package org.apache.lucene.analysis.miscellaneous

package org.apache.lucene.analysis.miscellaneous
Miscellaneous Tokenstreams.

- Class
  Description
  ASCIIFoldingFilter
  This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.
  ASCIIFoldingFilterFactory
  Factory for `ASCIIFoldingFilter`.
  CapitalizationFilter
  A filter to apply normal capitalization rules to Tokens.
  CapitalizationFilterFactory
  Factory for `CapitalizationFilter`.
  CodepointCountFilter
  Removes words that are too long or too short from the stream.
  CodepointCountFilterFactory
  Factory for `CodepointCountFilter`.
  ConcatenateGraphFilter
  Concatenates/Joins every incoming token with a separator into one output token for every path through the token stream (which is a graph).
  ConcatenateGraphFilter.BytesRefBuilderTermAttribute
  Attribute providing access to the term builder and UTF-16 conversion
  ConcatenateGraphFilter.BytesRefBuilderTermAttributeImpl
  Implementation of `ConcatenateGraphFilter.BytesRefBuilderTermAttribute`
  ConcatenateGraphFilterFactory
  Factory for `ConcatenateGraphFilter`.
  ConcatenatingTokenStream
  A TokenStream that takes an array of input TokenStreams as sources, and concatenates them together.
  ConditionalTokenFilter
  Allows skipping TokenFilters based on the current set of attributes.
  ConditionalTokenFilterFactory
  Abstract parent class for analysis factories that create `ConditionalTokenFilter` instances
  DateRecognizerFilter
  Filters all tokens that cannot be parsed to a date, using the provided `DateFormat`.
  DateRecognizerFilterFactory
  Factory for `DateRecognizerFilter`.
  DelimitedTermFrequencyTokenFilter
  Characters before the delimiter are the "token", the textual integer after is the term frequency.
  DelimitedTermFrequencyTokenFilterFactory
  Factory for `DelimitedTermFrequencyTokenFilter`.
  DropIfFlaggedFilter
  Allows Tokens with a given combination of flags to be dropped.
  DropIfFlaggedFilterFactory
  Provides a filter that will drop tokens matching a set of flags.
  EmptyTokenStream
  An always exhausted token stream.
  FingerprintFilter
  Filter outputs a single token which is a concatenation of the sorted and de-duplicated set of input tokens.
  FingerprintFilterFactory
  Factory for `FingerprintFilter`.
  FixBrokenOffsetsFilter
  Deprecated.
  Fix the token filters that create broken offsets in the first place.
  FixBrokenOffsetsFilterFactory
  Deprecated.
  HyphenatedWordsFilter
  When the plain text is extracted from documents, we will often have many words hyphenated and broken into two lines.
  HyphenatedWordsFilterFactory
  Factory for `HyphenatedWordsFilter`.
  KeepWordFilter
  A TokenFilter that only keeps tokens with text contained in the required words.
  KeepWordFilterFactory
  Factory for `KeepWordFilter`.
  KeywordMarkerFilter
  Marks terms as keywords via the `KeywordAttribute`.
  KeywordMarkerFilterFactory
  Factory for `KeywordMarkerFilter`.
  KeywordRepeatFilter
  This TokenFilter emits each incoming token twice once as keyword and once non-keyword, in other words once with `KeywordAttribute.setKeyword(boolean)` set to `true` and once set to `false`.
  KeywordRepeatFilterFactory
  Factory for `KeywordRepeatFilter`.
  LengthFilter
  Removes words that are too long or too short from the stream.
  LengthFilterFactory
  Factory for `LengthFilter`.
  LimitTokenCountAnalyzer
  This Analyzer limits the number of tokens while indexing.
  LimitTokenCountFilter
  This TokenFilter limits the number of tokens while indexing.
  LimitTokenCountFilterFactory
  Factory for `LimitTokenCountFilter`.
  LimitTokenOffsetFilter
  Lets all tokens pass through until it sees one with a start offset <= a configured limit, which won't pass and ends the stream.
  LimitTokenOffsetFilterFactory
  Factory for `LimitTokenOffsetFilter`.
  LimitTokenPositionFilter
  This TokenFilter limits its emitted tokens to those with positions that are not greater than the configured limit.
  LimitTokenPositionFilterFactory
  Factory for `LimitTokenPositionFilter`.
  PatternKeywordMarkerFilter
  Marks terms as keywords via the `KeywordAttribute`.
  PerFieldAnalyzerWrapper
  This analyzer is used to facilitate scenarios where different fields require different analysis techniques.
  ProtectedTermFilter
  A ConditionalTokenFilter that only applies its wrapped filters to tokens that are not contained in a protected set.
  ProtectedTermFilterFactory
  Factory for a `ProtectedTermFilter`
  RemoveDuplicatesTokenFilter
  A TokenFilter which filters out Tokens at the same position and Term text as the previous token in the stream.
  RemoveDuplicatesTokenFilterFactory
  Factory for `RemoveDuplicatesTokenFilter`.
  ScandinavianFoldingFilter
  This filter folds Scandinavian characters åÅäæÄÆ->a and öÖøØ->o.
  ScandinavianFoldingFilterFactory
  Factory for `ScandinavianFoldingFilter`.
  ScandinavianNormalizationFilter
  This filter normalize use of the interchangeable Scandinavian characters æÆäÄöÖøØ and folded variants (aa, ao, ae, oe and oo) by transforming them to åÅæÆøØ.
  ScandinavianNormalizationFilterFactory
  Factory for `ScandinavianNormalizationFilter`.
  ScandinavianNormalizer
  This Normalizer does the heavy lifting for a set of Scandinavian normalization filters, normalizing use of the interchangeable Scandinavian characters æÆäÄöÖøØ and folded variants (aa, ao, ae, oe and oo) by transforming them to åÅæÆøØ.
  ScandinavianNormalizer.Foldings
  List of possible foldings that can be used when configuring the filter
  SetKeywordMarkerFilter
  Marks terms as keywords via the `KeywordAttribute`.
  StemmerOverrideFilter
  Provides the ability to override any `KeywordAttribute` aware stemmer with custom dictionary-based stemming.
  StemmerOverrideFilter.Builder
  This builder builds an `FST` for the `StemmerOverrideFilter`
  StemmerOverrideFilter.StemmerOverrideMap
  A read-only 4-byte FST backed map that allows fast case-insensitive key value lookups for `StemmerOverrideFilter`
  StemmerOverrideFilterFactory
  Factory for `StemmerOverrideFilter`.
  TrimFilter
  Trims leading and trailing whitespace from Tokens in the stream.
  TrimFilterFactory
  Factory for `TrimFilter`.
  TruncateTokenFilter
  A token filter for truncating the terms into a specific length.
  TruncateTokenFilterFactory
  Factory for `TruncateTokenFilter`.
  TypeAsSynonymFilter
  Adds the `TypeAttribute.type()` as a synonym, i.e.
  TypeAsSynonymFilterFactory
  Factory for `TypeAsSynonymFilter`.
  WordDelimiterFilter
  Deprecated.
  Use `WordDelimiterGraphFilter` instead: it produces a correct token graph so that e.g.
  WordDelimiterFilterFactory
  Deprecated.
  Use `WordDelimiterGraphFilterFactory` instead: it produces a correct token graph so that e.g.
  WordDelimiterGraphFilter
  Splits words into subwords and performs optional transformations on subword groups, producing a correct token graph so that e.g.
  WordDelimiterGraphFilterFactory
  Factory for `WordDelimiterGraphFilter`.
  WordDelimiterIterator
  A BreakIterator-like API for iterating over subwords in text, according to WordDelimiterGraphFilter rules.