Package org.apache.lucene.analysis.fa

# Class PersianAnalyzer

java.lang.Object
org.apache.lucene.analysis.Analyzer
org.apache.lucene.analysis.StopwordAnalyzerBase
org.apache.lucene.analysis.fa.PersianAnalyzer

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public final class PersianAnalyzer extends StopwordAnalyzerBase
`Analyzer` for Persian.

This Analyzer uses `PersianCharFilter` which implies tokenizing around zero-width non-joiner in addition to whitespace. Some persian-specific variant forms (such as farsi yeh and keheh) are standardized. "Stemming" is accomplished via stopwords.

Since:
  3.1

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

  `Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final String`
  `DEFAULT_STOPWORD_FILE`
  File containing default Persian stopwords.
  `static final String`
  `STOPWORDS_COMMENT`
  The comment character in the stopwords file.

  ### Fields inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase

  `stopwords`

  ### Fields inherited from class org.apache.lucene.analysis.Analyzer

  `GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `PersianAnalyzer()`
  Builds an analyzer with the default stop words: `DEFAULT_STOPWORD_FILE`.
  `PersianAnalyzer(CharArraySet stopwords)`
  Builds an analyzer with the given stop words
  `PersianAnalyzer(CharArraySet stopwords, CharArraySet stemExclusionSet)`
  Builds an analyzer with the given stop word.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `protected Analyzer.TokenStreamComponents`
  `createComponents(String fieldName)`
  Creates `Analyzer.TokenStreamComponents` used to tokenize all the text in the provided `Reader`.
  `static CharArraySet`
  `getDefaultStopSet()`
  Returns an unmodifiable instance of the default stop-words set.
  `protected Reader`
  `initReader(String fieldName, Reader reader)`
  Wraps the Reader with `PersianCharFilter`
  `protected TokenStream`
  `normalize(String fieldName, TokenStream in)`

  ### Methods inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase

  `getStopwordSet, loadStopwordSet, loadStopwordSet`

  ### Methods inherited from class org.apache.lucene.analysis.Analyzer

  `attributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, initReaderForNormalization, normalize, tokenStream, tokenStream`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### DEFAULT_STOPWORD_FILE

    public static final String DEFAULT_STOPWORD_FILE
    File containing default Persian stopwords.

    Default stopword list is from http://members.unine.ch/jacques.savoy/clef/index.html The stopword list is BSD-Licensed.

    See Also:
      - Constant Field Values

  - ### STOPWORDS_COMMENT

    public static final String STOPWORDS_COMMENT
    The comment character in the stopwords file. All lines prefixed with this will be ignored
    See Also:
      - Constant Field Values

- ## Constructor Details

  - ### PersianAnalyzer

    public PersianAnalyzer()
    Builds an analyzer with the default stop words: `DEFAULT_STOPWORD_FILE`.

  - ### PersianAnalyzer

    public PersianAnalyzer(CharArraySet stopwords)
    Builds an analyzer with the given stop words
    Parameters:
      `stopwords` - a stopword set

  - ### PersianAnalyzer

    public PersianAnalyzer(CharArraySet stopwords, CharArraySet stemExclusionSet)
    Builds an analyzer with the given stop word. If a none-empty stem exclusion set is provided this analyzer will add a `SetKeywordMarkerFilter` before `PersianStemFilter`.
    Parameters:
      `stopwords` - a stopword set
      `stemExclusionSet` - a set of terms not to be stemmed

- ## Method Details

  - ### getDefaultStopSet

    public static CharArraySet getDefaultStopSet()
    Returns an unmodifiable instance of the default stop-words set.
    Returns:
      an unmodifiable instance of the default stop-words set.

  - ### createComponents

    protected Analyzer.TokenStreamComponents createComponents(String fieldName)
    Creates `Analyzer.TokenStreamComponents` used to tokenize all the text in the provided `Reader`.
    Specified by:
      `createComponents` in class `Analyzer`
    Returns:
      `Analyzer.TokenStreamComponents` built from a `StandardTokenizer` filtered with `LowerCaseFilter`, `DecimalDigitFilter`, `ArabicNormalizationFilter`, `PersianNormalizationFilter`, Persian Stop words, and `PersianStemFilter`.

  - ### normalize

    protected TokenStream normalize(String fieldName, TokenStream in)
    Overrides:
      `normalize` in class `Analyzer`

  - ### initReader

    protected Reader initReader(String fieldName, Reader reader)
    Wraps the Reader with `PersianCharFilter`
    Overrides:
      `initReader` in class `Analyzer`