Package org.apache.lucene.analysis.opennlp

# Class OpenNLPTokenizer

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.Tokenizer
org.apache.lucene.analysis.util.SegmentingTokenizerBase
org.apache.lucene.analysis.opennlp.OpenNLPTokenizer

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public final class OpenNLPTokenizer extends SegmentingTokenizerBase
Run OpenNLP SentenceDetector and Tokenizer. The index of each sentence is stored in SentenceAttribute.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

  `AttributeSource.State`

- ## Field Summary

  ### Fields inherited from class org.apache.lucene.analysis.util.SegmentingTokenizerBase

  `buffer, BUFFERMAX, offset`

  ### Fields inherited from class org.apache.lucene.analysis.Tokenizer

  `input`

  ### Fields inherited from class org.apache.lucene.analysis.TokenStream

  `DEFAULT_TOKEN_ATTRIBUTE_FACTORY`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `OpenNLPTokenizer(AttributeFactory factory, NLPSentenceDetectorOp sentenceOp, NLPTokenizerOp tokenizerOp)`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `close()`
  `protected boolean`
  `incrementWord()`
  `void`
  `reset()`
  `protected void`
  `setNextSentence(int sentenceStart, int sentenceEnd)`

  ### Methods inherited from class org.apache.lucene.analysis.util.SegmentingTokenizerBase

  `end, incrementToken, isSafeEnd`

  ### Methods inherited from class org.apache.lucene.analysis.Tokenizer

  `correctOffset, setReader, setReaderTestPoint`

  ### Methods inherited from class org.apache.lucene.util.AttributeSource

  `addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString`

  ### Methods inherited from class java.lang.Object

  `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- ## Constructor Details

  - ### OpenNLPTokenizer

    public OpenNLPTokenizer(AttributeFactory factory, NLPSentenceDetectorOp sentenceOp, NLPTokenizerOp tokenizerOp) throws IOException
    Throws:
      `IOException`

- ## Method Details

  - ### close

    public void close() throws IOException
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `Closeable`
    Overrides:
      `close` in class `Tokenizer`
    Throws:
      `IOException`

  - ### setNextSentence

    protected void setNextSentence(int sentenceStart, int sentenceEnd)
    Specified by:
      `setNextSentence` in class `SegmentingTokenizerBase`

  - ### incrementWord

    protected boolean incrementWord()
    Specified by:
      `incrementWord` in class `SegmentingTokenizerBase`

  - ### reset

    public void reset() throws IOException
    Overrides:
      `reset` in class `SegmentingTokenizerBase`
    Throws:
      `IOException`