org.apache.lucene.analysis.ja

## Class Token

- java.lang.Object
- - org.apache.lucene.analysis.ja.Token

-     public class Token
      extends Object

  Analyzed token with morphological data from its dictionary.

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `Token(int wordId, char[] surfaceForm, int offset, int length, JapaneseTokenizer.Type type, int position, Dictionary dictionary)`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `String` | `getBaseForm()`
    `String` | `getInflectionForm()`
    `String` | `getInflectionType()`
    `int` | `getLength()`
    `int` | `getOffset()`
    `String` | `getPartOfSpeech()`
    `int` | `getPosition()` Get index of this token in input text
    `int` | `getPositionLength()` Get the length (in tokens) of this token.
    `String` | `getPronunciation()`
    `String` | `getReading()`
    `char[]` | `getSurfaceForm()`
    `String` | `getSurfaceFormString()`
    `JapaneseTokenizer.Type` | `getType()` Returns the type of this token
    `boolean` | `isKnown()` Returns true if this token is known word
    `boolean` | `isUnknown()` Returns true if this token is unknown word
    `boolean` | `isUser()` Returns true if this token is defined in user dictionary
    `void` | `setPositionLength(int positionLength)` Set the position length (in tokens) of this token.
    `String` | `toString()`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### Token

          public Token(int wordId,
                       char[] surfaceForm,
                       int offset,
                       int length,
                       JapaneseTokenizer.Type type,
                       int position,
                       Dictionary dictionary)

  - ### Method Detail

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

    - #### getSurfaceForm

          public char[] getSurfaceForm()

      Returns:
        surfaceForm

    - #### getOffset

          public int getOffset()

      Returns:
        offset into surfaceForm

    - #### getLength

          public int getLength()

      Returns:
        length of surfaceForm

    - #### getSurfaceFormString

          public String getSurfaceFormString()

      Returns:
        surfaceForm as a String

    - #### getReading

          public String getReading()

      Returns:
        reading. null if token doesn't have reading.

    - #### getPronunciation

          public String getPronunciation()

      Returns:
        pronunciation. null if token doesn't have pronunciation.

    - #### getPartOfSpeech

          public String getPartOfSpeech()

      Returns:
        part of speech.

    - #### getInflectionType

          public String getInflectionType()

      Returns:
        inflection type or null

    - #### getInflectionForm

          public String getInflectionForm()

      Returns:
        inflection form or null

    - #### getBaseForm

          public String getBaseForm()

      Returns:
        base form or null if token is not inflected

    - #### getType

          public JapaneseTokenizer.Type getType()

      Returns the type of this token
      Returns:
        token type, not null

    - #### isKnown

          public boolean isKnown()

      Returns true if this token is known word
      Returns:
        true if this token is in standard dictionary. false if not.

    - #### isUnknown

          public boolean isUnknown()

      Returns true if this token is unknown word
      Returns:
        true if this token is unknown word. false if not.

    - #### isUser

          public boolean isUser()

      Returns true if this token is defined in user dictionary
      Returns:
        true if this token is in user dictionary. false if not.

    - #### getPosition

          public int getPosition()

      Get index of this token in input text
      Returns:
        position of token

    - #### setPositionLength

          public void setPositionLength(int positionLength)

      Set the position length (in tokens) of this token. For normal tokens this is 1; for compound tokens it's > 1.

    - #### getPositionLength

          public int getPositionLength()

      Get the length (in tokens) of this token. For normal tokens this is 1; for compound tokens it's > 1.
      Returns:
        position length of token

Copyright © 2000-2019 Apache Software Foundation. All Rights Reserved.