Package opennlp.tools.util.featuregen

# Class TokenPatternFeatureGenerator

java.lang.Object
opennlp.tools.util.featuregen.TokenPatternFeatureGenerator

All Implemented Interfaces:
  `AdaptiveFeatureGenerator`

public class TokenPatternFeatureGenerator extends Object implements AdaptiveFeatureGenerator
Partitions tokens into sub-tokens based on character classes and generates class features for each of the sub-tokens and combinations of those sub-tokens.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `TokenPatternFeatureGenerator()`
  Initializes a `TokenPatternFeatureGenerator`.
  `TokenPatternFeatureGenerator(Tokenizer supportTokenizer)`
  Initializes a `TokenPatternFeatureGenerator` instance.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `createFeatures(List<String> feats, String[] toks, int index, String[] preds)`
  Adds the appropriate features for the token at the specified `index` with the specified array of `previousOutcomes` to the specified list of features.

  ### Methods inherited from class java.lang.Object

  `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

  ### Methods inherited from interface opennlp.tools.util.featuregen.AdaptiveFeatureGenerator

  `clearAdaptiveData, updateAdaptiveData`

- ## Constructor Details

  - ### TokenPatternFeatureGenerator

    public TokenPatternFeatureGenerator()
    Initializes a `TokenPatternFeatureGenerator`. For tokenization the `SimpleTokenizer` is used.

  - ### TokenPatternFeatureGenerator

    public TokenPatternFeatureGenerator(Tokenizer supportTokenizer)
    Initializes a `TokenPatternFeatureGenerator` instance.
    Parameters:
      `supportTokenizer` - The `Tokenizer` to be used.

- ## Method Details

  - ### createFeatures

    public void createFeatures(List<String> feats, String[] toks, int index, String[] preds)
    Description copied from interface: `AdaptiveFeatureGenerator`
    Adds the appropriate features for the token at the specified `index` with the specified array of `previousOutcomes` to the specified list of features.
    Specified by:
      `createFeatures` in interface `AdaptiveFeatureGenerator`
    Parameters:
      `feats` - The list of features to be added to.
      `toks` - The tokens of the sentence or other text unit being processed.
      `index` - The index of the token which is currently being processed.
      `preds` - The outcomes for the tokens prior to the specified index.