Package org.apache.lucene.analysis.cn.smart.hhmm

# Class SegTokenFilter

java.lang.Object
org.apache.lucene.analysis.cn.smart.hhmm.SegTokenFilter

public class SegTokenFilter extends Object
Filters a `SegToken` by converting full-width latin to half-width, then lowercasing latin. Additionally, all punctuation is converted into `Utility.COMMON_DELIMITER`

WARNING: This API is experimental and might change in incompatible ways in the next release.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `SegTokenFilter()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `SegToken`
  `filter(SegToken token)`
  Filter an input `SegToken`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### SegTokenFilter

    public SegTokenFilter()

- ## Method Details

  - ### filter

    public SegToken filter(SegToken token)
    Filter an input `SegToken`

    Full-width latin will be converted to half-width, then all latin will be lowercased. All punctuation is converted into `Utility.COMMON_DELIMITER`

    Parameters:
      `token` - input `SegToken`
    Returns:
      normalized `SegToken`