# Package org.apache.lucene.analysis.tokenattributes

package org.apache.lucene.analysis.tokenattributes
General-purpose attributes for text analysis.

- Related Packages
  Package
  Description
  org.apache.lucene.analysis
  Text analysis.
  org.apache.lucene.analysis.standard
  Fast, general-purpose grammar-based tokenizer `StandardTokenizer` implements the Word Break rules from the Unicode Text Segmentation algorithm, as specified in Unicode Standard Annex #29.
- Class
  Description
  BytesTermAttribute
  This attribute can be used if you have the raw term bytes to be indexed.
  BytesTermAttributeImpl
  Implementation class for `BytesTermAttribute`.
  CharTermAttribute
  The term text of a Token.
  CharTermAttributeImpl
  Default implementation of `CharTermAttribute`.
  FlagsAttribute
  This attribute can be used to pass different flags down the `Tokenizer` chain, e.g.
  FlagsAttributeImpl
  Default implementation of `FlagsAttribute`.
  KeywordAttribute
  This attribute can be used to mark a token as a keyword.
  KeywordAttributeImpl
  Default implementation of `KeywordAttribute`.
  OffsetAttribute
  The start and end character offset of a Token.
  OffsetAttributeImpl
  Default implementation of `OffsetAttribute`.
  PackedTokenAttributeImpl
  Default implementation of the common attributes used by Lucene: `CharTermAttribute` `TypeAttribute` `PositionIncrementAttribute` `PositionLengthAttribute` `OffsetAttribute` `TermFrequencyAttribute`
  PayloadAttribute
  The payload of a Token.
  PayloadAttributeImpl
  Default implementation of `PayloadAttribute`.
  PositionIncrementAttribute
  Determines the position of this token relative to the previous Token in a TokenStream, used in phrase searching.
  PositionIncrementAttributeImpl
  Default implementation of `PositionIncrementAttribute`.
  PositionLengthAttribute
  Determines how many positions this token spans.
  PositionLengthAttributeImpl
  Default implementation of `PositionLengthAttribute`.
  SentenceAttribute
  This attribute tracks what sentence a given token belongs to as well as potentially other sentence specific attributes.
  SentenceAttributeImpl
  Default implementation of `SentenceAttribute`.
  TermFrequencyAttribute
  Sets the custom term frequency of a term within one document.
  TermFrequencyAttributeImpl
  Default implementation of `TermFrequencyAttribute`.
  TermToBytesRefAttribute
  This attribute is requested by TermsHashPerField to index the contents.
  TypeAttribute
  A Token's lexical type.
  TypeAttributeImpl
  Default implementation of `TypeAttribute`.