org.apache.lucene.codecs.blocktree

## Class BlockTreeTermsWriter

- java.lang.Object
- - org.apache.lucene.codecs.FieldsConsumer
  - - org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter

All Implemented Interfaces:
  Closeable, AutoCloseable

    public final class BlockTreeTermsWriter
    extends FieldsConsumer

Block-based terms index and dictionary writer.

Writes terms dict and index, block-encoding (column stride) each term's metadata for each set of terms between two index terms.

Files:

- `.tim`: Term Dictionary
- `.tip`: Term Index

### Term Dictionary

The .tim file contains the list of terms in each field along with per-term statistics (such as docfreq) and per-term metadata (typically pointers to the postings list for that term in the inverted index).

The .tim is arranged in blocks: with blocks containing a variable number of entries (by default 25-48), where each entry is either a term or a reference to a sub-block.

NOTE: The term dictionary can plug into different postings implementations: the postings writer/reader are actually responsible for encoding and decoding the Postings Metadata and Term Metadata sections.

- TermsDict (.tim) --> Header, PostingsHeader, NodeBlockNumBlocks, FieldSummary, DirOffset, Footer
- NodeBlock --> (OuterNode | InnerNode)
- OuterNode --> EntryCount, SuffixLength, ByteSuffixLength, StatsLength, < TermStats >EntryCount, MetaLength, <TermMetadata>EntryCount
- InnerNode --> EntryCount, SuffixLength[,Sub?], ByteSuffixLength, StatsLength, < TermStats ? >EntryCount, MetaLength, <TermMetadata ? >EntryCount
- TermStats --> DocFreq, TotalTermFreq
- FieldSummary --> NumFields, <FieldNumber, NumTerms, RootCodeLength, ByteRootCodeLength, SumTotalTermFreq?, SumDocFreq, DocCount, LongsSize, MinTerm, MaxTerm>NumFields
- Header --> `CodecHeader`
- DirOffset --> `Uint64`
- MinTerm,MaxTerm --> `VInt` length followed by the byte[]
- EntryCount,SuffixLength,StatsLength,DocFreq,MetaLength,NumFields, FieldNumber,RootCodeLength,DocCount,LongsSize --> `VInt`
- TotalTermFreq,NumTerms,SumTotalTermFreq,SumDocFreq --> `VLong`
- Footer --> `CodecFooter`

Notes:

- Header is a `CodecHeader` storing the version information for the BlockTree implementation.
- DirOffset is a pointer to the FieldSummary section.
- DocFreq is the count of documents which contain the term.
- TotalTermFreq is the total number of occurrences of the term. This is encoded as the difference between the total number of occurrences and the DocFreq.
- FieldNumber is the fields number from `FieldInfos`. (.fnm)
- NumTerms is the number of unique terms for the field.
- RootCode points to the root block for the field.
- SumDocFreq is the total number of postings, the number of term-document pairs across the entire field.
- DocCount is the number of documents that have at least one posting for this field.
- LongsSize records how many long values the postings writer/reader record per term (e.g., to hold freq/prox/doc file offsets).
- MinTerm, MaxTerm are the lowest and highest term in this field.
- PostingsHeader and TermMetadata are plugged into by the specific postings implementation: these contain arbitrary per-file data (such as parameters or versioning information) and per-term data (such as pointers to inverted files).
- For inner nodes of the tree, every entry will steal one bit to mark whether it points to child nodes(sub-block). If so, the corresponding TermStats and TermMetaData are omitted

### Term Index

The .tip file contains an index into the term dictionary, so that it can be accessed randomly. The index is also used to determine when a given term cannot exist on disk (in the .tim file), saving a disk seek.

- TermsIndex (.tip) --> Header, FSTIndexNumFields <IndexStartFP>NumFields, DirOffset, Footer
- Header --> `CodecHeader`
- DirOffset --> `Uint64`
- IndexStartFP --> `VLong`
- FSTIndex --> `FST<byte[]>`
- Footer --> `CodecFooter`

Notes:

- The .tip file contains a separate FST for each field. The FST maps a term prefix to the on-disk block that holds all terms starting with that prefix. Each field's IndexStartFP points to its FST.
- DirOffset is a pointer to the start of the IndexStartFPs for all fields
- It's possible that an on-disk block would contain too many terms (more than the allowed maximum (default: 48)). When this happens, the block is sub-divided into new blocks (called "floor blocks"), and then the output in the FST for the block's prefix encodes the leading byte of each sub-block, and its file pointer.

See Also:
  `BlockTreeTermsReader`
WARNING: This API is experimental and might change in incompatible ways in the next release.

### Field Summary

Fields
`static int` `DEFAULT_MAX_BLOCK_SIZE` Suggested default value for the `maxItemsInBlock` parameter to `BlockTreeTermsWriter(SegmentWriteState,PostingsWriterBase,int,int)`.
`static int` `DEFAULT_MIN_BLOCK_SIZE` Suggested default value for the `minItemsInBlock` parameter to `BlockTreeTermsWriter(SegmentWriteState,PostingsWriterBase,int,int)`.

### Constructor Summary

Constructors
Constructor and Description
`BlockTreeTermsWriter(SegmentWriteState state, PostingsWriterBase postingsWriter, int minItemsInBlock, int maxItemsInBlock)` Create a new writer.

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods
`void` `close()`
`static void` `validateSettings(int minItemsInBlock, int maxItemsInBlock)` Throws `IllegalArgumentException` if any of these settings is invalid.
`void` `write(Fields fields, NormsProducer norms)` Write all fields, terms and postings.

### Methods inherited from class org.apache.lucene.codecs.FieldsConsumer

`merge`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### DEFAULT_MIN_BLOCK_SIZE

    public static final int DEFAULT_MIN_BLOCK_SIZE

Suggested default value for the `minItemsInBlock` parameter to `BlockTreeTermsWriter(SegmentWriteState,PostingsWriterBase,int,int)`.

See Also:
  Constant Field Values

#### DEFAULT_MAX_BLOCK_SIZE

    public static final int DEFAULT_MAX_BLOCK_SIZE

Suggested default value for the `maxItemsInBlock` parameter to `BlockTreeTermsWriter(SegmentWriteState,PostingsWriterBase,int,int)`.

See Also:
  Constant Field Values

### Constructor Detail

#### BlockTreeTermsWriter

    public BlockTreeTermsWriter(SegmentWriteState state,
                                PostingsWriterBase postingsWriter,
                                int minItemsInBlock,
                                int maxItemsInBlock)
                         throws IOException

Create a new writer. The number of items (terms or sub-blocks) per block will aim to be between minItemsPerBlock and maxItemsPerBlock, though in some cases the blocks may be smaller than the min.

Throws:
  `IOException`

### Method Detail

#### validateSettings

    public static void validateSettings(int minItemsInBlock,
                                        int maxItemsInBlock)

Throws `IllegalArgumentException` if any of these settings is invalid.

#### write

    public void write(Fields fields,
                      NormsProducer norms)
               throws IOException

Description copied from class: `FieldsConsumer`
Write all fields, terms and postings. This the "pull" API, allowing you to iterate more than once over the postings, somewhat analogous to using a DOM API to traverse an XML tree.

Notes:

- You must compute index statistics, including each Term's docFreq and totalTermFreq, as well as the summary sumTotalTermFreq, sumTotalDocFreq and docCount.
- You must skip terms that have no docs and fields that have no terms, even though the provided Fields API will expose them; this typically requires lazily writing the field or term until you've actually seen the first term or document.
- The provided Fields instance is limited: you cannot call any methods that return statistics/counts; you cannot pass a non-null live docs when pulling docs/positions enums.

Specified by:
  `write` in class `FieldsConsumer`
Throws:
  `IOException`

#### close

    public void close()
               throws IOException

Specified by:
  `close` in interface `Closeable`
Specified by:
  `close` in interface `AutoCloseable`
Specified by:
  `close` in class `FieldsConsumer`
Throws:
  `IOException`