org.apache.lucene.util.bkd

## Class BKDWriter

- java.lang.Object
- - org.apache.lucene.util.bkd.BKDWriter

- All Implemented Interfaces:
    Closeable, AutoCloseable

      public class BKDWriter
      extends Object
      implements Closeable

  Recursively builds a block KD-tree to assign all incoming points in N-dim space to smaller and smaller N-dim rectangles (cells) until the number of points in a given rectangle is <= `maxPointsInLeafNode`. The tree is fully balanced, which means the leaf nodes will have between 50% and 100% of the requested `maxPointsInLeafNode`. Values that fall exactly on a cell boundary may be in either cell.

  The number of dimensions can be 1 to 8, but every byte[] value is fixed length.

  See this paper for details.

  This consumes heap during writing: it allocates a `LongBitSet(numPoints)`, and then uses up to the specified `maxMBSortInHeap` heap space for writing.

  NOTE: This can write at most Integer.MAX_VALUE * `maxPointsInLeafNode` total points.

  WARNING: This API is experimental and might change in incompatible ways in the next release.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected int` | `bytesPerDim` How many bytes each value in each dimension takes.
    `static String` | `CODEC_NAME`
    `static float` | `DEFAULT_MAX_MB_SORT_IN_HEAP` Default maximum heap to use, before spilling to (slower) disk
    `static int` | `DEFAULT_MAX_POINTS_IN_LEAF_NODE` Default maximum number of point in each leaf block
    `protected FixedBitSet` | `docsSeen`
    `protected boolean` | `longOrds` true if we have so many values that we must write ords using long (8 bytes) instead of int (4 bytes)
    `static int` | `MAX_DIMS` Maximum number of dimensions
    `protected byte[]` | `maxPackedValue` Maximum per-dim values, packed
    `protected int` | `maxPointsInLeafNode`
    `protected byte[]` | `minPackedValue` Minimum per-dim values, packed
    `protected int` | `numDims` How many dimensions we are indexing
    `protected OfflineSorter.BufferSize` | `offlineSorterBufferMB` How much heap OfflineSorter is allowed to use
    `protected int` | `offlineSorterMaxTempFiles` How much heap OfflineSorter is allowed to use
    `protected int` | `packedBytesLength` numDims * bytesPerDim
    `protected long` | `pointCount`
    `protected boolean` | `singleValuePerDoc` True if every document has at most one value.
    `static int` | `VERSION_COMPRESSED_DOC_IDS`
    `static int` | `VERSION_COMPRESSED_VALUES`
    `static int` | `VERSION_CURRENT`
    `static int` | `VERSION_IMPLICIT_SPLIT_DIM_1D`
    `static int` | `VERSION_LEAF_STORES_BOUNDS`
    `static int` | `VERSION_PACKED_INDEX`
    `static int` | `VERSION_START`

  - ### Constructor Summary

    Constructors
    Modifier | Constructor and Description
    --- | ---
    `` | `BKDWriter(int maxDoc, Directory tempDir, String tempFileNamePrefix, int numDims, int bytesPerDim, int maxPointsInLeafNode, double maxMBSortInHeap, long totalPointCount, boolean singleValuePerDoc)`
    `protected` | `BKDWriter(int maxDoc, Directory tempDir, String tempFileNamePrefix, int numDims, int bytesPerDim, int maxPointsInLeafNode, double maxMBSortInHeap, long totalPointCount, boolean singleValuePerDoc, boolean longOrds, long offlineSorterBufferMB, int offlineSorterMaxTempFiles)`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `add(byte[] packedValue, int docID)`
    `void` | `close()`
    `long` | `finish(IndexOutput out)` Writes the BKD tree to the provided `IndexOutput` and returns the file offset where index was written.
    `long` | `getPointCount()` How many points have been added so far
    `long` | `merge(IndexOutput out, List<MergeState.DocMap> docMaps, List<BKDReader> readers)` More efficient bulk-add for incoming `BKDReader`s.
    `protected int` | `split(byte[] minPackedValue, byte[] maxPackedValue, int[] parentSplits)` Pick the next dimension to split.
    `static void` | `verifyParams(int numDims, int maxPointsInLeafNode, double maxMBSortInHeap, long totalPointCount)`
    `long` | `writeField(IndexOutput out, String fieldName, MutablePointValues reader)` Write a field from a `MutablePointValues`.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### CODEC_NAME

          public static final String CODEC_NAME

      See Also:
        Constant Field Values

    - #### VERSION_START

          public static final int VERSION_START

      See Also:
        Constant Field Values

    - #### VERSION_COMPRESSED_DOC_IDS

          public static final int VERSION_COMPRESSED_DOC_IDS

      See Also:
        Constant Field Values

    - #### VERSION_COMPRESSED_VALUES

          public static final int VERSION_COMPRESSED_VALUES

      See Also:
        Constant Field Values

    - #### VERSION_IMPLICIT_SPLIT_DIM_1D

          public static final int VERSION_IMPLICIT_SPLIT_DIM_1D

      See Also:
        Constant Field Values

    - #### VERSION_PACKED_INDEX

          public static final int VERSION_PACKED_INDEX

      See Also:
        Constant Field Values

    - #### VERSION_LEAF_STORES_BOUNDS

          public static final int VERSION_LEAF_STORES_BOUNDS

      See Also:
        Constant Field Values

    - #### VERSION_CURRENT

          public static final int VERSION_CURRENT

      See Also:
        Constant Field Values

    - #### DEFAULT_MAX_POINTS_IN_LEAF_NODE

          public static final int DEFAULT_MAX_POINTS_IN_LEAF_NODE

      Default maximum number of point in each leaf block
      See Also:
        Constant Field Values

    - #### DEFAULT_MAX_MB_SORT_IN_HEAP

          public static final float DEFAULT_MAX_MB_SORT_IN_HEAP

      Default maximum heap to use, before spilling to (slower) disk
      See Also:
        Constant Field Values

    - #### MAX_DIMS

          public static final int MAX_DIMS

      Maximum number of dimensions
      See Also:
        Constant Field Values

    - #### numDims

          protected final int numDims

      How many dimensions we are indexing

    - #### bytesPerDim

          protected final int bytesPerDim

      How many bytes each value in each dimension takes.

    - #### packedBytesLength

          protected final int packedBytesLength

      numDims * bytesPerDim

    - #### docsSeen

          protected final FixedBitSet docsSeen

    - #### maxPointsInLeafNode

          protected final int maxPointsInLeafNode

    - #### minPackedValue

          protected final byte[] minPackedValue

      Minimum per-dim values, packed

    - #### maxPackedValue

          protected final byte[] maxPackedValue

      Maximum per-dim values, packed

    - #### pointCount

          protected long pointCount

    - #### longOrds

          protected final boolean longOrds

      true if we have so many values that we must write ords using long (8 bytes) instead of int (4 bytes)

    - #### singleValuePerDoc

          protected final boolean singleValuePerDoc

      True if every document has at most one value. We specialize this case by not bothering to store the ord since it's redundant with docID.

    - #### offlineSorterBufferMB

          protected final OfflineSorter.BufferSize offlineSorterBufferMB

      How much heap OfflineSorter is allowed to use

    - #### offlineSorterMaxTempFiles

          protected final int offlineSorterMaxTempFiles

      How much heap OfflineSorter is allowed to use

  - ### Constructor Detail

    - #### BKDWriter

          public BKDWriter(int maxDoc,
                           Directory tempDir,
                           String tempFileNamePrefix,
                           int numDims,
                           int bytesPerDim,
                           int maxPointsInLeafNode,
                           double maxMBSortInHeap,
                           long totalPointCount,
                           boolean singleValuePerDoc)
                    throws IOException

      Throws:
        `IOException`

    - #### BKDWriter

          protected BKDWriter(int maxDoc,
                              Directory tempDir,
                              String tempFileNamePrefix,
                              int numDims,
                              int bytesPerDim,
                              int maxPointsInLeafNode,
                              double maxMBSortInHeap,
                              long totalPointCount,
                              boolean singleValuePerDoc,
                              boolean longOrds,
                              long offlineSorterBufferMB,
                              int offlineSorterMaxTempFiles)
                       throws IOException

      Throws:
        `IOException`

  - ### Method Detail

    - #### verifyParams

          public static void verifyParams(int numDims,
                                          int maxPointsInLeafNode,
                                          double maxMBSortInHeap,
                                          long totalPointCount)

    - #### add

          public void add(byte[] packedValue,
                          int docID)
                   throws IOException

      Throws:
        `IOException`

    - #### getPointCount

          public long getPointCount()

      How many points have been added so far

    - #### writeField

          public long writeField(IndexOutput out,
                                 String fieldName,
                                 MutablePointValues reader)
                          throws IOException

      Write a field from a `MutablePointValues`. This way of writing points is faster than regular writes with `add(byte[], int)` since there is opportunity for reordering points before writing them to disk. This method does not use transient disk in order to reorder points.
      Throws:
        `IOException`

    - #### merge

          public long merge(IndexOutput out,
                            List<MergeState.DocMap> docMaps,
                            List<BKDReader> readers)
                     throws IOException

      More efficient bulk-add for incoming `BKDReader`s. This does a merge sort of the already sorted values and currently only works when numDims==1. This returns -1 if all documents containing dimensional values were deleted.
      Throws:
        `IOException`

    - #### finish

          public long finish(IndexOutput out)
                      throws IOException

      Writes the BKD tree to the provided `IndexOutput` and returns the file offset where index was written.
      Throws:
        `IOException`

    - #### close

          public void close()
                     throws IOException

      Specified by:
        `close` in interface `Closeable`
      Specified by:
        `close` in interface `AutoCloseable`
      Throws:
        `IOException`

    - #### split

          protected int split(byte[] minPackedValue,
                              byte[] maxPackedValue,
                              int[] parentSplits)

      Pick the next dimension to split.
      Parameters:
        `minPackedValue` - the min values for all dimensions
        `maxPackedValue` - the max values for all dimensions
        `parentSplits` - how many times each dim has been split on the parent levels
      Returns:
        the dimension to split

Copyright © 2000-2018 Apache Software Foundation. All Rights Reserved.