org.apache.lucene.index

## Class TieredMergePolicy

- java.lang.Object
- - org.apache.lucene.index.MergePolicy
  - - org.apache.lucene.index.TieredMergePolicy

-     public class TieredMergePolicy
      extends MergePolicy

  Merges segments of approximately equal size, subject to an allowed number of segments per tier. This is similar to `LogByteSizeMergePolicy`, except this merge policy is able to merge non-adjacent segment, and separates how many segments are merged at once (`setMaxMergeAtOnce(int)`) from how many segments are allowed per tier (`setSegmentsPerTier(double)`). This merge policy also does not over-merge (i.e. cascade merges).

  For normal merging, this policy first computes a "budget" of how many segments are allowed to be in the index. If the index is over-budget, then the policy sorts segments by decreasing size (pro-rating by percent deletes), and then finds the least-cost merge. Merge cost is measured by a combination of the "skew" of the merge (size of largest segment divided by smallest segment), total merge size and percent deletes reclaimed, so that merges with lower skew, smaller size and those reclaiming more deletes, are favored.

  If a merge will produce a segment that's larger than `setMaxMergedSegmentMB(double)`, then the policy will merge fewer segments (down to 1 at once, if that one has deletions) to keep the segment size under budget.

  NOTE: this policy freely merges non-adjacent segments; if this is a problem, use `LogMergePolicy`.

  NOTE: This policy always merges by byte size of the segments, always pro-rates by percent deletes

  NOTE Starting with Lucene 7.5, there are several changes: - findForcedMerges and findForcedDeletesMerges) respect the max segment size by default. - When findforcedmerges is called with maxSegmentCount other than 1, the resulting index is not guaranteed to have <= maxSegmentCount segments. Rather it is on a "best effort" basis. Specifically the theoretical ideal segment size is calculated and a "fudge factor" of 25% is added as the new maxSegmentSize, which is respected. - findForcedDeletesMerges will not produce segments greater than maxSegmentSize.

  WARNING: This API is experimental and might change in incompatible ways in the next release.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `protected static class` | `TieredMergePolicy.MergeScore` Holds score and explanation for a single candidate merge.

    - ### Nested classes/interfaces inherited from class org.apache.lucene.index.MergePolicy

      `MergePolicy.MergeAbortedException, MergePolicy.MergeContext, MergePolicy.MergeException, MergePolicy.MergeSpecification, MergePolicy.OneMerge, MergePolicy.OneMergeProgress`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static double` | `DEFAULT_NO_CFS_RATIO` Default noCFSRatio.

    - ### Fields inherited from class org.apache.lucene.index.MergePolicy

      `DEFAULT_MAX_CFS_SEGMENT_SIZE, maxCFSSegmentSize, noCFSRatio`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `TieredMergePolicy()` Sole constructor, setting all settings to their defaults.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `MergePolicy.MergeSpecification` | `findForcedDeletesMerges(SegmentInfos infos, MergePolicy.MergeContext mergeContext)` Determine what set of merge operations is necessary in order to expunge all deletes from the index.
    `MergePolicy.MergeSpecification` | `findForcedMerges(SegmentInfos infos, int maxSegmentCount, Map<SegmentCommitInfo,Boolean> segmentsToMerge, MergePolicy.MergeContext mergeContext)` Determine what set of merge operations is necessary in order to merge to `<=` the specified segment count.
    `MergePolicy.MergeSpecification` | `findMerges(MergeTrigger mergeTrigger, SegmentInfos infos, MergePolicy.MergeContext mergeContext)` Determine what set of merge operations are now necessary on the index.
    `double` | `getDeletesPctAllowed()` Returns the current deletesPctAllowed setting.
    `double` | `getFloorSegmentMB()` Returns the current floorSegmentMB.
    `double` | `getForceMergeDeletesPctAllowed()` Returns the current forceMergeDeletesPctAllowed setting.
    `int` | `getMaxMergeAtOnce()` Returns the current maxMergeAtOnce setting.
    `int` | `getMaxMergeAtOnceExplicit()` Returns the current maxMergeAtOnceExplicit setting.
    `double` | `getMaxMergedSegmentMB()` Returns the current maxMergedSegmentMB setting.
    `double` | `getSegmentsPerTier()` Returns the current segmentsPerTier setting.
    `protected TieredMergePolicy.MergeScore` | `score(List<SegmentCommitInfo> candidate, boolean hitTooLarge, Map<SegmentCommitInfo,org.apache.lucene.index.TieredMergePolicy.SegmentSizeAndDocs> segmentsSizes)` Expert: scores one merge; subclasses can override.
    `TieredMergePolicy` | `setDeletesPctAllowed(double v)` Controls the maximum percentage of deleted documents that is tolerated in the index.
    `TieredMergePolicy` | `setFloorSegmentMB(double v)` Segments smaller than this are "rounded up" to this size, ie treated as equal (floor) size for merge selection.
    `TieredMergePolicy` | `setForceMergeDeletesPctAllowed(double v)` When forceMergeDeletes is called, we only merge away a segment if its delete percentage is over this threshold.
    `TieredMergePolicy` | `setMaxMergeAtOnce(int v)` Maximum number of segments to be merged at a time during "normal" merging.
    `TieredMergePolicy` | `setMaxMergeAtOnceExplicit(int v)` Maximum number of segments to be merged at a time, during forceMerge or forceMergeDeletes.
    `TieredMergePolicy` | `setMaxMergedSegmentMB(double v)` Maximum sized segment to produce during normal merging.
    `TieredMergePolicy` | `setSegmentsPerTier(double v)` Sets the allowed number of segments per tier.
    `String` | `toString()`

    - ### Methods inherited from class org.apache.lucene.index.MergePolicy

      `assertDelCount, getMaxCFSSegmentSizeMB, getNoCFSRatio, isMerged, keepFullyDeletedSegment, message, numDeletesToMerge, segString, setMaxCFSSegmentSizeMB, setNoCFSRatio, size, useCompoundFile, verbose`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### DEFAULT_NO_CFS_RATIO

          public static final double DEFAULT_NO_CFS_RATIO

      Default noCFSRatio. If a merge's size is `>= 10%` of the index, then we disable compound file for it.
      See Also:
        `MergePolicy.setNoCFSRatio(double)`, Constant Field Values

  - ### Constructor Detail

    - #### TieredMergePolicy

          public TieredMergePolicy()

      Sole constructor, setting all settings to their defaults.

  - ### Method Detail

    - #### setMaxMergeAtOnce

          public TieredMergePolicy setMaxMergeAtOnce(int v)

      Maximum number of segments to be merged at a time during "normal" merging. For explicit merging (eg, forceMerge or forceMergeDeletes was called), see `setMaxMergeAtOnceExplicit(int)`. Default is 10.

    - #### getMaxMergeAtOnce

          public int getMaxMergeAtOnce()

      Returns the current maxMergeAtOnce setting.
      See Also:
        `setMaxMergeAtOnce(int)`

    - #### setMaxMergeAtOnceExplicit

          public TieredMergePolicy setMaxMergeAtOnceExplicit(int v)

      Maximum number of segments to be merged at a time, during forceMerge or forceMergeDeletes. Default is 30.

    - #### getMaxMergeAtOnceExplicit

          public int getMaxMergeAtOnceExplicit()

      Returns the current maxMergeAtOnceExplicit setting.
      See Also:
        `setMaxMergeAtOnceExplicit(int)`

    - #### setMaxMergedSegmentMB

          public TieredMergePolicy setMaxMergedSegmentMB(double v)

      Maximum sized segment to produce during normal merging. This setting is approximate: the estimate of the merged segment size is made by summing sizes of to-be-merged segments (compensating for percent deleted docs). Default is 5 GB.

    - #### getMaxMergedSegmentMB

          public double getMaxMergedSegmentMB()

      Returns the current maxMergedSegmentMB setting.
      See Also:
        `setMaxMergedSegmentMB(double)`

    - #### setDeletesPctAllowed

          public TieredMergePolicy setDeletesPctAllowed(double v)

      Controls the maximum percentage of deleted documents that is tolerated in the index. Lower values make the index more space efficient at the expense of increased CPU and I/O activity. Values must be between 20 and 50. Default value is 33.

    - #### getDeletesPctAllowed

          public double getDeletesPctAllowed()

      Returns the current deletesPctAllowed setting.
      See Also:
        `setDeletesPctAllowed(double)`

    - #### setFloorSegmentMB

          public TieredMergePolicy setFloorSegmentMB(double v)

      Segments smaller than this are "rounded up" to this size, ie treated as equal (floor) size for merge selection. This is to prevent frequent flushing of tiny segments from allowing a long tail in the index. Default is 2 MB.

    - #### getFloorSegmentMB

          public double getFloorSegmentMB()

      Returns the current floorSegmentMB.
      See Also:
        `setFloorSegmentMB(double)`

    - #### setForceMergeDeletesPctAllowed

          public TieredMergePolicy setForceMergeDeletesPctAllowed(double v)

      When forceMergeDeletes is called, we only merge away a segment if its delete percentage is over this threshold. Default is 10%.

    - #### getForceMergeDeletesPctAllowed

          public double getForceMergeDeletesPctAllowed()

      Returns the current forceMergeDeletesPctAllowed setting.
      See Also:
        `setForceMergeDeletesPctAllowed(double)`

    - #### setSegmentsPerTier

          public TieredMergePolicy setSegmentsPerTier(double v)

      Sets the allowed number of segments per tier. Smaller values mean more merging but fewer segments.

      Default is 10.0.

    - #### getSegmentsPerTier

          public double getSegmentsPerTier()

      Returns the current segmentsPerTier setting.
      See Also:
        `setSegmentsPerTier(double)`

    - #### findMerges

          public MergePolicy.MergeSpecification findMerges(MergeTrigger mergeTrigger,
                                                           SegmentInfos infos,
                                                           MergePolicy.MergeContext mergeContext)
                                                    throws IOException

      Description copied from class: `MergePolicy`
      Determine what set of merge operations are now necessary on the index. `IndexWriter` calls this whenever there is a change to the segments. This call is always synchronized on the `IndexWriter` instance so only one thread at a time will call this method.
      Specified by:
        `findMerges` in class `MergePolicy`
      Parameters:
        `mergeTrigger` - the event that triggered the merge
        `infos` - the total set of segments in the index
        `mergeContext` - the IndexWriter to find the merges on
      Throws:
        `IOException`

    - #### score

          protected TieredMergePolicy.MergeScore score(List<SegmentCommitInfo> candidate,
                                                       boolean hitTooLarge,
                                                       Map<SegmentCommitInfo,org.apache.lucene.index.TieredMergePolicy.SegmentSizeAndDocs> segmentsSizes)
                                                throws IOException

      Expert: scores one merge; subclasses can override.
      Throws:
        `IOException`

    - #### findForcedMerges

          public MergePolicy.MergeSpecification findForcedMerges(SegmentInfos infos,
                                                                 int maxSegmentCount,
                                                                 Map<SegmentCommitInfo,Boolean> segmentsToMerge,
                                                                 MergePolicy.MergeContext mergeContext)
                                                          throws IOException

      Description copied from class: `MergePolicy`
      Determine what set of merge operations is necessary in order to merge to `<=` the specified segment count. `IndexWriter` calls this when its `IndexWriter.forceMerge(int)` method is called. This call is always synchronized on the `IndexWriter` instance so only one thread at a time will call this method.
      Specified by:
        `findForcedMerges` in class `MergePolicy`
      Parameters:
        `infos` - the total set of segments in the index
        `maxSegmentCount` - requested maximum number of segments in the index (currently this is always 1)
        `segmentsToMerge` - contains the specific SegmentInfo instances that must be merged away. This may be a subset of all SegmentInfos. If the value is True for a given SegmentInfo, that means this segment was an original segment present in the to-be-merged index; else, it was a segment produced by a cascaded merge.
        `mergeContext` - the IndexWriter to find the merges on
      Throws:
        `IOException`

    - #### findForcedDeletesMerges

          public MergePolicy.MergeSpecification findForcedDeletesMerges(SegmentInfos infos,
                                                                        MergePolicy.MergeContext mergeContext)
                                                                 throws IOException

      Description copied from class: `MergePolicy`
      Determine what set of merge operations is necessary in order to expunge all deletes from the index.
      Specified by:
        `findForcedDeletesMerges` in class `MergePolicy`
      Parameters:
        `infos` - the total set of segments in the index
        `mergeContext` - the IndexWriter to find the merges on
      Throws:
        `IOException`

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

Copyright © 2000-2019 Apache Software Foundation. All Rights Reserved.