org.apache.lucene.util

## Class OfflineSorter

- java.lang.Object
- - org.apache.lucene.util.OfflineSorter

-     public class OfflineSorter
      extends Object

  On-disk sorting of byte arrays. Each byte array (entry) is a composed of the following fields:
  - (two bytes) length of the following byte array,
  - exactly the above count of bytes for the sequence to be sorted.
  See Also:
    `sort(String)`
  WARNING: This API is experimental and might change in incompatible ways in the next release.
  NOTE: This API is for internal purposes only and might change in incompatible ways in the next release.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `OfflineSorter.BufferSize` A bit more descriptive unit for constructors.
    `static class` | `OfflineSorter.ByteSequencesReader` Utility class to read length-prefixed byte[] entries from an input.
    `static class` | `OfflineSorter.ByteSequencesWriter` Utility class to emit length-prefixed byte[] entries to an output stream for sorting.
    `class` | `OfflineSorter.SortInfo` Sort info (debugging mostly).

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static long` | `ABSOLUTE_MIN_SORT_BUFFER_SIZE` Absolute minimum required buffer size for sorting.
    `static Comparator<BytesRef>` | `DEFAULT_COMPARATOR` Default comparator: sorts in binary (codepoint) order
    `static long` | `GB` Convenience constant for gigabytes
    `static int` | `MAX_TEMPFILES` Maximum number of temporary files before doing an intermediate merge.
    `static long` | `MB` Convenience constant for megabytes
    `static long` | `MIN_BUFFER_SIZE_MB` Minimum recommended buffer size for sorting.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `OfflineSorter(Directory dir, String tempFileNamePrefix)` Defaults constructor.
    `OfflineSorter(Directory dir, String tempFileNamePrefix, Comparator<BytesRef> comparator)` Defaults constructor with a custom comparator.
    `OfflineSorter(Directory dir, String tempFileNamePrefix, Comparator<BytesRef> comparator, OfflineSorter.BufferSize ramBufferSize, int maxTempfiles, int valueLength, ExecutorService exec, int maxPartitionsInRAM)` All-details constructor.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `Comparator<BytesRef>` | `getComparator()` Returns the comparator in use to sort entries
    `Directory` | `getDirectory()` Returns the `Directory` we use to create temp files.
    `protected OfflineSorter.ByteSequencesReader` | `getReader(ChecksumIndexInput in, String name)` Subclasses can override to change how byte sequences are read from disk.
    `String` | `getTempFileNamePrefix()` Returns the temp file name prefix passed to `Directory.createTempOutput(java.lang.String, java.lang.String, org.apache.lucene.store.IOContext)` to generate temporary files.
    `protected OfflineSorter.ByteSequencesWriter` | `getWriter(IndexOutput out, long itemCount)` Subclasses can override to change how byte sequences are written to disk.
    `String` | `sort(String inputFileName)` Sort input to a new temp file, returning its name.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### MB

          public static final long MB

      Convenience constant for megabytes
      See Also:
        Constant Field Values

    - #### GB

          public static final long GB

      Convenience constant for gigabytes
      See Also:
        Constant Field Values

    - #### MIN_BUFFER_SIZE_MB

          public static final long MIN_BUFFER_SIZE_MB

      Minimum recommended buffer size for sorting.
      See Also:
        Constant Field Values

    - #### ABSOLUTE_MIN_SORT_BUFFER_SIZE

          public static final long ABSOLUTE_MIN_SORT_BUFFER_SIZE

      Absolute minimum required buffer size for sorting.
      See Also:
        Constant Field Values

    - #### MAX_TEMPFILES

          public static final int MAX_TEMPFILES

      Maximum number of temporary files before doing an intermediate merge.
      See Also:
        Constant Field Values

    - #### DEFAULT_COMPARATOR

          public static final Comparator<BytesRef> DEFAULT_COMPARATOR

      Default comparator: sorts in binary (codepoint) order

  - ### Constructor Detail

    - #### OfflineSorter

          public OfflineSorter(Directory dir,
                               String tempFileNamePrefix)
                        throws IOException

      Defaults constructor.
      Throws:
        `IOException`
      See Also:
        `OfflineSorter.BufferSize.automatic()`

    - #### OfflineSorter

          public OfflineSorter(Directory dir,
                               String tempFileNamePrefix,
                               Comparator<BytesRef> comparator)
                        throws IOException

      Defaults constructor with a custom comparator.
      Throws:
        `IOException`
      See Also:
        `OfflineSorter.BufferSize.automatic()`

    - #### OfflineSorter

          public OfflineSorter(Directory dir,
                               String tempFileNamePrefix,
                               Comparator<BytesRef> comparator,
                               OfflineSorter.BufferSize ramBufferSize,
                               int maxTempfiles,
                               int valueLength,
                               ExecutorService exec,
                               int maxPartitionsInRAM)

      All-details constructor. If `valueLength` is -1 (the default), the length of each value differs; otherwise, all values have the specified length. If you pass a non-null `ExecutorService` then it will be used to run sorting operations that can be run concurrently, and maxPartitionsInRAM is the maximum concurrent in-memory partitions. Thus the maximum possible RAM used by this class while sorting is `maxPartitionsInRAM * ramBufferSize`.

  - ### Method Detail

    - #### getDirectory

          public Directory getDirectory()

      Returns the `Directory` we use to create temp files.

    - #### getTempFileNamePrefix

          public String getTempFileNamePrefix()

      Returns the temp file name prefix passed to `Directory.createTempOutput(java.lang.String, java.lang.String, org.apache.lucene.store.IOContext)` to generate temporary files.

    - #### sort

          public String sort(String inputFileName)
                      throws IOException

      Sort input to a new temp file, returning its name.
      Throws:
        `IOException`

    - #### getWriter

          protected OfflineSorter.ByteSequencesWriter getWriter(IndexOutput out,
                                                                long itemCount)
                                                         throws IOException

      Subclasses can override to change how byte sequences are written to disk.
      Throws:
        `IOException`

    - #### getReader

          protected OfflineSorter.ByteSequencesReader getReader(ChecksumIndexInput in,
                                                                String name)
                                                         throws IOException

      Subclasses can override to change how byte sequences are read from disk.
      Throws:
        `IOException`

    - #### getComparator

          public Comparator<BytesRef> getComparator()

      Returns the comparator in use to sort entries

Copyright © 2000-2024 Apache Software Foundation. All Rights Reserved.