org.apache.lucene.index

## Class CheckIndex

- java.lang.Object
- - org.apache.lucene.index.CheckIndex

- All Implemented Interfaces:
    Closeable, AutoCloseable

      public final class CheckIndex
      extends Object
      implements Closeable

  Basic tool and API to check the health of an index and write a new segments file that removes reference to problematic segments.

  As this tool checks every byte in the index, on a large index it can take quite a long time to run.

  WARNING: This API is experimental and might change in incompatible ways in the next release.
    Please make a complete backup of your index before using this to exorcise corrupted documents from your index!

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `CheckIndex.Options` Run-time configuration options for CheckIndex commands.
    `static class` | `CheckIndex.Status` Returned from `checkIndex()` detailing the health and status of the index.
    `static class` | `CheckIndex.VerifyPointsVisitor` Walks the entire N-dimensional points space, verifying that all points fall within the last cell's boundaries.

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `CheckIndex(Directory dir)` Create a new CheckIndex on the directory.
    `CheckIndex(Directory dir, Lock writeLock)` Expert: create a directory with the specified lock.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static boolean` | `assertsOn()` Check whether asserts are enabled or not.
    `CheckIndex.Status` | `checkIndex()` Returns a `CheckIndex.Status` instance detailing the state of the index.
    `CheckIndex.Status` | `checkIndex(List<String> onlySegments)` Returns a `CheckIndex.Status` instance detailing the state of the index.
    `void` | `close()`
    `int` | `doCheck(CheckIndex.Options opts)` Actually perform the index check
    `void` | `exorciseIndex(CheckIndex.Status result)` Repairs the index using previously returned result from `checkIndex()`.
    `boolean` | `getChecksumsOnly()` See `getChecksumsOnly()`.
    `boolean` | `getCrossCheckTermVectors()` See `setCrossCheckTermVectors(boolean)`.
    `boolean` | `getFailFast()` See `setFailFast(boolean)`.
    `static void` | `main(String[] args)` Command-line interface to check and exorcise corrupt segments from an index.
    `static CheckIndex.Options` | `parseOptions(String[] args)` Parse command line args into fields
    `void` | `setChecksumsOnly(boolean v)` If true, only validate physical integrity for all files.
    `void` | `setCrossCheckTermVectors(boolean v)` If true, term vectors are compared against postings to make sure they are the same.
    `void` | `setFailFast(boolean v)` If true, just throw the original exception immediately when corruption is detected, rather than continuing to iterate to other segments looking for more corruption.
    `void` | `setInfoStream(PrintStream out)` Set infoStream where messages should go.
    `void` | `setInfoStream(PrintStream out, boolean verbose)` Set infoStream where messages should go.
    `static CheckIndex.Status.DocValuesStatus` | `testDocValues(CodecReader reader, PrintStream infoStream, boolean failFast)` Test docvalues.
    `static CheckIndex.Status.FieldInfoStatus` | `testFieldInfos(CodecReader reader, PrintStream infoStream, boolean failFast)` Test field infos.
    `static CheckIndex.Status.FieldNormStatus` | `testFieldNorms(CodecReader reader, PrintStream infoStream, boolean failFast)` Test field norms.
    `static CheckIndex.Status.LiveDocStatus` | `testLiveDocs(CodecReader reader, PrintStream infoStream, boolean failFast)` Test live docs.
    `static CheckIndex.Status.PointsStatus` | `testPoints(CodecReader reader, PrintStream infoStream, boolean failFast)` Test the points index
    `static CheckIndex.Status.TermIndexStatus` | `testPostings(CodecReader reader, PrintStream infoStream)` Test the term index.
    `static CheckIndex.Status.TermIndexStatus` | `testPostings(CodecReader reader, PrintStream infoStream, boolean verbose, boolean failFast)` Test the term index.
    `static CheckIndex.Status.IndexSortStatus` | `testSort(CodecReader reader, Sort sort, PrintStream infoStream, boolean failFast)` Tests index sort order.
    `static CheckIndex.Status.StoredFieldStatus` | `testStoredFields(CodecReader reader, PrintStream infoStream, boolean failFast)` Test stored fields.
    `static CheckIndex.Status.TermVectorStatus` | `testTermVectors(CodecReader reader, PrintStream infoStream)` Test term vectors.
    `static CheckIndex.Status.TermVectorStatus` | `testTermVectors(CodecReader reader, PrintStream infoStream, boolean verbose, boolean crossCheckTermVectors, boolean failFast)` Test term vectors.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### CheckIndex

          public CheckIndex(Directory dir)
                     throws IOException

      Create a new CheckIndex on the directory.
      Throws:
        `IOException`

    - #### CheckIndex

          public CheckIndex(Directory dir,
                            Lock writeLock)
                     throws IOException

      Expert: create a directory with the specified lock. This should really not be used except for unit tests!!!! It exists only to support special tests (such as TestIndexWriterExceptions*), that would otherwise be more complicated to debug if they had to close the writer for each check.
      Throws:
        `IOException`

  - ### Method Detail

    - #### close

          public void close()
                     throws IOException

      Specified by:
        `close` in interface `Closeable`
      Specified by:
        `close` in interface `AutoCloseable`
      Throws:
        `IOException`

    - #### setCrossCheckTermVectors

          public void setCrossCheckTermVectors(boolean v)

      If true, term vectors are compared against postings to make sure they are the same. This will likely drastically increase time it takes to run CheckIndex!

    - #### getCrossCheckTermVectors

          public boolean getCrossCheckTermVectors()

      See `setCrossCheckTermVectors(boolean)`.

    - #### setFailFast

          public void setFailFast(boolean v)

      If true, just throw the original exception immediately when corruption is detected, rather than continuing to iterate to other segments looking for more corruption.

    - #### getFailFast

          public boolean getFailFast()

      See `setFailFast(boolean)`.

    - #### getChecksumsOnly

          public boolean getChecksumsOnly()

      See `getChecksumsOnly()`.

    - #### setChecksumsOnly

          public void setChecksumsOnly(boolean v)

      If true, only validate physical integrity for all files. Note that the returned nested status objects (e.g. storedFieldStatus) will be null.

    - #### setInfoStream

          public void setInfoStream(PrintStream out,
                                    boolean verbose)

      Set infoStream where messages should go. If null, no messages are printed. If verbose is true then more details are printed.

    - #### setInfoStream

          public void setInfoStream(PrintStream out)

      Set infoStream where messages should go. See `setInfoStream(PrintStream,boolean)`.

    - #### checkIndex

          public CheckIndex.Status checkIndex()
                                       throws IOException

      Returns a `CheckIndex.Status` instance detailing the state of the index.

      As this method checks every byte in the index, on a large index it can take quite a long time to run.

      WARNING: make sure you only call this when the index is not opened by any writer.

      Throws:
        `IOException`

    - #### checkIndex

          public CheckIndex.Status checkIndex(List<String> onlySegments)
                                       throws IOException

      Returns a `CheckIndex.Status` instance detailing the state of the index.
      Parameters:
        `onlySegments` - list of specific segment names to check

        As this method checks every byte in the specified segments, on a large index it can take quite a long time to run.

      Throws:
        `IOException`

    - #### testSort

          public static CheckIndex.Status.IndexSortStatus testSort(CodecReader reader,
                                                                   Sort sort,
                                                                   PrintStream infoStream,
                                                                   boolean failFast)
                                                            throws IOException

      Tests index sort order.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testLiveDocs

          public static CheckIndex.Status.LiveDocStatus testLiveDocs(CodecReader reader,
                                                                     PrintStream infoStream,
                                                                     boolean failFast)
                                                              throws IOException

      Test live docs.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testFieldInfos

          public static CheckIndex.Status.FieldInfoStatus testFieldInfos(CodecReader reader,
                                                                         PrintStream infoStream,
                                                                         boolean failFast)
                                                                  throws IOException

      Test field infos.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testFieldNorms

          public static CheckIndex.Status.FieldNormStatus testFieldNorms(CodecReader reader,
                                                                         PrintStream infoStream,
                                                                         boolean failFast)
                                                                  throws IOException

      Test field norms.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testPostings

          public static CheckIndex.Status.TermIndexStatus testPostings(CodecReader reader,
                                                                       PrintStream infoStream)
                                                                throws IOException

      Test the term index.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testPostings

          public static CheckIndex.Status.TermIndexStatus testPostings(CodecReader reader,
                                                                       PrintStream infoStream,
                                                                       boolean verbose,
                                                                       boolean failFast)
                                                                throws IOException

      Test the term index.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testPoints

          public static CheckIndex.Status.PointsStatus testPoints(CodecReader reader,
                                                                  PrintStream infoStream,
                                                                  boolean failFast)
                                                           throws IOException

      Test the points index
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testStoredFields

          public static CheckIndex.Status.StoredFieldStatus testStoredFields(CodecReader reader,
                                                                             PrintStream infoStream,
                                                                             boolean failFast)
                                                                      throws IOException

      Test stored fields.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testDocValues

          public static CheckIndex.Status.DocValuesStatus testDocValues(CodecReader reader,
                                                                        PrintStream infoStream,
                                                                        boolean failFast)
                                                                 throws IOException

      Test docvalues.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testTermVectors

          public static CheckIndex.Status.TermVectorStatus testTermVectors(CodecReader reader,
                                                                           PrintStream infoStream)
                                                                    throws IOException

      Test term vectors.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### testTermVectors

          public static CheckIndex.Status.TermVectorStatus testTermVectors(CodecReader reader,
                                                                           PrintStream infoStream,
                                                                           boolean verbose,
                                                                           boolean crossCheckTermVectors,
                                                                           boolean failFast)
                                                                    throws IOException

      Test term vectors.
      Throws:
        `IOException`
      WARNING: This API is experimental and might change in incompatible ways in the next release.

    - #### exorciseIndex

          public void exorciseIndex(CheckIndex.Status result)
                             throws IOException

      Repairs the index using previously returned result from `checkIndex()`. Note that this does not remove any of the unreferenced files after it's done; you must separately open an `IndexWriter`, which deletes unreferenced files when it's created.

      WARNING: this writes a new segments file into the index, effectively removing all documents in broken segments from the index. BE CAREFUL.

      Throws:
        `IOException`

    - #### assertsOn

          public static boolean assertsOn()

      Check whether asserts are enabled or not.
      Returns:
        true iff asserts are enabled

    - #### main

          public static void main(String[] args)
                           throws IOException,
                                  InterruptedException

      Command-line interface to check and exorcise corrupt segments from an index.

      Run it like this:

              java -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex pathToIndex [-exorcise] [-verbose] [-segment X] [-segment Y]

      - `-exorcise`: actually write a new segments_N file, removing any problematic segments. *LOSES DATA*
      - `-segment X`: only check the specified segment(s). This can be specified multiple times, to check more than one segment, eg `-segment _2 -segment _a`. You can't use this with the -exorcise option.

      WARNING: `-exorcise` should only be used on an emergency basis as it will cause documents (perhaps many) to be permanently removed from the index. Always make a backup copy of your index before running this! Do not run this tool on an index that is actively being written to. You have been warned!

      Run without -exorcise, this tool will open the index, report version information and report any exceptions it hits and what action it would take if -exorcise were specified. With -exorcise, this tool will remove any segments that have issues and write a new segments_N file. This means all documents contained in the affected segments will be removed.

      This tool exits with exit code 1 if the index cannot be opened or has any corruption, else 0.

      Throws:
        `IOException`
        `InterruptedException`

    - #### parseOptions

          public static CheckIndex.Options parseOptions(String[] args)

      Parse command line args into fields
      Parameters:
        `args` - The command line arguments
      Returns:
        An Options struct
      Throws:
        `IllegalArgumentException` - if any of the CLI args are invalid

    - #### doCheck

          public int doCheck(CheckIndex.Options opts)
                      throws IOException,
                             InterruptedException

      Actually perform the index check
      Parameters:
        `opts` - The options to use for this check
      Returns:
        0 iff the index is clean, 1 otherwise
      Throws:
        `IOException`
        `InterruptedException`

Copyright © 2000-2019 Apache Software Foundation. All Rights Reserved.