org.apache.lucene.index

## Class CheckIndex

- java.lang.Object
- - org.apache.lucene.index.CheckIndex

All Implemented Interfaces:
  Closeable, AutoCloseable

    public final class CheckIndex
    extends Object
    implements Closeable

Basic tool and API to check the health of an index and write a new segments file that removes reference to problematic segments.

As this tool checks every byte in the index, on a large index it can take quite a long time to run.

WARNING: This API is experimental and might change in incompatible ways in the next release.
  Please make a complete backup of your index before using this to exorcise corrupted documents from your index!

### Nested Class Summary

Nested Classes
`static class` `CheckIndex.Options` Run-time configuration options for CheckIndex commands.
`static class` `CheckIndex.Status` Returned from `checkIndex()` detailing the health and status of the index.
`static class` `CheckIndex.VerifyPointsVisitor` Walks the entire N-dimensional points space, verifying that all points fall within the last cell's boundaries.

### Constructor Summary

Constructors
Constructor and Description
`CheckIndex(Directory dir)` Create a new CheckIndex on the directory.
`CheckIndex(Directory dir, Lock writeLock)` Expert: create a directory with the specified lock.

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods
`static boolean` `assertsOn()` Check whether asserts are enabled or not.
`CheckIndex.Status` `checkIndex()` Returns a `CheckIndex.Status` instance detailing the state of the index.
`CheckIndex.Status` `checkIndex(List<String> onlySegments)` Returns a `CheckIndex.Status` instance detailing the state of the index.
`void` `close()`
`int` `doCheck(CheckIndex.Options opts)` Actually perform the index check
`boolean` `doSlowChecks()` See `setDoSlowChecks(boolean)`.
`void` `exorciseIndex(CheckIndex.Status result)` Repairs the index using previously returned result from `checkIndex()`.
`boolean` `getChecksumsOnly()` See `getChecksumsOnly()`.
`boolean` `getFailFast()` See `setFailFast(boolean)`.
`static void` `main(String[] args)` Command-line interface to check and exorcise corrupt segments from an index.
`static CheckIndex.Options` `parseOptions(String[] args)` Parse command line args into fields
`void` `setChecksumsOnly(boolean v)` If true, only validate physical integrity for all files.
`void` `setDoSlowChecks(boolean v)` If true, additional slow checks are performed.
`void` `setFailFast(boolean v)` If true, just throw the original exception immediately when corruption is detected, rather than continuing to iterate to other segments looking for more corruption.
`void` `setInfoStream(PrintStream out)` Set infoStream where messages should go.
`void` `setInfoStream(PrintStream out, boolean verbose)` Set infoStream where messages should go.
`static CheckIndex.Status.DocValuesStatus` `testDocValues(CodecReader reader, PrintStream infoStream, boolean failFast)` Test docvalues.
`static CheckIndex.Status.FieldInfoStatus` `testFieldInfos(CodecReader reader, PrintStream infoStream, boolean failFast)` Test field infos.
`static CheckIndex.Status.FieldNormStatus` `testFieldNorms(CodecReader reader, PrintStream infoStream, boolean failFast)` Test field norms.
`static CheckIndex.Status.LiveDocStatus` `testLiveDocs(CodecReader reader, PrintStream infoStream, boolean failFast)` Test live docs.
`static CheckIndex.Status.PointsStatus` `testPoints(CodecReader reader, PrintStream infoStream, boolean failFast)` Test the points index
`static CheckIndex.Status.TermIndexStatus` `testPostings(CodecReader reader, PrintStream infoStream)` Test the term index.
`static CheckIndex.Status.TermIndexStatus` `testPostings(CodecReader reader, PrintStream infoStream, boolean verbose, boolean doSlowChecks, boolean failFast)` Test the term index.
`static CheckIndex.Status.IndexSortStatus` `testSort(CodecReader reader, Sort sort, PrintStream infoStream, boolean failFast)` Tests index sort order.
`static CheckIndex.Status.StoredFieldStatus` `testStoredFields(CodecReader reader, PrintStream infoStream, boolean failFast)` Test stored fields.
`static CheckIndex.Status.TermVectorStatus` `testTermVectors(CodecReader reader, PrintStream infoStream)` Test term vectors.
`static CheckIndex.Status.TermVectorStatus` `testTermVectors(CodecReader reader, PrintStream infoStream, boolean verbose, boolean doSlowChecks, boolean failFast)` Test term vectors.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### CheckIndex

    public CheckIndex(Directory dir)
               throws IOException

Create a new CheckIndex on the directory.

Throws:
  `IOException`

#### CheckIndex

    public CheckIndex(Directory dir,
                      Lock writeLock)

Expert: create a directory with the specified lock. This should really not be used except for unit tests!!!! It exists only to support special tests (such as TestIndexWriterExceptions*), that would otherwise be more complicated to debug if they had to close the writer for each check.

### Method Detail

#### close

    public void close()
               throws IOException

Specified by:
  `close` in interface `Closeable`
Specified by:
  `close` in interface `AutoCloseable`
Throws:
  `IOException`

#### setDoSlowChecks

    public void setDoSlowChecks(boolean v)

If true, additional slow checks are performed. This will likely drastically increase time it takes to run CheckIndex!

#### doSlowChecks

    public boolean doSlowChecks()

See `setDoSlowChecks(boolean)`.

#### setFailFast

    public void setFailFast(boolean v)

If true, just throw the original exception immediately when corruption is detected, rather than continuing to iterate to other segments looking for more corruption.

#### getFailFast

    public boolean getFailFast()

See `setFailFast(boolean)`.

#### getChecksumsOnly

    public boolean getChecksumsOnly()

See `getChecksumsOnly()`.

#### setChecksumsOnly

    public void setChecksumsOnly(boolean v)

If true, only validate physical integrity for all files. Note that the returned nested status objects (e.g. storedFieldStatus) will be null.

#### setInfoStream

    public void setInfoStream(PrintStream out,
                              boolean verbose)

Set infoStream where messages should go. If null, no messages are printed. If verbose is true then more details are printed.

#### setInfoStream

    public void setInfoStream(PrintStream out)

Set infoStream where messages should go. See `setInfoStream(PrintStream,boolean)`.

#### checkIndex

    public CheckIndex.Status checkIndex()
                                 throws IOException

Returns a `CheckIndex.Status` instance detailing the state of the index.

As this method checks every byte in the index, on a large index it can take quite a long time to run.

WARNING: make sure you only call this when the index is not opened by any writer.

Throws:
  `IOException`

#### checkIndex

    public CheckIndex.Status checkIndex(List<String> onlySegments)
                                 throws IOException

Returns a `CheckIndex.Status` instance detailing the state of the index.

Parameters:
  `onlySegments` - list of specific segment names to check

  As this method checks every byte in the specified segments, on a large index it can take quite a long time to run.

Throws:
  `IOException`

#### testSort

    public static CheckIndex.Status.IndexSortStatus testSort(CodecReader reader,
                                                             Sort sort,
                                                             PrintStream infoStream,
                                                             boolean failFast)
                                                      throws IOException

Tests index sort order.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testLiveDocs

    public static CheckIndex.Status.LiveDocStatus testLiveDocs(CodecReader reader,
                                                               PrintStream infoStream,
                                                               boolean failFast)
                                                        throws IOException

Test live docs.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testFieldInfos

    public static CheckIndex.Status.FieldInfoStatus testFieldInfos(CodecReader reader,
                                                                   PrintStream infoStream,
                                                                   boolean failFast)
                                                            throws IOException

Test field infos.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testFieldNorms

    public static CheckIndex.Status.FieldNormStatus testFieldNorms(CodecReader reader,
                                                                   PrintStream infoStream,
                                                                   boolean failFast)
                                                            throws IOException

Test field norms.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testPostings

    public static CheckIndex.Status.TermIndexStatus testPostings(CodecReader reader,
                                                                 PrintStream infoStream)
                                                          throws IOException

Test the term index.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testPostings

    public static CheckIndex.Status.TermIndexStatus testPostings(CodecReader reader,
                                                                 PrintStream infoStream,
                                                                 boolean verbose,
                                                                 boolean doSlowChecks,
                                                                 boolean failFast)
                                                          throws IOException

Test the term index.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testPoints

    public static CheckIndex.Status.PointsStatus testPoints(CodecReader reader,
                                                            PrintStream infoStream,
                                                            boolean failFast)
                                                     throws IOException

Test the points index

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testStoredFields

    public static CheckIndex.Status.StoredFieldStatus testStoredFields(CodecReader reader,
                                                                       PrintStream infoStream,
                                                                       boolean failFast)
                                                                throws IOException

Test stored fields.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testDocValues

    public static CheckIndex.Status.DocValuesStatus testDocValues(CodecReader reader,
                                                                  PrintStream infoStream,
                                                                  boolean failFast)
                                                           throws IOException

Test docvalues.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testTermVectors

    public static CheckIndex.Status.TermVectorStatus testTermVectors(CodecReader reader,
                                                                     PrintStream infoStream)
                                                              throws IOException

Test term vectors.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### testTermVectors

    public static CheckIndex.Status.TermVectorStatus testTermVectors(CodecReader reader,
                                                                     PrintStream infoStream,
                                                                     boolean verbose,
                                                                     boolean doSlowChecks,
                                                                     boolean failFast)
                                                              throws IOException

Test term vectors.

Throws:
  `IOException`
WARNING: This API is experimental and might change in incompatible ways in the next release.

#### exorciseIndex

    public void exorciseIndex(CheckIndex.Status result)
                       throws IOException

Repairs the index using previously returned result from `checkIndex()`. Note that this does not remove any of the unreferenced files after it's done; you must separately open an `IndexWriter`, which deletes unreferenced files when it's created.

WARNING: this writes a new segments file into the index, effectively removing all documents in broken segments from the index. BE CAREFUL.

Throws:
  `IOException`

#### assertsOn

    public static boolean assertsOn()

Check whether asserts are enabled or not.

Returns:
  true iff asserts are enabled

#### main

    public static void main(String[] args)
                     throws IOException,
                            InterruptedException

Command-line interface to check and exorcise corrupt segments from an index.

Run it like this:

        java -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex pathToIndex [-exorcise] [-verbose] [-segment X] [-segment Y]

- `-exorcise`: actually write a new segments_N file, removing any problematic segments. *LOSES DATA*
- `-segment X`: only check the specified segment(s). This can be specified multiple times, to check more than one segment, eg `-segment _2 -segment _a`. You can't use this with the -exorcise option.

WARNING: `-exorcise` should only be used on an emergency basis as it will cause documents (perhaps many) to be permanently removed from the index. Always make a backup copy of your index before running this! Do not run this tool on an index that is actively being written to. You have been warned!

Run without -exorcise, this tool will open the index, report version information and report any exceptions it hits and what action it would take if -exorcise were specified. With -exorcise, this tool will remove any segments that have issues and write a new segments_N file. This means all documents contained in the affected segments will be removed.

This tool exits with exit code 1 if the index cannot be opened or has any corruption, else 0.

Throws:
  `IOException`
  `InterruptedException`

#### parseOptions

    public static CheckIndex.Options parseOptions(String[] args)

Parse command line args into fields

Parameters:
  `args` - The command line arguments
Returns:
  An Options struct
Throws:
  `IllegalArgumentException` - if any of the CLI args are invalid

#### doCheck

    public int doCheck(CheckIndex.Options opts)
                throws IOException,
                       InterruptedException

Actually perform the index check

Parameters:
  `opts` - The options to use for this check
Returns:
  0 iff the index is clean, 1 otherwise
Throws:
  `IOException`
  `InterruptedException`