Package org.apache.lucene.benchmark.byTask.feeds

# Class TrecContentSource

java.lang.Object
org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource
org.apache.lucene.benchmark.byTask.feeds.ContentSource
org.apache.lucene.benchmark.byTask.feeds.TrecContentSource

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public class TrecContentSource extends ContentSource
Implements a `ContentSource` over the TREC collection.

Supports the following configuration parameters (on top of `ContentSource`):

- work.dir - specifies the working directory. Required if "docs.dir" denotes a relative path (default=work).
- docs.dir - specifies the directory where the TREC files reside. Can be set to a relative path if "work.dir" is also specified (default=trec).
- trec.doc.parser - specifies the `TrecDocParser` class to use for parsing the TREC documents content (default=TrecGov2Parser).
- html.parser - specifies the `HTMLParser` class to use for parsing the HTML parts of the TREC documents content (default=DemoHTMLParser).
- content.source.encoding - if not specified, ISO-8859-1 is used.
- content.source.excludeIteration - if true, do not append iteration number to docname

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final String`
  `DOC`
  `static final String`
  `DOCNO`
  `static final String`
  `NEW_LINE`
  separator between lines in the byffer
  `static final String`
  `TERMINATING_DOC`
  `static final String`
  `TERMINATING_DOCNO`

  ### Fields inherited from class org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource

  `encoding, forever, logStep, verbose`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `TrecContentSource()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `close()`
  Called when reading from this content source is no longer required.
  `DocData`
  `getNextDocData(DocData docData)`
  Returns the next `DocData` from the content source.
  `Date`
  `parseDate(String dateStr)`
  `void`
  `resetInputs()`
  Resets the input for this content source, so that the test would behave as if it was just started, input-wise.
  `void`
  `setConfig(Config config)`
  Sets the `Config` for this content source.

  ### Methods inherited from class org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource

  `addBytes, addItem, collectFiles, getBytesCount, getConfig, getItemsCount, getTotalBytesCount, getTotalItemsCount, printStatistics, shouldLog`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### DOCNO

    public static final String DOCNO
    See Also:
      - Constant Field Values

  - ### TERMINATING_DOCNO

    public static final String TERMINATING_DOCNO
    See Also:
      - Constant Field Values

  - ### DOC

    public static final String DOC
    See Also:
      - Constant Field Values

  - ### TERMINATING_DOC

    public static final String TERMINATING_DOC
    See Also:
      - Constant Field Values

  - ### NEW_LINE

    public static final String NEW_LINE
    separator between lines in the byffer

- ## Constructor Details

  - ### TrecContentSource

    public TrecContentSource()

- ## Method Details

  - ### parseDate

    public Date parseDate(String dateStr)

  - ### close

    public void close() throws IOException
    Description copied from class: `ContentItemsSource`
    Called when reading from this content source is no longer required.
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `Closeable`
    Specified by:
      `close` in class `ContentItemsSource`
    Throws:
      `IOException`

  - ### getNextDocData

    public DocData getNextDocData(DocData docData) throws NoMoreDataException, IOException
    Description copied from class: `ContentSource`
    Returns the next `DocData` from the content source. Implementations must account for multi-threading, as multiple threads can call this method simultaneously.
    Specified by:
      `getNextDocData` in class `ContentSource`
    Throws:
      `NoMoreDataException`
      `IOException`

  - ### resetInputs

    public void resetInputs() throws IOException
    Description copied from class: `ContentItemsSource`
    Resets the input for this content source, so that the test would behave as if it was just started, input-wise.

    NOTE: the default implementation resets the number of bytes and items generated since the last reset, so it's important to call super.resetInputs in case you override this method.

    Overrides:
      `resetInputs` in class `ContentItemsSource`
    Throws:
      `IOException`

  - ### setConfig

    public void setConfig(Config config)
    Description copied from class: `ContentItemsSource`
    Sets the `Config` for this content source. If you override this method, you must call super.setConfig.
    Overrides:
      `setConfig` in class `ContentItemsSource`