org.apache.nutch.parse

## Class ParserChecker

- java.lang.Object
- - org.apache.nutch.parse.ParserChecker

- All Implemented Interfaces:
    Configurable, Tool

      public class ParserChecker
      extends Object
      implements Tool

  Parser checker, useful for testing parser. It also accurately reports possible fetching and parsing failures and presents protocol status signals to aid debugging. The tool enables us to retrieve the following data from any url:
  1. `contentType`: The URL `Content` type.
  2. `signature`: Digest is used to identify pages (like unique ID) and is used to remove duplicates during the dedup procedure. It is calculated using `MD5Signature` or `TextProfileSignature`.
  3. `Version`: From `org.apache.nutch.parse.ParseData`.
  4. `Status`: From `org.apache.nutch.parse.ParseData`.
  5. `Title`: of the URL
  6. `Outlinks`: associated with the URL
  7. `Content Metadata`: such as X-AspNet-Version, Date, Content-length, servedBy, Content-Type, Cache-Control, etc.
  8. `Parse Metadata`: such as CharEncodingForConversion, OriginalCharEncoding, language, etc.
  9. `ParseText`: The page parse text which varies in length depdnecing on `content.length` configuration.
  Author:
    John Xing

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static org.slf4j.Logger` | `LOG`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `ParserChecker()`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `Configuration` | `getConf()`
    `static void` | `main(String[] args)`
    `int` | `run(String[] args)`
    `void` | `setConf(Configuration c)`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### LOG

          public static final org.slf4j.Logger LOG

  - ### Constructor Detail

    - #### ParserChecker

          public ParserChecker()

  - ### Method Detail

    - #### run

          public int run(String[] args)
                  throws Exception

      Specified by:
        `run` in interface `Tool`
      Throws:
        `Exception`

    - #### getConf

          public Configuration getConf()

      Specified by:
        `getConf` in interface `Configurable`

    - #### setConf

          public void setConf(Configuration c)

      Specified by:
        `setConf` in interface `Configurable`

    - #### main

          public static void main(String[] args)
                           throws Exception

      Throws:
        `Exception`

Copyright © 2015 The Apache Software Foundation