org.apache.nutch.protocol

## Interface Protocol

- All Superinterfaces:
    Configurable, FieldPluggable, Pluggable
  All Known Implementing Classes:
    File, Ftp, Http, Http, HttpBase, Sftp

      public interface Protocol
      extends FieldPluggable, Configurable

  A retriever of url content. Implemented by protocol extensions.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static java.lang.String` | `CHECK_BLOCKING` Property name.
    `static java.lang.String` | `CHECK_ROBOTS` Property name.
    `static java.lang.String` | `X_POINT_ID` The name of the extension point.

  - ### Method Summary

    All Methods Instance Methods Abstract Methods
    Modifier and Type | Method and Description
    --- | ---
    `ProtocolOutput` | `getProtocolOutput(java.lang.String url, WebPage page)`
    `crawlercommons.robots.BaseRobotRules` | `getRobotRules(java.lang.String url, WebPage page)` Retrieve robot rules applicable for this url.

    - ### Methods inherited from interface org.apache.nutch.plugin.FieldPluggable

      `getFields`

    - ### Methods inherited from interface org.apache.hadoop.conf.Configurable

      `getConf, setConf`

- - ### Field Detail

    - #### X_POINT_ID

          static final java.lang.String X_POINT_ID

      The name of the extension point.

    - #### CHECK_BLOCKING

          static final java.lang.String CHECK_BLOCKING

      Property name. If in the current configuration this property is set to true, protocol implementations should handle "politeness" limits internally. If this is set to false, it is assumed that these limits are enforced elsewhere, and protocol implementations should not enforce them internally.
      See Also:
        Constant Field Values

    - #### CHECK_ROBOTS

          static final java.lang.String CHECK_ROBOTS

      Property name. If in the current configuration this property is set to true, protocol implementations should handle robot exclusion rules internally. If this is set to false, it is assumed that these limits are enforced elsewhere, and protocol implementations should not enforce them internally.
      See Also:
        Constant Field Values

  - ### Method Detail

    - #### getProtocolOutput

          ProtocolOutput getProtocolOutput(java.lang.String url,
                                           WebPage page)

    - #### getRobotRules

          crawlercommons.robots.BaseRobotRules getRobotRules(java.lang.String url,
                                                             WebPage page)

      Retrieve robot rules applicable for this url.
      Parameters:
        `url` - url to check
        `page` -
      Returns:
        robot rules (specific for this url or default), never null

Copyright © 2019 The Apache Software Foundation