org.apache.nutch.urlfilter.domain

## Class DomainURLFilter

- java.lang.Object
- - org.apache.nutch.urlfilter.domain.DomainURLFilter

- All Implemented Interfaces:
    Configurable, URLFilter, Pluggable

      public class DomainURLFilter
      extends java.lang.Object
      implements URLFilter

  Filters URLs based on a file containing domain suffixes, domain names, and hostnames. Only a url that matches one of the suffixes, domains, or hosts present in the file is allowed.

  Urls are checked in order of domain suffix, domain name, and hostname against entries in the domain file. The domain file would be setup as follows with one entry per line:

       com apache.org www.apache.org

  The first line is an example of a filter that would allow all .com domains. The second line allows all urls from apache.org and all of its subdomains such as lucene.apache.org and hadoop.apache.org. The third line would allow only urls from www.apache.org. There is no specific ordering to entries. The entries are from more general to more specific with the more general overridding the more specific.

  The domain file defaults to domain-urlfilter.txt in the classpath but can be overridden using the:
  - property "urlfilter.domain.file" in ./conf/nutch-*.xml, and
  - attribute "file" in plugin.xml of this plugin
  the attribute "file" has higher precedence if defined.

- - ### Field Summary

    - ### Fields inherited from interface org.apache.nutch.net.URLFilter

      `X_POINT_ID`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `DomainURLFilter()` Default constructor.
    `DomainURLFilter(java.lang.String domainFile)` Constructor that specifies the domain file to use.

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `java.lang.String` | `filter(java.lang.String url)`
    `Configuration` | `getConf()`
    `void` | `setConf(Configuration conf)` Sets the configuration.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### DomainURLFilter

          public DomainURLFilter()

      Default constructor.

    - #### DomainURLFilter

          public DomainURLFilter(java.lang.String domainFile)

      Constructor that specifies the domain file to use.
      Parameters:
        `domainFile` - The domain file, overrides domain-urlfilter.text default.

  - ### Method Detail

    - #### setConf

          public void setConf(Configuration conf)

      Sets the configuration.
      Specified by:
        `setConf` in interface `Configurable`

    - #### getConf

          public Configuration getConf()

      Specified by:
        `getConf` in interface `Configurable`

    - #### filter

          public java.lang.String filter(java.lang.String url)

      Specified by:
        `filter` in interface `URLFilter`

Copyright © 2019 The Apache Software Foundation