# apache-nutch 2.3.1 API

Apache Nutch 2.X is a branch of the Apache Nutch open source web-search software project.

See: Description

Core
Package | Description
--- | ---
org.apache.nutch.api | REST API to run and control crawl jobs.
org.apache.nutch.api.impl | Implementations of REST API interfaces.
org.apache.nutch.api.impl.db |
org.apache.nutch.api.misc |
org.apache.nutch.api.model.request |
org.apache.nutch.api.model.response |
org.apache.nutch.api.resources |
org.apache.nutch.crawl | Crawl control code and tools to run the crawler.
org.apache.nutch.fetcher | The Nutch robot.
org.apache.nutch.host | Host database to store metadata per host.
org.apache.nutch.indexer | Index content, configure and run indexing and cleaning jobs to add, update, and delete documents from an index.
org.apache.nutch.indexer.solr |
org.apache.nutch.metadata | A Multi-valued Metadata container, and set of constant fields for Nutch Metadata.
org.apache.nutch.net | Web-related interfaces: URL `filters` and `normalizers`.
org.apache.nutch.net.protocols | Helper classes related to the `Protocol` interface, sea also `org.apache.nutch.protocol`.
org.apache.nutch.parse | The `Parse` interface and related classes.
org.apache.nutch.plugin | The Nutch `Plugin` System.
org.apache.nutch.protocol | Classes related to the `Protocol` interface, see also `org.apache.nutch.net.protocols`.
org.apache.nutch.scoring | The `ScoringFilter` interface.
org.apache.nutch.storage | Representation (`web pages`, `host metadata`) of data in abstracted storage.
org.apache.nutch.tools | Miscellaneous tools.
org.apache.nutch.tools.arc | Tools to read the Arc file format.
org.apache.nutch.tools.proxy | Proxy to `benchmark` the crawler.
org.apache.nutch.util | Miscellaneous utility classes.
org.apache.nutch.util.domain | Classes for domain name analysis.
org.apache.nutch.webui |
org.apache.nutch.webui.client |
org.apache.nutch.webui.client.impl |
org.apache.nutch.webui.client.model |
org.apache.nutch.webui.config |
org.apache.nutch.webui.model |
org.apache.nutch.webui.pages |
org.apache.nutch.webui.pages.assets |
org.apache.nutch.webui.pages.components |
org.apache.nutch.webui.pages.crawls |
org.apache.nutch.webui.pages.instances |
org.apache.nutch.webui.pages.menu |
org.apache.nutch.webui.pages.seed |
org.apache.nutch.webui.pages.settings |
org.apache.nutch.webui.service |
org.apache.nutch.webui.service.impl |

Plugins API
Package | Description
--- | ---
org.apache.nutch.protocol.http.api | Common API used by HTTP plugins (`http`, `httpclient`)
org.apache.nutch.urlfilter.api | Generic `URL filter` library, abstracting away from regular expression implementations.

Protocol Plugins
Package | Description
--- | ---
org.apache.nutch.protocol.file | Protocol plugin which supports retrieving local file resources.
org.apache.nutch.protocol.ftp | Protocol plugin which supports retrieving documents via the ftp protocol.
org.apache.nutch.protocol.http | Protocol plugin which supports retrieving documents via the http protocol.
org.apache.nutch.protocol.httpclient | Protocol plugin which supports retrieving documents via the HTTP and HTTPS protocols, optionally with Basic, Digest and NTLM authentication schemes for web server as well as proxy server.
org.apache.nutch.protocol.sftp | Protocol plugin which supports retrieving documents via the sftp protocol.

URL Filter Plugins
Package | Description
--- | ---
org.apache.nutch.urlfilter.automaton | URL filter plugin based on dk.brics.automaton Finite-State Automata for JavaTM.
org.apache.nutch.urlfilter.domain | URL filter plugin to include only URLs which match an element in a given list of domain suffixes, domain names, and/or host names.
org.apache.nutch.urlfilter.prefix | URL filter plugin to include only URLs which match one of a given list of URL prefixes.
org.apache.nutch.urlfilter.regex | URL filter plugin to include and/or exclude URLs matching Java regular expressions.
org.apache.nutch.urlfilter.suffix | URL filter plugin to either exclude or include only URLs which match one of the given (path) suffixes.
org.apache.nutch.urlfilter.validator | URL filter plugin that validates given urls.

URL Normalizer Plugins
Package | Description
--- | ---
org.apache.nutch.net.urlnormalizer.basic | URL normalizer performing basic normalizations: remove default ports and dot segments in path.
org.apache.nutch.net.urlnormalizer.pass | URL normalizer dummy which does not change URLs.
org.apache.nutch.net.urlnormalizer.regex | URL normalizer with configurable rules based on regular expressions (`Pattern`).

Scoring Plugins
Package | Description
--- | ---
org.apache.nutch.scoring.link | Scoring filter used in conjunction with `org.apache.nutch.scoring.webgraph.WebGraph`.
org.apache.nutch.scoring.opic | Scoring filter implementing a variant of the Online Page Importance Computation (OPIC) algorithm.
org.apache.nutch.scoring.tld | Top Level Domain Scoring plugin.

Parse Plugins
Package | Description
--- | ---
org.apache.nutch.parse.html | An HTML document parsing plugin.
org.apache.nutch.parse.js | Parser and parse filter plugin to extract all (possible) links from JavaScript files and embedded JavaScript code snippets.
org.apache.nutch.parse.tika | Parse various document formats with help of Apache Tika.

Parse Filter Plugins
Package | Description
--- | ---
org.apache.nutch.parse.metatags | Parse filter to extract meta tags: keywords, description, etc.

Indexing Filter Plugins
Package | Description
--- | ---
org.apache.nutch.indexer.anchor | An indexing plugin for inbound anchor text.
org.apache.nutch.indexer.basic | A basic indexing plugin, adds basic fields: url, host, title, content, etc.
org.apache.nutch.indexer.html | Index raw HTML content.
org.apache.nutch.indexer.metadata | Indexing filter to add document metadata to the index.
org.apache.nutch.indexer.more | A more indexing plugin, adds "more" index fields: last modified date, MIME type, content length.
org.apache.nutch.indexer.subcollection | Indexing filter to assign documents to subcollections.
org.apache.nutch.indexer.tld | Top Level Domain Indexing plugin.

Indexer Plugins
Package | Description
--- | ---
org.apache.nutch.indexwriter.elastic | Index writer plugin for Elasticsearch.
org.apache.nutch.indexwriter.solr | Index writer plugin for Apache Solr.

Misc. Plugins
Package | Description
--- | ---
org.apache.nutch.analysis.lang | Text document language identifier.
org.apache.nutch.collection | Subcollection is a subset of an index.
org.apache.nutch.microformats.reltag | A microformats Rel-Tag Parser/Indexer/Querier plugin.
org.creativecommons.nutch | Sample plugins that parse and index Creative Commons medadata.

Copyright © 2015 The Apache Software Foundation