org.apache.nutch.indexer

Class CleaningJob

java.lang.Object

org.apache.nutch.indexer.CleaningJob

All Implemented Interfaces:

Configurable, Tool

public class CleaningJob
extends java.lang.Object
implements Tool

The class scans CrawlDB looking for entries with status DB_GONE (404) or DB_DUPLICATE and sends delete requests to indexers for those documents.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CleaningJob.DBFilter
static class	CleaningJob.DeleterReducer

Constructor Summary

Constructors

Constructor and Description
CleaningJob()

Method Summary

Modifier and Type	Method and Description
void	delete(java.lang.String crawldb, boolean noCommit)
Configuration	getConf()
static void	main(java.lang.String[] args)
int	run(java.lang.String[] args)
void	setConf(Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CleaningJob

public CleaningJob()

Method Detail

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

delete

public void delete(java.lang.String crawldb,
                   boolean noCommit)
            throws java.io.IOException,
                   java.lang.InterruptedException,
                   java.lang.ClassNotFoundException

Throws:

java.io.IOException

java.lang.InterruptedException

java.lang.ClassNotFoundException

run

public int run(java.lang.String[] args)
        throws java.io.IOException

Specified by:

run in interface Tool

Throws:

java.io.IOException

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception