org.apache.nutch.crawl

Class LinkDbFilter

java.lang.Object

org.apache.nutch.crawl.LinkDbFilter

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, JobConfigurable, Mapper<Text,Inlinks,Text,Inlinks>

public class LinkDbFilter
extends java.lang.Object
implements Mapper<Text,Inlinks,Text,Inlinks>

This class provides a way to separate the URL normalization and filtering steps from the rest of LinkDb manipulation code.

Author:

Andrzej Bialecki

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	URL_FILTERING
static java.lang.String	URL_NORMALIZING
static java.lang.String	URL_NORMALIZING_SCOPE

Constructor Summary

Constructors

Constructor and Description
LinkDbFilter()

Method Summary

Modifier and Type	Method and Description
void	close()
void	configure(JobConf job)
void	map(Text key, Inlinks value, OutputCollector<Text,Inlinks> output, Reporter reporter)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

URL_FILTERING

public static final java.lang.String URL_FILTERING

See Also:

Constant Field Values

URL_NORMALIZING

public static final java.lang.String URL_NORMALIZING

See Also:

Constant Field Values

URL_NORMALIZING_SCOPE

public static final java.lang.String URL_NORMALIZING_SCOPE

See Also:

Constant Field Values

Constructor Detail

LinkDbFilter

public LinkDbFilter()

Method Detail

configure

public void configure(JobConf job)

Specified by:

configure in interface JobConfigurable

close

public void close()

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

map

public void map(Text key,
                Inlinks value,
                OutputCollector<Text,Inlinks> output,
                Reporter reporter)
         throws java.io.IOException

Specified by:

map in interface Mapper<Text,Inlinks,Text,Inlinks>

Throws:

java.io.IOException