org.apache.nutch.crawl

Class LinkDbFilter

java.lang.Object

org.apache.hadoop.mapreduce.Mapper<Text,Inlinks,Text,Inlinks>

org.apache.nutch.crawl.LinkDbFilter

public class LinkDbFilter
extends Mapper<Text,Inlinks,Text,Inlinks>

This class provides a way to separate the URL normalization and filtering steps from the rest of LinkDb manipulation code.

Author:

Andrzej Bialecki

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Mapper

Mapper.Context

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	URL_FILTERING
static java.lang.String	URL_NORMALIZING
static java.lang.String	URL_NORMALIZING_SCOPE

Constructor Summary

Constructors

Constructor and Description
LinkDbFilter()

Method Summary

Modifier and Type	Method and Description
void	close()
void	map(Text key, Inlinks value, Mapper.Context context)
void	setup(Mapper.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Mapper

cleanup, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

URL_FILTERING

public static final java.lang.String URL_FILTERING

See Also:

Constant Field Values

URL_NORMALIZING

public static final java.lang.String URL_NORMALIZING

See Also:

Constant Field Values

URL_NORMALIZING_SCOPE

public static final java.lang.String URL_NORMALIZING_SCOPE

See Also:

Constant Field Values

Constructor Detail

LinkDbFilter

public LinkDbFilter()

Method Detail

setup

public void setup(Mapper.Context context)

Overrides:

setup in class Mapper<Text,Inlinks,Text,Inlinks>

close

public void close()

map

public void map(Text key,
                Inlinks value,
                Mapper.Context context)
         throws java.io.IOException,
                java.lang.InterruptedException

Overrides:

map in class Mapper<Text,Inlinks,Text,Inlinks>

Throws:

java.io.IOException

java.lang.InterruptedException