org.apache.nutch.scoring.webgraph

Class NodeDumper.Sorter

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.nutch.scoring.webgraph.NodeDumper.Sorter

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, Configurable, JobConfigurable, Mapper<Text,Node,FloatWritable,Text>, Reducer<FloatWritable,Text,Text,FloatWritable>

Enclosing class:

NodeDumper

public static class NodeDumper.Sorter
extends Configured
implements Mapper<Text,Node,FloatWritable,Text>, Reducer<FloatWritable,Text,Text,FloatWritable>

Outputs the top urls sorted in descending order. Depending on the flag set on the command line, the top urls could be for number of inlinks, for number of outlinks, or for link analysis score.

Constructor Summary

Constructors

Constructor and Description
Sorter()

Method Summary

Modifier and Type	Method and Description
void	close()
void	configure(JobConf conf) Configures the job, sets the flag for type of content and the topN number if any.
void	map(Text key, Node node, OutputCollector<FloatWritable,Text> output, Reporter reporter) Outputs the url with the appropriate number of inlinks, outlinks, or for score.
void	reduce(FloatWritable key, java.util.Iterator<Text> values, OutputCollector<Text,FloatWritable> output, Reporter reporter) Flips and collects the url and numeric sort value.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Sorter

public Sorter()

Method Detail

configure

public void configure(JobConf conf)

Configures the job, sets the flag for type of content and the topN number if any.

Specified by:

configure in interface JobConfigurable

close

public void close()

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

map

public void map(Text key,
                Node node,
                OutputCollector<FloatWritable,Text> output,
                Reporter reporter)
         throws java.io.IOException

Outputs the url with the appropriate number of inlinks, outlinks, or for score.

Specified by:

map in interface Mapper<Text,Node,FloatWritable,Text>

Throws:

java.io.IOException

reduce

public void reduce(FloatWritable key,
                   java.util.Iterator<Text> values,
                   OutputCollector<Text,FloatWritable> output,
                   Reporter reporter)
            throws java.io.IOException

Flips and collects the url and numeric sort value.

Specified by:

reduce in interface Reducer<FloatWritable,Text,Text,FloatWritable>

Throws:

java.io.IOException