org.apache.nutch.indexwriter.csv

Class CSVIndexWriter

java.lang.Object

org.apache.nutch.indexwriter.csv.CSVIndexWriter

All Implemented Interfaces:

Configurable, IndexWriter, Pluggable

public class CSVIndexWriter
extends java.lang.Object
implements IndexWriter

Write Nutch documents to a CSV file (comma separated values), i.e., dump index as CSV or tab-separated plain text table. Format (encoding, separators, etc.) is configurable by a couple of options, see output of describe().

Note: works only in local mode, to be used with index option -noCommit.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	CSVIndexWriter.Separator represent separators (also quote and escape characters) as char(s) and byte(s) in the output encoding for efficiency.

Field Summary

Fields

Modifier and Type	Field and Description
protected FSDataOutputStream	csvout
protected java.nio.charset.Charset	encoding encoding of CSV file
static org.slf4j.Logger	LOG

Fields inherited from interface org.apache.nutch.indexer.IndexWriter

X_POINT_ID

Constructor Summary

Constructors

Constructor and Description
CSVIndexWriter()

Method Summary

Modifier and Type	Method and Description
void	close()
void	commit() (nothing to commit)
void	delete(java.lang.String key) (deletion of documents is not supported)
java.util.Map<java.lang.String,java.util.Map.Entry<java.lang.String,java.lang.Object>>	describe() Returns Map with the specific parameters the IndexWriter instance can take.
Configuration	getConf()
static void	main(java.lang.String[] args)
void	open(Configuration conf, java.lang.String name)
void	open(IndexWriterParams parameters) Initializes the internal variables from a given index writer configuration.
void	setConf(Configuration conf)
void	update(NutchDocument doc)
void	write(NutchDocument doc)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

encoding

protected java.nio.charset.Charset encoding

encoding of CSV file

csvout

protected FSDataOutputStream csvout

Constructor Detail

CSVIndexWriter

public CSVIndexWriter()

Method Detail

open

public void open(Configuration conf,
                 java.lang.String name)
          throws java.io.IOException

Specified by:

open in interface IndexWriter

Throws:

java.io.IOException

open

public void open(IndexWriterParams parameters)
          throws java.io.IOException

Initializes the internal variables from a given index writer configuration.

Specified by:

open in interface IndexWriter

Parameters:

parameters - Params from the index writer configuration.

Throws:

java.io.IOException - Some exception thrown by writer.

write

public void write(NutchDocument doc)
           throws java.io.IOException

Specified by:

write in interface IndexWriter

Throws:

java.io.IOException

delete

public void delete(java.lang.String key)

(deletion of documents is not supported)

Specified by:

delete in interface IndexWriter

update

public void update(NutchDocument doc)
            throws java.io.IOException

Specified by:

update in interface IndexWriter

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Specified by:

close in interface IndexWriter

Throws:

java.io.IOException

commit

public void commit()

(nothing to commit)

Specified by:

commit in interface IndexWriter

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable

describe

public java.util.Map<java.lang.String,java.util.Map.Entry<java.lang.String,java.lang.Object>> describe()

Returns Map with the specific parameters the IndexWriter instance can take.

Specified by:

describe in interface IndexWriter

Returns:

The values of each row. It must have the form >.

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception