org.apache.storm.hdfs.bolt

Class AvroGenericRecordBolt

java.lang.Object

org.apache.storm.topology.base.BaseComponent

org.apache.storm.topology.base.BaseRichBolt

org.apache.storm.hdfs.bolt.AbstractHdfsBolt

org.apache.storm.hdfs.bolt.AvroGenericRecordBolt

All Implemented Interfaces:

Serializable, IBolt, IComponent, IRichBolt

public class AvroGenericRecordBolt
extends AbstractHdfsBolt

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.storm.hdfs.bolt.AbstractHdfsBolt

collector, configKey, fileNameFormat, fileRetryCount, fs, fsUrl, hdfsConfig, offset, rotation, rotationActions, rotationPolicy, rotationTimer, syncPolicy, tickTupleInterval, writeLock

Constructor Summary

Constructors

Constructor and Description
AvroGenericRecordBolt()

Method Summary

Modifier and Type	Method and Description
AvroGenericRecordBolt	addRotationAction(RotationAction action)
protected void	closeOutputFile()
protected org.apache.hadoop.fs.Path	createOutputFile()
protected void	doPrepare(Map conf, TopologyContext topologyContext, OutputCollector collector)
protected void	syncTuples() Make the best effort to sync written data to the underlying file system.
AvroGenericRecordBolt	withConfigKey(String configKey)
AvroGenericRecordBolt	withFileNameFormat(FileNameFormat fileNameFormat)
AvroGenericRecordBolt	withFsUrl(String fsUrl)
AvroGenericRecordBolt	withRotationPolicy(FileRotationPolicy rotationPolicy)
AvroGenericRecordBolt	withSchemaAsString(String schemaAsString)
AvroGenericRecordBolt	withSyncPolicy(SyncPolicy syncPolicy)
AvroGenericRecordBolt	withTickTupleIntervalSeconds(int interval)
protected void	writeTuple(Tuple tuple) writes a tuple to the underlying filesystem but makes no guarantees about syncing data.

Methods inherited from class org.apache.storm.hdfs.bolt.AbstractHdfsBolt

cleanup, declareOutputFields, execute, getComponentConfiguration, prepare, rotateOutputFile

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroGenericRecordBolt

public AvroGenericRecordBolt()

Method Detail

withSchemaAsString

public AvroGenericRecordBolt withSchemaAsString(String schemaAsString)

withFsUrl

public AvroGenericRecordBolt withFsUrl(String fsUrl)

withConfigKey

public AvroGenericRecordBolt withConfigKey(String configKey)

withFileNameFormat

public AvroGenericRecordBolt withFileNameFormat(FileNameFormat fileNameFormat)

withSyncPolicy

public AvroGenericRecordBolt withSyncPolicy(SyncPolicy syncPolicy)

withRotationPolicy

public AvroGenericRecordBolt withRotationPolicy(FileRotationPolicy rotationPolicy)

addRotationAction

public AvroGenericRecordBolt addRotationAction(RotationAction action)

withTickTupleIntervalSeconds

public AvroGenericRecordBolt withTickTupleIntervalSeconds(int interval)

doPrepare

protected void doPrepare(Map conf,
                         TopologyContext topologyContext,
                         OutputCollector collector)
                  throws IOException

Specified by:

doPrepare in class AbstractHdfsBolt

Throws:

IOException

writeTuple

protected void writeTuple(Tuple tuple)
                   throws IOException

Description copied from class: AbstractHdfsBolt

writes a tuple to the underlying filesystem but makes no guarantees about syncing data.

this.offset is also updated to reflect additional data written

Specified by:

writeTuple in class AbstractHdfsBolt

Throws:

IOException

syncTuples

protected void syncTuples()
                   throws IOException

Description copied from class: AbstractHdfsBolt

Make the best effort to sync written data to the underlying file system. Concrete classes should very clearly state the file state that sync guarantees. For example, HdfsBolt can make a much stronger guarantee than SequenceFileBolt.

Specified by:

syncTuples in class AbstractHdfsBolt

Throws:

IOException

closeOutputFile

protected void closeOutputFile()
                        throws IOException

Specified by:

closeOutputFile in class AbstractHdfsBolt

Throws:

IOException

createOutputFile

protected org.apache.hadoop.fs.Path createOutputFile()
                                              throws IOException

Specified by:

createOutputFile in class AbstractHdfsBolt

Throws:

IOException