Package org.apache.storm.hdfs.bolt

# Class AbstractHdfsBolt

java.lang.Object
org.apache.storm.topology.base.BaseComponent
org.apache.storm.topology.base.BaseRichBolt
org.apache.storm.hdfs.bolt.AbstractHdfsBolt

All Implemented Interfaces:
  `Serializable`, `IBolt`, `IComponent`, `IRichBolt`

Direct Known Subclasses:
  `AvroGenericRecordBolt`, `HdfsBolt`, `SequenceFileBolt`

public abstract class AbstractHdfsBolt extends BaseRichBolt

See Also:
  - Serialized Form

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `protected OutputCollector`
  `collector`
  `protected String`
  `configKey`
  `protected FileNameFormat`
  `fileNameFormat`
  `protected Integer`
  `fileRetryCount`
  `protected org.apache.hadoop.fs.FileSystem`
  `fs`
  `protected String`
  `fsUrl`
  `protected org.apache.hadoop.conf.Configuration`
  `hdfsConfig`
  `protected Integer`
  `maxOpenFiles`
  `protected long`
  `offset`
  `protected Partitioner`
  `partitioner`
  `protected List<RotationAction>`
  `rotationActions`
  `protected Map<String,Integer>`
  `rotationCounterMap`
  `protected FileRotationPolicy`
  `rotationPolicy`
  `protected Timer`
  `rotationTimer`
  `protected SyncPolicy`
  `syncPolicy`
  `protected Integer`
  `tickTupleInterval`
  `protected Object`
  `writeLock`
  `protected Map<String,Writer>`
  `writers`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `AbstractHdfsBolt()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `cleanup()`
  Called when an IBolt is going to be shutdown.
  `void`
  `declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer)`
  Declare the output schema for all the streams of this topology.
  `protected abstract void`
  `doPrepare(Map<String,Object> conf, TopologyContext topologyContext, OutputCollector collector)`
  `final void`
  `execute(Tuple tuple)`
  Process a single tuple of input.
  `protected org.apache.hadoop.fs.Path`
  `getBasePathForNextFile(Tuple tuple)`
  `Map<String,Object>`
  `getComponentConfiguration()`
  Declare configuration specific to this component.
  `protected abstract String`
  `getWriterKey(Tuple tuple)`
  `protected abstract Writer`
  `makeNewWriter(org.apache.hadoop.fs.Path path, Tuple tuple)`
  `final void`
  `prepare(Map<String,Object> conf, TopologyContext topologyContext, OutputCollector collector)`
  Marked as final to prevent override.
  `protected void`
  `rotateOutputFile(Writer writer)`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### writers

    protected Map<String,Writer> writers

  - ### rotationCounterMap

    protected Map<String,Integer> rotationCounterMap

  - ### rotationActions

    protected List<RotationAction> rotationActions

  - ### collector

    protected OutputCollector collector

  - ### fs

    protected transient org.apache.hadoop.fs.FileSystem fs

  - ### syncPolicy

    protected SyncPolicy syncPolicy

  - ### rotationPolicy

    protected FileRotationPolicy rotationPolicy

  - ### fileNameFormat

    protected FileNameFormat fileNameFormat

  - ### fsUrl

    protected String fsUrl

  - ### configKey

    protected String configKey

  - ### writeLock

    protected transient Object writeLock

  - ### rotationTimer

    protected transient Timer rotationTimer

  - ### offset

    protected long offset

  - ### fileRetryCount

    protected Integer fileRetryCount

  - ### tickTupleInterval

    protected Integer tickTupleInterval

  - ### maxOpenFiles

    protected Integer maxOpenFiles

  - ### partitioner

    protected Partitioner partitioner

  - ### hdfsConfig

    protected transient org.apache.hadoop.conf.Configuration hdfsConfig

- ## Constructor Details

  - ### AbstractHdfsBolt

    public AbstractHdfsBolt()

- ## Method Details

  - ### rotateOutputFile

    protected void rotateOutputFile(Writer writer) throws IOException
    Throws:
      `IOException`

  - ### prepare

    public final void prepare(Map<String,Object> conf, TopologyContext topologyContext, OutputCollector collector)
    Marked as final to prevent override. Subclasses should implement the doPrepare() method.
    Parameters:
      `conf` - The Storm configuration for this bolt. This is the configuration provided to the topology merged in with cluster configuration on this machine.
      `topologyContext` - This object can be used to get information about this task's place within the topology, including the task id and component id of this task, input and output information, etc.
      `collector` - The collector is used to emit tuples from this bolt. Tuples can be emitted at any time, including the prepare and cleanup methods. The collector is thread-safe and should be saved as an instance variable of this bolt object.

  - ### execute

    public final void execute(Tuple tuple)
    Description copied from interface: `IBolt`
    Process a single tuple of input. The Tuple object contains metadata on it about which component/stream/task it came from. The values of the Tuple can be accessed using Tuple#getValue. The IBolt does not have to process the Tuple immediately. It is perfectly fine to hang onto a tuple and process it later (for instance, to do an aggregation or join).

    Tuples should be emitted using the OutputCollector provided through the prepare method. It is required that all input tuples are acked or failed at some point using the OutputCollector. Otherwise, Storm will be unable to determine when tuples coming off the spouts have been completed.

    For the common case of acking an input tuple at the end of the execute method, see IBasicBolt which automates this.

    Parameters:
      `tuple` - The input tuple to be processed.

  - ### getComponentConfiguration

    public Map<String,Object> getComponentConfiguration()
    Description copied from interface: `IComponent`
    Declare configuration specific to this component. Only a subset of the "topology.*" configs can be overridden. The component configuration can be further overridden when constructing the topology using `TopologyBuilder`
    Specified by:
      `getComponentConfiguration` in interface `IComponent`
    Overrides:
      `getComponentConfiguration` in class `BaseComponent`

  - ### declareOutputFields

    public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer)
    Description copied from interface: `IComponent`
    Declare the output schema for all the streams of this topology.
    Parameters:
      `outputFieldsDeclarer` - this is used to declare output stream ids, output fields, and whether or not each output stream is a direct stream

  - ### cleanup

    public void cleanup()
    Description copied from interface: `IBolt`
    Called when an IBolt is going to be shutdown. Storm will make a best-effort attempt to call this if the worker shutdown is orderly. The `Config.SUPERVISOR_WORKER_SHUTDOWN_SLEEP_SECS` setting controls how long orderly shutdown is allowed to take. There is no guarantee that cleanup will be called if shutdown is not orderly, or if the shutdown exceeds the time limit.

    The one context where cleanup is guaranteed to be called is when a topology is killed when running Storm in local mode.

    Specified by:
      `cleanup` in interface `IBolt`
    Overrides:
      `cleanup` in class `BaseRichBolt`

  - ### getBasePathForNextFile

    protected org.apache.hadoop.fs.Path getBasePathForNextFile(Tuple tuple)

  - ### doPrepare

    protected abstract void doPrepare(Map<String,Object> conf, TopologyContext topologyContext, OutputCollector collector) throws IOException
    Throws:
      `IOException`

  - ### getWriterKey

    protected abstract String getWriterKey(Tuple tuple)

  - ### makeNewWriter

    protected abstract Writer makeNewWriter(org.apache.hadoop.fs.Path path, Tuple tuple) throws IOException
    Throws:
      `IOException`