org.apache.storm.kafka.bolt

Class KafkaBolt<K,V>

java.lang.Object

org.apache.storm.topology.base.BaseComponent

org.apache.storm.topology.base.BaseRichBolt

org.apache.storm.topology.base.BaseTickTupleAwareRichBolt

org.apache.storm.kafka.bolt.KafkaBolt<K,V>

All Implemented Interfaces:

Serializable, IBolt, IComponent, IRichBolt

public class KafkaBolt<K,V>
extends BaseTickTupleAwareRichBolt

Bolt implementation that can send Tuple data to Kafka.

Most configuration for this bolt should be through the various setter methods in the bolt. For backwards compatibility it supports the producer configuration and topic to be placed in the storm config under

‘kafka.broker.properties’ and ‘topic’

respectively.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static String	TOPIC

Constructor Summary

Constructors

Constructor and Description
KafkaBolt()

Method Summary

Modifier and Type	Method and Description
void	cleanup()
void	declareOutputFields(OutputFieldsDeclarer declarer)
protected org.apache.kafka.clients.producer.KafkaProducer<K,V>	mkProducer(Properties props) Intended to be overridden for tests.
void	prepare(Map stormConf, TopologyContext context, OutputCollector collector)
protected void	process(Tuple input)
void	setAsync(boolean async) If set to true(the default) the bolt will not wait for the message to be fully sent to Kafka before getting another tuple to send.
void	setFireAndForget(boolean fireAndForget) If set to true the bolt will assume that sending a message to kafka will succeed and will ack the tuple as soon as it has handed the message off to the producer API if false (the default) the message will be acked after it was successfully sent to kafka or failed if it was not successfully sent.
String	toString()
KafkaBolt<K,V>	withProducerProperties(Properties producerProperties)
KafkaBolt<K,V>	withTopicSelector(KafkaTopicSelector selector)
KafkaBolt<K,V>	withTopicSelector(String topic) Set the messages to be published to a single topic
KafkaBolt<K,V>	withTupleToKafkaMapper(TupleToKafkaMapper<K,V> mapper)

Methods inherited from class org.apache.storm.topology.base.BaseTickTupleAwareRichBolt

execute, onTickTuple

Methods inherited from class org.apache.storm.topology.base.BaseComponent

getComponentConfiguration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.storm.topology.IComponent

getComponentConfiguration

Field Detail

TOPIC

public static final String TOPIC

See Also:

Constant Field Values

Constructor Detail

KafkaBolt

public KafkaBolt()

Method Detail

withTupleToKafkaMapper

public KafkaBolt<K,V> withTupleToKafkaMapper(TupleToKafkaMapper<K,V> mapper)

withTopicSelector

public KafkaBolt<K,V> withTopicSelector(String topic)

Set the messages to be published to a single topic

Parameters:

topic - the topic to publish to

Returns:

this

withTopicSelector

public KafkaBolt<K,V> withTopicSelector(KafkaTopicSelector selector)

withProducerProperties

public KafkaBolt<K,V> withProducerProperties(Properties producerProperties)

prepare

public void prepare(Map stormConf,
                    TopologyContext context,
                    OutputCollector collector)

mkProducer

protected org.apache.kafka.clients.producer.KafkaProducer<K,V> mkProducer(Properties props)

Intended to be overridden for tests. Make the producer with the given props

process

protected void process(Tuple input)

Specified by:

process in class BaseTickTupleAwareRichBolt

declareOutputFields

public void declareOutputFields(OutputFieldsDeclarer declarer)

cleanup

public void cleanup()

Specified by:

cleanup in interface IBolt

Overrides:

cleanup in class BaseRichBolt

setFireAndForget

public void setFireAndForget(boolean fireAndForget)

If set to true the bolt will assume that sending a message to kafka will succeed and will ack the tuple as soon as it has handed the message off to the producer API if false (the default) the message will be acked after it was successfully sent to kafka or failed if it was not successfully sent.

Parameters:

fireAndForget -

setAsync

public void setAsync(boolean async)

If set to true(the default) the bolt will not wait for the message to be fully sent to Kafka before getting another tuple to send.

Parameters:

async - true to have multiple tuples in flight to kafka, else false.

toString

public String toString()

Overrides:

toString in class Object