[[flink-component]]
= Flink Component
:page-source: components/camel-flink/src/main/docs/flink-component.adoc

*Since Camel 2.18*

// HEADER START
*Only producer is supported*
// HEADER END

*Since Camel 2.18*


This documentation page covers the https://flink.apache.org[Apache Flink]
component for the Apache Camel. The *camel-flink* component provides a
bridge between Camel connectors and Flink tasks. +
This Camel Flink connector provides a way to route message from various
transports, dynamically choosing a flink task to execute, use incoming
message as input data for the task and finally deliver the results back to
the Camel pipeline.

Maven users will need to add the following dependency to
their `pom.xml` for this component:

[source,xml]
------------------------------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-flink</artifactId>
    <version>x.x.x</version>
    <!-- use the same version as your Camel core version -->
</dependency>
------------------------------------------------------------

== URI Format

Currently, the Flink Component supports only Producers. One can create DataSet, DataStream jobs.

[source,java]
-------------------------------------------------
flink:dataset?dataset=#myDataSet&dataSetCallback=#dataSetCallback
flink:datastream?datastream=#myDataStream&dataStreamCallback=#dataStreamCallback
-------------------------------------------------

[[Flink-FlinkEndpointOptions]]
FlinkEndpoint Options



// endpoint options: START
The Flink endpoint is configured using URI syntax:

----
flink:endpointType
----

with the following path and query parameters:

=== Path Parameters (1 parameters):


[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *endpointType* | *Required* Type of the endpoint (dataset, datastream). |  | EndpointType
|===


=== Query Parameters (8 parameters):


[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *collect* (producer) | Indicates if results should be collected or counted. | true | boolean
| *dataSet* (producer) | DataSet to compute against. |  | DataSet
| *dataSetCallback* (producer) | Function performing action against a DataSet. |  | DataSetCallback
| *dataStream* (producer) | DataStream to compute against. |  | DataStream
| *dataStreamCallback* (producer) | Function performing action against a DataStream. |  | DataStreamCallback
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *basicPropertyBinding* (advanced) | Whether the endpoint should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *synchronous* (advanced) | Sets whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported). | false | boolean
|===
// endpoint options: END
// spring-boot-auto-configure options: START
== Spring Boot Auto-Configuration

When using Spring Boot make sure to use the following Maven dependency to have support for auto configuration:

[source,xml]
----
<dependency>
  <groupId>org.apache.camel.springboot</groupId>
  <artifactId>camel-flink-starter</artifactId>
  <version>x.x.x</version>
  <!-- use the same version as your Camel core version -->
</dependency>
----


The component supports 8 options, which are listed below.



[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *camel.component.flink.basic-property-binding* | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | Boolean
| *camel.component.flink.bridge-error-handler* | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | Boolean
| *camel.component.flink.data-set* | DataSet to compute against. The option is a org.apache.flink.api.java.DataSet type. |  | String
| *camel.component.flink.data-set-callback* | Function performing action against a DataSet. The option is a org.apache.camel.component.flink.DataSetCallback type. |  | String
| *camel.component.flink.data-stream* | DataStream to compute against. The option is a org.apache.flink.streaming.api.datastream.DataStream type. |  | String
| *camel.component.flink.data-stream-callback* | Function performing action against a DataStream. The option is a org.apache.camel.component.flink.DataStreamCallback type. |  | String
| *camel.component.flink.enabled* | Whether to enable auto configuration of the flink component. This is enabled by default. |  | Boolean
| *camel.component.flink.lazy-start-producer* | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | Boolean
|===
// spring-boot-auto-configure options: END




== FlinkComponent Options




// component options: START
The Flink component supports 7 options, which are listed below.



[width="100%",cols="2,5,^1,2",options="header"]
|===
| Name | Description | Default | Type
| *dataSet* (producer) | DataSet to compute against. |  | DataSet
| *dataStream* (producer) | DataStream to compute against. |  | DataStream
| *dataSetCallback* (producer) | Function performing action against a DataSet. |  | DataSetCallback
| *dataStreamCallback* (producer) | Function performing action against a DataStream. |  | DataStreamCallback
| *basicPropertyBinding* (advanced) | Whether the component should use basic property binding (Camel 2.x) or the newer property binding with additional capabilities | false | boolean
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
| *bridgeErrorHandler* (consumer) | Allows for bridging the consumer to the Camel routing Error Handler, which mean any exceptions occurred while the consumer is trying to pickup incoming messages, or the likes, will now be processed as a message and handled by the routing Error Handler. By default the consumer will use the org.apache.camel.spi.ExceptionHandler to deal with exceptions, that will be logged at WARN or ERROR level and ignored. | false | boolean
|===
// component options: END





== Flink DataSet Callback

[source,java]
-----------------------------------
@Bean
public DataSetCallback<Long> dataSetCallback() {
    return new DataSetCallback<Long>() {
        public Long onDataSet(DataSet dataSet, Object... objects) {
            try {
                 dataSet.print();
                 return new Long(0);
            } catch (Exception e) {
                 return new Long(-1);
            }
        }
    };
}
-----------------------------------

== Flink DataStream Callback

[source,java]
---------------------------
@Bean
public VoidDataStreamCallback dataStreamCallback() {
    return new VoidDataStreamCallback() {
        @Override
        public void doOnDataStream(DataStream dataStream, Object... objects) throws Exception {
            dataStream.flatMap(new Splitter()).print();

            environment.execute("data stream test");
        }
    };
}
---------------------------

== Camel-Flink Producer call

[source,java]
-----------------------------------
CamelContext camelContext = new SpringCamelContext(context);

String pattern = "foo";

try {
    ProducerTemplate template = camelContext.createProducerTemplate();
    camelContext.start();
    Long count = template.requestBody("flink:dataSet?dataSet=#myDataSet&dataSetCallback=#countLinesContaining", pattern, Long.class);
    } finally {
        camelContext.stop();
    }
-----------------------------------
