// Do not edit directly!
// This file was generated by camel-quarkus-maven-plugin:update-extension-doc-page
= Spark
:linkattrs:
:cq-artifact-id: camel-quarkus-spark
:cq-native-supported: false
:cq-status: Preview
:cq-status-deprecation: Preview
:cq-description: Send RDD or DataFrame jobs to Apache Spark clusters.
:cq-deprecated: false
:cq-jvm-since: 1.3.0
:cq-native-since: n/a

[.badges]
[.badge-key]##JVM since##[.badge-supported]##1.3.0## [.badge-key]##Native##[.badge-unsupported]##unsupported##

Send RDD or DataFrame jobs to Apache Spark clusters.

== What's inside

* xref:{cq-camel-components}::spark-component.adoc[Spark component], URI syntax: `spark:endpointType`

Please refer to the above link for usage and configuration details.

== Maven coordinates

[source,xml]
----
<dependency>
    <groupId>org.apache.camel.quarkus</groupId>
    <artifactId>camel-quarkus-spark</artifactId>
</dependency>
----

Check the xref:user-guide/index.adoc[User guide] for more information about writing Camel Quarkus applications.

== Usage

=== Special BOM

`camel-quarkus-spark` is special compared to other Camel Quarkus extensions which are managed in
`camel-quarkus-bom` together with their important dependencies. This is because the Camel Spark extension depends
on rather old versions of various libraries, that may conflict with their newer versions required by other Quarkus and
Camel Quarkus extensions. Therefore, `camel-quarkus-spark` extension and its dependencies are managed in a special
BOM called `camel-quarkus-spark-bom`.

For `camel-quarkus-spark` to work properly in end user applications, `camel-quarkus-spark-bom` need to be
imported before any other Quarkus or Camel Quarkus BOMs.

[source,xml]
----
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-spark-bom</artifactId>
            <version>${camel-quarkus.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>

        <!-- Other BOMs -->
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-bom</artifactId>
            <version>${camel-quarkus.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>
----

[WARNING]
====
The above setup will make `camel-quarkus-spark` work properly but it may break other extensions which are
sensitive to downgrades of the transitive dependencies done in `camel-quarkus-spark-bom`.
====


== Camel Quarkus limitations

* `spark:dataframe` does not work on Java 9+ because Spark 2.x does not support Java 9+. This will be solved when
  Spark is upgraded to 3.x in Camel, see https://github.com/apache/camel-quarkus/issues/1955[issue #1955].
* `spark:hive` is does not work, see https://github.com/apache/camel-quarkus/issues/1956[issue #1956].

