[[composite_builds]]
= Composing builds

[[composite_build_intro]]
== What is a composite build?

A composite build is simply a build that includes other builds.
In many ways a composite build is similar to a Gradle multi-project build, except that instead of including single `projects`, complete `builds` are included.

Composite builds allow you to:

* combine builds that are usually developed independently, for instance when trying out a bug fix in a library that your application uses
* decompose a large multi-project build into smaller, more isolated chunks that can be worked in independently or together as needed

A build that is included in a composite build is referred to, naturally enough, as an "included build". Included builds do not share any configuration with the composite build, or the other included builds. Each included build is configured and executed in isolation.

Included builds interact with other builds via <<resolution_rules.adoc#sec:dependency_substitution_rules,_dependency substitution_>>.
If any build in the composite has a dependency that can be satisfied by the included build, then that dependency will be replaced by a project dependency on the included build.
_Because of the reliance on dependency substitution, composite builds may force configurations to be resolved earlier, when composing the task execution graph.
This can have a negative impact on overall build performance, because these configurations are not resolved in parallel._

By default, Gradle will attempt to determine the dependencies that can be substituted by an included build. However for more flexibility, it is possible to explicitly declare these substitutions if the default ones determined by Gradle are not correct for the composite. See <<#included_build_declaring_substitutions,Declaring substitutions>>.

As well as consuming outputs via project dependencies, a composite build can directly declare task dependencies on included builds. Included builds are isolated, and are not able to declare task dependencies on the composite build or on other included builds.
See <<#included_build_task_dependencies,Depending on tasks in an included build>>.

[[defining_composite_builds]]
== Defining a composite build

The following examples demonstrate the various ways that 2 Gradle builds that are normally developed separately can be combined into a composite build. For these examples, the `my-utils` multi-project build produces 2 different java libraries (`number-utils` and `string-utils`), and the `my-app` build produces an executable using functions from those libraries.

The `my-app` build does not have direct dependencies on `my-utils`.
Instead, it declares binary dependencies on the libraries produced by `my-utils`.

.Dependencies of my-app
====
include::sample[dir="samples/build-organization/composite-builds/basic/groovy",files="my-app/app/build.gradle[]"]
include::sample[dir="samples/build-organization/composite-builds/basic/kotlin",files="my-app/app/build.gradle.kts[]"]
====


[[command_line_composite]]
=== Defining a composite build via `--include-build`

The `--include-build` command-line argument turns the executed build into a composite, substituting dependencies from the included build into the executed build.

=== Example: Declaring a command-line composite

.Output of **`gradle --include-build ../my-utils run`**
----
> gradle --include-build ../my-utils run
include::{samplesPath}/build-organization/composite-builds/basic/tests/basicCli.out[]
----

[[settings_defined_composite]]
=== Defining a composite build via the settings file

It's possible to make the above arrangement persistent, by using link:{groovyDslPath}/org.gradle.api.initialization.Settings.html#org.gradle.api.initialization.Settings:includeBuild(java.lang.Object)[Settings.includeBuild(java.lang.Object)] to declare the included build in the `settings.gradle` (or `settings.gradle.kts` in Kotlin) file.
The settings file can be used to add subprojects and included builds at the same time. Included builds are added by location.
See the examples below for more details.

[[separate_composite]]
=== Defining a separate composite build

One downside of the above approach is that it requires you to modify an existing build, rendering it less useful as a standalone build. One way to avoid this is to define a separate composite build, whose only purpose is to combine otherwise separate builds.

.Declaring a separate composite
====
include::sample[dir="samples/build-organization/composite-builds/basic/groovy",files="settings.gradle[]"]
include::sample[dir="samples/build-organization/composite-builds/basic/kotlin",files="settings.gradle.kts[]"]
====

In this scenario, the 'main' build that is executed is the composite, and it doesn't define any useful tasks to execute itself. In order to execute the 'run' task in the 'my-app' build, the composite build must define a delegating task.

.Depending on task from included build
====
include::sample[dir="samples/build-organization/composite-builds/basic/groovy",files="build.gradle[tags=run]"]
include::sample[dir="samples/build-organization/composite-builds/basic/kotlin",files="build.gradle.kts[tags=run]"]
====

More details about tasks that depend on included build tasks are below.

[[included_builds]]
=== Restrictions on included builds

Most builds can be included into a composite, including other composite builds.
However there are some limitations.

Every included build:

* must not have a `rootProject.name` the same as another included build.
* must not have a `rootProject.name` the same as a top-level project of the composite build.
* must not have a `rootProject.name` the same as the composite build `rootProject.name`.


[[interacting_with_composite_builds]]
== Interacting with a composite build

In general, interacting with a composite build is much the same as a regular multi-project build.
Tasks can be executed, tests can be run, and builds can be imported into the IDE.


[[composite_build_executing_tasks]]
=== Executing tasks

Tasks from the composite build can be executed from the command-line or from your IDE.
Executing a task will result in direct task dependencies being executed, as well as those tasks required to build dependency artifacts from included builds.

You call a task in an included build using a fully qualified path, which usually is `:included-build-name:subproject-name:taskName`.
Subproject and task names can be <<command_line_interface.adoc#sec:name_abbreviation,abbreviated>>.
This is not supported for included build names.

----
$ ./gradlew :included-build:subproject-a:compileJava
> Task :included-build:subproject-a:compileJava

$ ./gradlew :included-build:sA:cJ
> Task :included-build:subproject-a:compileJava
----

To <<command_line_interface.adoc#sec:excluding_tasks_from_the_command_line,exclude a task from the command line>>, you also need to provide the fully qualified path to the task.

[NOTE]
====
Included build tasks are automatically executed in order to generate required dependency artifacts, or the <<#included_build_task_dependencies,including build can declare a dependency on a task from an included build>>.
====

[[composite_build_ide_integration]]
=== Importing into the IDE

One of the most useful features of composite builds is IDE integration. By applying the <<idea_plugin.adoc#idea_plugin,idea>> or <<eclipse_plugin.adoc#eclipse_plugin,eclipse>> plugin to your build, it is possible to generate a single IDEA or Eclipse project that permits all builds in the composite to be developed together.

In addition to these Gradle plugins, recent versions of https://www.jetbrains.com/idea/[IntelliJ IDEA] and https://projects.eclipse.org/projects/tools.buildship[Eclipse Buildship] support direct import of a composite build.

Importing a composite build permits sources from separate Gradle builds to be easily developed together. For every included build, each sub-project is included as an IDEA Module or Eclipse Project.
Source dependencies are configured, providing cross-build navigation and refactoring.

[[included_build_declaring_substitutions]]
== Declaring the dependencies substituted by an included build

By default, Gradle will configure each included build in order to determine the dependencies it can provide.
The algorithm for doing this is very simple: Gradle will inspect the group and name for the projects in the included build, and substitute project dependencies for any external dependency matching `${project.group}:${project.name}`.

NOTE: By default, substitutions are not registered for the _main_ build.
To make the (sub)projects of the main build addressable by `${project.group}:${project.name}`, you can tell Gradle to treat the main build like an included build by self-including it: `includeBuild(".")`.

There are cases when the default substitutions determined by Gradle are not sufficient, or they are not correct for a particular composite.
For these cases it is possible to explicitly declare the substitutions for an included build.
Take for example a single-project build 'anonymous-library', that produces a java utility library but does not declare a value for the group attribute:

.Build that does not declare group attribute
====
include::sample[dir="samples/build-organization/composite-builds/declared-substitution/groovy/anonymous-library",files="build.gradle[]"]
include::sample[dir="samples/build-organization/composite-builds/declared-substitution/kotlin/anonymous-library",files="build.gradle.kts[]"]
====

When this build is included in a composite, it will attempt to substitute for the dependency module "undefined:anonymous-library" ("undefined" being the default value for `project.group`, and "anonymous-library" being the root project name).
Clearly this isn't going to be very useful in a composite build.
To use the unpublished library unmodified in a composite build, the composing build can explicitly declare the substitutions that it provides:

.Declaring the substitutions for an included build
====
include::sample[dir="samples/build-organization/composite-builds/declared-substitution/groovy",files="settings.gradle[]"]
include::sample[dir="samples/build-organization/composite-builds/declared-substitution/kotlin",files="settings.gradle.kts[]"]
====

With this configuration, the "my-app" composite build will substitute any dependency on `org.sample:number-utils` with a dependency on the root project of "anonymous-library".

[[included_build_substitution_requirements]]
=== Cases where included build substitutions must be declared

Many builds that use the `uploadArchives` task to publish artifacts will function automatically as an included build, without declared substitutions.
Here are some common cases where declared substitutions are required:

* When the `archivesBaseName` property is used to set the name of the published artifact.
* When a configuration other than `default` is published: this usually means a task other than `uploadArchives` is used.
* When the `MavenPom.addFilter()` is used to publish artifacts that don't match the project name.
* When the `maven-publish` or `ivy-publish` plugins are used for publishing, and the publication coordinates don't match `${project.group}:${project.name}`.

[[included_build_substitution_limitations]]
=== Cases where composite build substitutions won't work

Some builds won't function correctly when included in a composite, even when dependency substitutions are explicitly declared. This limitation is due to the fact that a project dependency that is substituted will always point to the `default` configuration of the target project.
Any time that the artifacts and dependencies specified for the default configuration of a project don't match what is actually published to a repository, then the composite build may exhibit different behaviour.

Here are some cases where the publish module metadata may be different from the project default configuration:

* When a configuration other than `default` is published.
* When the `maven-publish` or `ivy-publish` plugins are used.
* When the `POM` or `ivy.xml` file is tweaked as part of publication.

Builds using these features function incorrectly when included in a composite build.
We plan to improve this in the future.

[[included_build_task_dependencies]]
== Depending on tasks in an included build

While included builds are isolated from one another and cannot declare direct dependencies, a composite build is able to declare task dependencies on its included builds. The included builds are accessed using link:{groovyDslPath}/org.gradle.api.invocation.Gradle.html#org.gradle.api.invocation.Gradle:includedBuilds[Gradle.getIncludedBuilds()] or link:{groovyDslPath}/org.gradle.api.invocation.Gradle.html#org.gradle.api.invocation.Gradle:includedBuild(java.lang.String)[Gradle.includedBuild(java.lang.String)], and a task reference is obtained via the link:{groovyDslPath}/org.gradle.api.initialization.IncludedBuild.html#org.gradle.api.initialization.IncludedBuild:task(java.lang.String)[IncludedBuild.task(java.lang.String)] method.

Using these APIs, it is possible to declare a dependency on a task in a particular included build, or tasks with a certain path in all or some of the included builds.

.Depending on a single task from an included build
====
include::sample[dir="samples/build-organization/composite-builds/basic/groovy",files="build.gradle[tags=run]"]
include::sample[dir="samples/build-organization/composite-builds/basic/kotlin",files="build.gradle.kts[tags=run]"]
====

.Depending on a task with path in all included builds
====
include::sample[dir="samples/build-organization/composite-builds/hierarchical-multirepo/groovy",files="build.gradle"]
include::sample[dir="samples/build-organization/composite-builds/hierarchical-multirepo/kotlin",files="build.gradle.kts"]
====

[[current_limitations_and_future_work]]
== Current limitations and future plans for composite builds

We think composite builds are pretty useful already.
However, there are some things that don't yet work the way we'd like, and other improvements that we think will make things work even better.

Limitations of the current implementation include:

* No support for included builds that have publications that don't mirror the project default configuration. See <<#included_build_substitution_limitations,Cases where composite builds won't work>>.
* Software model based native builds are not supported. (Binary dependencies are not yet supported for native builds).
* Multiple composite builds may conflict when run in parallel, if more than one includes the same build. Gradle does not share the project lock of a shared composite build to between Gradle invocation to prevent concurrent execution.

Improvements we have planned for upcoming releases include:

* Better detection of dependency substitution, for builds that publish with custom coordinates, builds that produce multiple components, etc. This will reduce the cases where dependency substitution needs to be explicitly declared for an included build.
* Making the implicit `buildSrc` project an included build.
