[[structure_large_projects]]
= Structuring Large Projects

As software projects grow, it is common to organize large systems into components that are connected following a certain software architecture.
Usually, it makes sense to also organize the artifacts (source code etc.) that make up the software in repositories and folder structures that reflect component boundaries and architecture.
If Gradle is used to build such a software system, it can help you to perform this organisation and enforce boundaries between components.

You can look at Gradle as a modelling tool for your software:
It allows you to describe the coarse grained structure and architecture of your software in a model written in Gradle's DSLs.
The build tool Gradle can then interpret this model to build the software.

== An example

How you architect your software and how you divide it into components depends on what you are building.
There is no one-fits-all solution.
Hence, Gradle does not enforce a particular structure on you, but rather offers the tools to model your individual setup.

Still, to exemplify these features, we explore a sample project with the following architecture:

image::software_architecture.png[]

[NOTE]
You can download link:../samples/sample_structuring_software_projects.html[the full sample] to explore, build and run it.

The structure follows a typical setup that can be found in similar form in many commonly used software architectures.

- At the bottom we define our domain model.
There are two components:
a _domain-model_ component contains the model definition (i.e. a set of data classes) and
a _state_ component is responsible for managing a modifiable state of the model during application runtime.
- On top of the model, business logic for different features is implemented independently of each other and independently of a concrete application technology.
In this example, we have two features: _user_ and _admin_.
- At the top, we have concrete applications users use to interact with the features.
In the example, we build a Spring Boot application that supports both features via a web browser.
And an Android app that only supports the _user_ feature.

Our components may rely on existing components that are retrieved from binary repositories.
For example, the Spring Boot and Android frameworks.

Apart from the production code, there are also components that deal with building and delivering the product:

- The _build-logic_ component contains the configuration details about building the software.
For example, defining a Java version to use or configuring a test framework.
It may also contain additional build logic for Gradle (<<custom_plugins.adoc#custom_plugins,custom plugins>> with <<custom_tasks.adoc#custom_tasks,custom tasks>>) that is not covered by commonly available Gradle plugins.
- The _platforms_ component is a central place to define which versions of external components are to be used in all of our own components.
By that, it defines the constraints for the environments – that is, _the platforms_ – to build, test and run the software product.
- The _aggregation_ component contains the setup of the delivery pipeline that is required to push the product to production and
doing automated end-to-end testing as part of that.
Basically, this is the part of the build that is not required on local development machines.

[NOTE]
The domain of our example is to get informed about link:https://gradle.org/releases/[Gradle Build Tool releases].
Concretely, the application lists Gradle releases with links to release notes (_user_ feature) and
offers an administration interface for the range of releases to be listed (_admin_ feature).

== Reflecting software architecture in project structure

Let's look at how to implement the architecture of the sample with Gradle.
We can represent each of our _components_ as a separate _Gradle build_.
We'll get to the details of what that means and how components are connected in a bit.

Each Gradle build has its own folder.
The minimum to make these folders Gradle builds is to add an empty `settings.gradle(.kts)` file to each of them.
Let's do this for all the components we have in our software:

[source.multi-language-sample,groovy]
----
├── android-app
│   └── settings.gradle
├── server-application
│   └── settings.gradle
│
├── admin-feature
│   └── settings.gradle
├── user-feature
│   └── settings.gradle
│
├── state
│   └── settings.gradle
│
├── domain-model
│   └── settings.gradle
│
├── build-logic
│   └── settings.gradle
│
├── platforms
│   └── settings.gradle
│
└── aggregation
    └── settings.gradle
----

[source.multi-language-sample,kotlin]
----
├── android-app
│   └── settings.gradle.kts
├── server-application
│   └── settings.gradle.kts
│
├── admin-feature
│   └── settings.gradle.kts
├── user-feature
│   └── settings.gradle.kts
│
├── state
│   └── settings.gradle.kts
│
├── domain-model
│   └── settings.gradle.kts
│
├── build-logic
│   └── settings.gradle.kts
│
├── platforms
│   └── settings.gradle.kts
│
└── aggregation
    └── settings.gradle.kts
----

In the listing, each component lives in a separate folder.
Here we arrange them as a flat list in a root folder.
This root folder can be used as root of a Git repository for example.

This is only the setup of the sample.
You can freely choose where to physically locate your components.
For instance, you can group all components that live in one "layer" in a common subfolder.
Or, since these are all independent Gradle builds, you can have each component live in a separate repository.
It's up to you to decide what works best for you, the software you are building, and the teams working on it.

== Defining an inner structure for components

Before we get to the topic of connecting the components, lets first look at them individually.

So far, each component is just an empty folder with an empty `settings.gradle(.kts)` file indicating that this is a component Gradle can work with in some form.
To fill the component with content, you should define at least one project (referred to as subproject in Gradle's DSLs) in it.

You can start with components consisting of a single project each, but introduce additional projects later to structure a single component more internally.
In our sample, we start with a single project in each component.

A project is added by using the `include()` construct in the settings file.

.Use include() to add a project to a component
====
include::sample[dir="samples/build-organization/structuring-software-projects/groovy",files="domain-model/settings.gradle[tags=include]"]
include::sample[dir="samples/build-organization/structuring-software-projects/kotlin",files="domain-model/settings.gradle.kts[tags=include]"]
====

Once included, you may create a folder matching the project name and create a `build.gradle(.kts)` file in it to configure that part of the component.
You can find more information in the chapter about <<multi_project_builds.adoc#multi_project_builds,structuring Gradle builds for a single software component>>.

== Assigning types to components

Let's zoom into the _domain-model_ component:

[source.multi-language-sample,groovy]
----
└── domain-model              <-- component
    ├── settings.gradle       <-- define inner structure of component and where to locate other components
    └── release               <-- project in component
        └── build.gradle      <-- defines type of the project and its dependencies
----

[source.multi-language-sample,kotlin]
----
└── domain-model              <-- component
    ├── settings.gradle.kts   <-- define inner structure of component and where to locate other components
    └── release               <-- project in component
        └── build.gradle.kts  <-- defines type of the project and its dependencies
----

Initially, `release/build.gradle(.kts)` is empty.
The project is of no specific type and does not offer any useful content.
If we add more files to the `domain-model/release` folder now, for example Java source files, Gradle won't know what to do with these files and just ignore them.
We need to assign a type to the project to make Gradle aware of the purpose of such files in the project.

In Gradle, you assign a type to a project by <<plugins.adoc#plugins,applying a plugin>>.
The simplest thing you can do is applying one of Gradle's core plugins, like `base` or `java-library`.

However, usually you have additional configuration to do in the context of the product you are building.
For example, if your project should be a "Java Library", it would not only apply the `java-library` plugin but also configure details such as setting the Java version to 11.
You can add details like that directly in `release/build.gradle(.kts)` but than you would have to repeat them in other components that also contain "Java Library" projects.

Thus, the recommendation is to start using custom project types right away:

.Assign the custom type `com.example.java-library` to a project in the `user-feature` component
====
include::sample[dir="samples/build-organization/structuring-software-projects/groovy",files="user-feature/data/build.gradle[tags=project-type]"]
include::sample[dir="samples/build-organization/structuring-software-projects/kotlin",files="user-feature/data/build.gradle.kts[tags=project-type]"]
====
.Assign the custom type `com.example.kotlin-library` to a project in the `domain-model` component
====
include::sample[dir="samples/build-organization/structuring-software-projects/groovy",files="domain-model/release/build.gradle[tags=project-type]"]
include::sample[dir="samples/build-organization/structuring-software-projects/kotlin",files="domain-model/release/build.gradle.kts[tags=project-type]"]
====

As stated above, a project type is represented by a plugin in Gradle.
We thus define custom project types, such as `com.example.java-library` and `com.example.kotlin-library`, as plugins.
The next section explains how to define such plugins.

== Defining custom project types as convention plugins

Where do we get `com.example.kotlin-library` plugin from?
This is what our `build-logic` component is for.

The `build-logic` component contains project types that Gradle itself understands as build configuration.
That is, Gradle plugins for your specific needs, which we call _convention plugins_.

Currently, there are different project types you can use to define _convention plugins_ depending on which tools and languages you prefer.
In general, any JVM language (Java, Groovy, Kotlin, Scala) can be used to write Gradle plugins as classes that implement the `Plugin<Project>` interface.
The most compact way however, is to write them as scripts in Gradle's Groovy or Kotlin DSL.

Which method you choose is up to you.
If you are familiar with one of Gradle's DSLs you may choose that, as it is the most compact way to write _convention plugins_.
If you are new to Gradle (and also new to Groovy and Kotlin) you may prefer to write the _convention plugins_ in Java or another language like Scala.
Then you reduce the interaction with Gradle's Groovy or Kotlin DSL to a minimum.

You need to use one of the following project types (i.e. Gradle core plugins) in projects of your `build-logic` component:

- `kotlin-dsl` – Build logic projects with this type (i.e., that apply the `kotlin-dsl` plugin) allow you to write convention plugins as `.gradle.kts` files in `src/main/kotlin`.
- `groovy-gradle-plugin` – Build logic projects with this type (i.e., that apply the `groovy-gradle-plugin` plugin) allow you to write convention plugins as `.gradle` files in `src/main/groovy`.
- `java-gradle-plugin` – Build logic projects with this type (i.e., that apply the `java-gradle-plugin` plugin) allow you to write convention plugins as `.java` classes that implement the `Plugin<Project>` interface in `src/main/java`.
If you apply other JVM language plugins on top, like `groovy`, `scala` or `org.jetbrains.kotlin.jvm`, you can also write the plugin class in the corresponding language.

In our sample, we choose the option of using Gradle's DSLs for the convention plugins.
The `build-logic` component has several projects where each of them defines a project type through a convention plugin –
`java-library`, `kotlin-library`, `spring-application`, `android-application`.
Also, there is a project called `commons` for build configuration shared by all our project types.

.Assign a custom type to a component's project by applying a convention plugin
====
include::sample[dir="samples/build-organization/structuring-software-projects/groovy",files="build-logic/spring-boot-application/build.gradle"]
include::sample[dir="samples/build-organization/structuring-software-projects/kotlin",files="build-logic/spring-boot-application/build.gradle.kts"]
====

Looking at the `build.gradle(.kts)` of the `build-logic` project for spring boot applications, we see:

<1> That it is of type `groovy-gradle-plugin` or `kotlin-dsl` to allow convention plugins written in the corresponding DSL
<2> It depends on our own `plugins-platform` from the `platforms` component
<3> It depends on the `commons` project from `build-logic` to have access to our own commons convention plugin
<4> It depends on the Spring Boot Gradle plugin from the Gradle Plugin Portal so that we may apply it

Now, we can write the convention plugin for Spring application like this:

.Defining a custom project type as convention plugin
====
include::sample[dir="samples/build-organization/structuring-software-projects/groovy",files="build-logic/spring-boot-application/src/main/groovy/com.example.spring-boot-application.gradle"]
include::sample[dir="samples/build-organization/structuring-software-projects/kotlin",files="build-logic/spring-boot-application/src/main/kotlin/com.example.spring-boot-application.gradle.kts"]
====

We see that we apply our own `com.example.commons` plugin which is another convention plugin that, among other things, configures the Java version we are targeting and adds a dependency to a platform (`com.example.platform:product-platform` from our `platforms` component).
And we apply the spring boot plugin.
Furthermore, we add two dependencies that Spring Boot projects should always have in our context.

Similarly, we define convention plugins for "Java Libraries", "Kotlin Libraries" and "Android Applications".
With that, we have four different project types defined that we assign to the projects of our production code components.

You can find more information about writing convention plugins in section on <<sharing_build_logic_between_subprojects.adoc#sharing_build_logic_between_subprojects,sharing build logic>> and the link:../samples/sample_convention_plugins.html[associated sample].
For using classes to implement plugins, and for writing more advanced custom build logic, consult the chapter on <<custom_plugins.adoc#custom_plugins,Gradle plugin development>>.

== Connecting components

As demonstrated in the architecture figure, our production code components depend on each other.
Above, we already saw that the `platforms` component is used in the `build-logic` component.
We also said that we want to use the `build-logic` component, which declares project types through convention plugins,
to asssign those types to the projects in our production code components.

How do you define these dependencies?
There are two distinct things to do:

1. _Make components (builds) known to each other._ This is done by adding `includeBuild(...)` statements to `settings.gradle(.kts)`.
This is _not_ adding a dependency between (projects of) components.
It just makes the physical location of one component known to another.
In that sense it is similar to repository declarations to discover binary components.
Consult <<composite_builds.adoc#defining_composite_builds,the section on defining composite builds>> for more information about how to include builds.
2. _Declare dependencies between (projects of) components._
This is done similarly to <<declaring_dependencies.adoc#declaring-dependencies,declaring dependencies to binary components>> by using GA (_group_ and _artifact_) coordinates in the `dependencies { }` block of a `build.gradle(.kts)` file:
`implementation("com.example.platform:product-platform")`.
Or, if the included component provides a plugin, you <<plugins.adoc#sec:plugins_block,apply the plugin by ID>> similar to how you would apply a plugin from the plugin portal:
`plugins { id("com.example.java-library") }`

As another example, consider the setup of our `server-application` component:

.Defining component locations and inner structure of the `server-application` component
====
include::sample[dir="samples/build-organization/structuring-software-projects/groovy",files="server-application/settings.gradle"]
include::sample[dir="samples/build-organization/structuring-software-projects/kotlin",files="server-application/settings.gradle.kts"]
====

We see that the `settings.gradle(.kts)` file only defines location for build logic components, other production code components and the inner structue of the component.
Only the `build.gradle(.kts)` file in the `app` project then defines actual dependencies by applying a convention plugin and utilizing the _dependencies_ block.

.Configuring type and dependencies of the `app` project inside the `server-application` component
====
include::sample[dir="samples/build-organization/structuring-software-projects/groovy",files="server-application/app/build.gradle"]
include::sample[dir="samples/build-organization/structuring-software-projects/kotlin",files="server-application/app/build.gradle.kts"]
====

== The model of your software

That's it.
This chapter gave an overview of which techniques to use to structure a software project into components with Gradle by following a sample.
Download link:../samples/sample_structuring_software_projects.html[the full sample] to explore further details.
The <<structuring_software_products_details.adoc#tweaking_structure,next chapter>> covers more details about how to work with and evolve this kind of project structure.
The <<composite_builds.adoc#composite_builds,chapter on composite builds>> gives you more technical background about the capabilities build composition offers.

To summarize, if you follow the suggestions from this chapter, your setup should clearly separate the following concerns to give you a flexible and clean model of your software product:

1. _Write compact `build.gradle(.kts)` files._
   While in traditional Gradle builds these files tend to grow and mix a lot of different concerns, the structure presented here keeps these files compact.
   In most cases they only declare a project type by applying a single convention plugin and dependencies in the `dependencies {}` block.
   They might include minimal project-specific configuration, but these should be kept as minimal as possible.
   This also makes builds less dependent on Gradle's DSLs: If you put your build logic into convention plugins, you can write it directly in Java if you like.
2. _Isolate cross-cutting technical concerns into project types._
   Technical motivated build configuration is often cross-cutting the software architecture.
   If a project is a "Java Library" or "Kotlin Library" may be totally independent of where it is located in the hierarchy of your components.
   Using convention plugins, allows you to isolate the definition of such project types in a central place, while still reusing them wherever needed.
   (This is a huge advantage over so called <<sharing_build_logic_between_subprojects.adoc#sec:convention_plugins_vs_cross_configuration,cross project configuration>>, which has been popular with older Gradle versions, but is bound to the hierarchy of your project.)
3. _Declare the origins of components in a central place._
   In this structure, the places where a build can find other components, independent of whether they are <<declaring_repositories.adoc#sub:centralized-repository-declaration,located in a binary repository>> or are <<composite_builds.adoc#settings_defined_composite,available locally as other Gradle builds>>, are defined centrally in the `settings.gradle(.kts)` file.
   This makes it easy to change the origin of a component and <<structuring_software_products_details.adoc#binary_vs_source_components,move from a binary to a from-source version of a component>>.
   Note that <<structuring_software_products_details.adoc#settings_convention_plugins,there are different strategies to avoid duplicating this information>> in each `settings.gradle(.kts)` of each of your components.
4. _Declare platforms in a central place._
   Having a platform component as in the example is optional.
   You could do things without, e.g. by declaring <<dependency_constraints.adoc#dependency-constraints,dependency constraints>> directly in your convention plugins.
   However, <<platforms.adoc#sec:sharing-dep-versions-between-projects,platforms>> are a good option to have all the boundaries for the environment in which your software operates defined in a central place.
