// Copyright (C) 2023 Gradle, Inc.
//
// Licensed under the Creative Commons Attribution-Noncommercial-ShareAlike 4.0 International License.;
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      https://creativecommons.org/licenses/by-nc-sa/4.0/
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

[[test_report_aggregation_plugin]]
= The Test Report Aggregation Plugin

The Test Report Aggregation plugin (plugin id: `test-report-aggregation`) provides tasks and configurations used to aggregate the results of multiple link:{groovyDslPath}/org.gradle.api.tasks.testing.Test.html[Test] task invocations (potentially spanning multiple Gradle projects) into a single HTML report.

[[sec:test_report_aggregation_usage]]
== Usage

To use the Test Report Aggregation plugin, include the following in your build script:

[.multi-language-text.lang-groovy]
----
plugins {
    id 'test-report-aggregation'
}
----
[.multi-language-text.lang-kotlin]
----
plugins {
    id("test-report-aggregation")
}
----

Note that this plugin takes no action unless applied in concert with the <<jvm_test_suite_plugin#jvm_test_suite_plugin,JVM Test Suite Plugin>>.  The <<java_plugin#java_plugin, Java Plugin>> automatically applies the JVM Test Suite Plugin.

There are now two ways to collect test results across multiple subprojects:

1. From the distribution's project, such as an application or WAR subproject -> link:../samples/sample_jvm_multi_project_with_test_aggregation_distribution.html[distribution sample]
2. Using a standalone project to specify subprojects -> link:../samples/sample_jvm_multi_project_with_test_aggregation_standalone.html[standalone sample]

Example 2 could also be used to aggregate results via the root project.

WARNING: The Test Report Aggregation plugin does not currently work with the ``com.android.application`` plugin.

[[sec:test_report_aggregation_tasks]]
== Tasks

When the project also applies the `jvm-test-suite` plugin, the following tasks are added for each test suite:

`__testSuite__AggregateTestReport` — link:{groovyDslPath}/org.gradle.api.tasks.testing.TestReport.html[TestReport]::
_Depends on_: Artifacts of variants matching the below attributes
+
Collects variants of direct and transitive project dependencies via the `testReportAggregation` configuration. The following Attributes will be matched:

[source]
----
    - org.gradle.category              = verification   // <1>
    - org.gradle.testsuite.type        = unit-test      // <2>
    - org.gradle.verificationtype      = test-results   // <3>
----
<1> Category attribute; value is fixed.
<2> TestSuiteType attribute; value is derived from link:{javadocPath}/org/gradle/api/plugins/jvm/JvmTestSuite.html#getTestType--[JvmTestSuite#getTestType()].
<3> VerificationType attribute; value is fixed.

More information about the variants produced by test execution are available in the <<jvm_test_suite_plugin#sec:outgoing_variants,Outgoing Variants>> section of the JVM Test Suite Plugin documentation.

== Reports

[IMPORTANT]
====
By default, Gradle stops executing tasks when any task fails -- including test failures.
To ensure that your builds always generate aggregation reports, specify the `--continue` option in your Gradle command.
For more information, see <<command_line_interface.adoc#sec:continue_build_on_failure, continuing the build when a failure occurs>>.
====

=== Automatic report creation

When the project also applies the `jvm-test-suite` plugin, the following reporting objects are added for each test suite:

`__testSuite__AggregateTestReport` — link:{javadocPath}/org/gradle/api/tasks/testing/AggregateTestReport.html[AggregateTestReport]::
Creates a container to parameterize the link:{javadocPath}/org/gradle/api/attributes/TestSuiteType.html[TestSuiteType] value.

=== Manual report creation

When the project does not apply the `jvm-test-suite` plugin, you must manually register one or more reports:

.Create a reporting container
====
include::sample[dir="samples/incubating/java/jvm-multi-project-with-test-aggregation-standalone/kotlin/test-results",files="build.gradle.kts[tags=create_report]"]
include::sample[dir="samples/incubating/java/jvm-multi-project-with-test-aggregation-standalone/groovy/test-results",files="build.gradle[tags=create_report]"]
====
<.> Creates a report named `testAggregateTestReport` of type `AggregateTestReport`.  For convenience, sets `TestType` using a constant value from the  link:{javadocPath}/org/gradle/api/attributes/TestSuiteType.html[TestSuiteType] class.  Any String value is acceptable.

Report creation automatically creates backing tasks to aggregate by the given test suite type value.

[[sec:dependency_management]]
== Dependency management
The Test Report Aggregation plugin adds the following dependency configurations:

.Test Report Aggregation plugin - dependency configurations
[cols="a,a", options="header"]
|===
| Name
| Meaning

| `testReportAggregation`
| The configuration used to declare all project dependencies having test result data to be aggregated.
| `aggregateTestReportResults`
| Consumes the project dependencies from the `testReportAggregation` configuration using <<variant_model#sec:variant-aware-matching,variant-aware matching>> to find the appropriate test suite type.
|===

It is not necessary to explicitly add dependencies to the `testReportAggregation` configuration if the project also applies the `jvm-test-suite` plugin.
