// Copyright 2021 the original author or authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

[[test_report_aggregation_plugin]]
= The Test Report Aggregation Plugin

The Test Report Aggregation plugin (plugin id: `test-report-aggregation`) provides tasks and configurations used to aggregate the results of multiple link:{groovyDslPath}/org.gradle.api.tasks.testing.Test.html[Test] task invocations (potentially spanning multiple Gradle projects) into a single HTML report.

[[sec:test_report_aggregation_usage]]
== Usage

To use the Test Report Aggregation plugin, include the following in your build script:

[.multi-language-text.lang-groovy]
----
plugins {
    id 'test-report-aggregation'
}
----
[.multi-language-text.lang-kotlin]
----
plugins {
    id("test-report-aggregation")
}
----

Note that this plugin takes no action unless applied in concert with the <<jvm_test_suite_plugin#jvm_test_suite_plugin,JVM Test Suite Plugin>>.  The <<java_plugin#java_plugin, Java Plugin>> automatically applies the JVM Test Suite Plugin.

There are now two ways to collect test results across multiple subprojects:

1. From the distribution's project, such as an application or WAR subproject -> link:../samples/sample_jvm_multi_project_with_test_aggregation_distribution.html[distribution sample]
2. Using a standalone project to specify subprojects -> link:../samples/sample_jvm_multi_project_with_test_aggregation_standalone.html[standalone sample]

Example 2 could also be used to aggregate results via the root project.

[WARNING]
====
The Test Report Aggregation plugin does not currently work with the ``com.android.application`` plugin.
====

[[sec:test_report_aggregation_tasks]]
== Tasks

When the project also applies the `jvm-test-suite` plugin, the following tasks are added for each test suite:

`__testSuite__AggregateTestReport` — link:{groovyDslPath}/org.gradle.api.tasks.testing.TestReport.html[TestReport]::
_Depends on_: Artifacts of variants matching the below attributes
+
Collects variants of direct and transitive project dependencies via the `testReportAggregation` configuration. The following Attributes will be matched:

[source]
----
    - org.gradle.category              = verification   // <1>
    - org.gradle.testsuite.type        = unit-test      // <2>
    - org.gradle.verificationtype      = test-results   // <3>
----
<1> Category attribute; value is fixed.
<2> TestSuiteType attribute; value is derived from link:{javadocPath}/org/gradle/api/plugins/jvm/JvmTestSuite.html#getTestType--[JvmTestSuite#getTestType()].
<3> VerificationType attribute; value is fixed.

More information about the variants produced by test execution are available in the <<jvm_test_suite_plugin#sec:outgoing_variants,Outgoing Variants>> section of the JVM Test Suite Plugin documentation.

== Reports

[IMPORTANT]
====
By default, Gradle stops executing tasks when any task fails -- including test failures. To ensure that your builds always generate aggregation reports, specify the `--continue` option in your Gradle command. For more information, see <<command_line_interface.adoc#sec:continue_build_on_failure, continuing the build when a failure occurs>>.
====

=== Automatic report creation

When the project also applies the `jvm-test-suite` plugin, the following reporting objects are added for each test suite:

`__testSuite__AggregateTestReport` — link:{javadocPath}/org/gradle/api/tasks/testing/AggregateTestReport.html[AggregateTestReport]::
Creates a container to parameterize the link:{javadocPath}/org/gradle/api/attributes/TestSuiteType.html[TestSuiteType] value.

=== Manual report creation

When the project does not apply the `jvm-test-suite` plugin, you must manually register one or more reports:

.Create a reporting container
====
include::sample[dir="samples/incubating/java/jvm-multi-project-with-test-aggregation-standalone/groovy/test-results",files="build.gradle[tags=create_report]"]
include::sample[dir="samples/incubating/java/jvm-multi-project-with-test-aggregation-standalone/kotlin/test-results",files="build.gradle.kts[tags=create_report]"]
====
<.> Creates a report named `testAggregateTestReport` of type `AggregateTestReport`.  For convenience, sets `TestType` using a constant value from the  link:{javadocPath}/org/gradle/api/attributes/TestSuiteType.html[TestSuiteType] class.  Any String value is acceptable.

Report creation automatically creates backing tasks to aggregate by the given test suite type value.

[[sec:dependency_management]]
== Dependency management
The Test Report Aggregation plugin adds the following dependency configurations:

.Test Report Aggregation plugin - dependency configurations
[cols="a,a", options="header"]
|===
| Name
| Meaning

| `testReportAggregation`
| The configuration used to declare all project dependencies having test result data to be aggregated.
| `__testReportName__Results`
| Consumes the project dependencies from the `testReportAggregation` configuration using <<variant_model#sec:variant-aware-matching,variant-aware matching>> to find the appropriate test suite type.
|===

It is not necessary to explicitly add dependencies to the `testReportAggregation` configuration if the project also applies the `jvm-test-suite` plugin.

Note the `__testReportName__Results` configuration may display resolution errors for external dependencies, as these do not expose test result data.  This is expected behavior and does not affect the outcome of the build or test report aggregation process.
