# Advanced Tasks

version 9.3.0

## Incremental tasks

In Gradle, implementing a task that skips execution when its inputs and outputs are already `UP-TO-DATE` is simple and efficient, thanks to the Incremental Build feature.

However, there are times when only a few input files have changed since the last execution, and it is best to avoid reprocessing all the unchanged inputs. This situation is common in tasks that transform input files into output files on a one-to-one basis.

To optimize your build process you can use an incremental task. This approach ensures that only out-of-date input files are processed, improving build performance.

### Implementing an incremental task

For a task to process inputs incrementally, that task must contain an incremental task action.

This is a task action method that has a single InputChanges parameter. That parameter tells Gradle that the action only wants to process the changed inputs.

In addition, the task needs to declare at least one incremental file input property by using either `@Incremental` or `@SkipWhenEmpty`:

build.gradle.kts

    public class IncrementalReverseTask : DefaultTask() {

        @get:Incremental
        @get:InputDirectory
        val inputDir: DirectoryProperty = project.objects.directoryProperty()

        @get:OutputDirectory
        val outputDir: DirectoryProperty = project.objects.directoryProperty()

        @get:Input
        val inputProperty: RegularFileProperty = project.objects.fileProperty() // File input property

        @TaskAction
        fun execute(inputs: InputChanges) { // InputChanges parameter
            val msg = if (inputs.isIncremental) "CHANGED inputs are out of date"
                      else "ALL inputs are out of date"
            println(msg)
        }
    }

build.gradle

    class IncrementalReverseTask extends DefaultTask {

        @Incremental
        @InputDirectory
        def File inputDir

        @OutputDirectory
        def File outputDir

        @Input
        def inputProperty // File input property

        @TaskAction
        void execute(InputChanges inputs) { // InputChanges parameter
            println inputs.incremental ? "CHANGED inputs are out of date"
                                       : "ALL inputs are out of date"
        }
    }

| To query incremental changes for an input file property, that property must always return the same instance. The easiest way to accomplish this is to use one of the following property types: `RegularFileProperty`, `DirectoryProperty` or `ConfigurableFileCollection`. You can learn more about `RegularFileProperty` and `DirectoryProperty` in Lazy Configuration.

The incremental task action can use `InputChanges.getFileChanges()` to find out what files have changed for a given file-based input property, be it of type `RegularFileProperty`, `DirectoryProperty` or `ConfigurableFileCollection`.

The method returns an `Iterable` of type FileChanges, which in turn can be queried for the following:

- the affected file

- the change type (`ADDED`, `REMOVED` or `MODIFIED`)

- the normalized path of the changed file

- the file type of the changed file

The following example demonstrates an incremental task that has a directory input. It assumes that the directory contains a collection of text files and copies them to an output directory, reversing the text within each file:

build.gradle.kts

    abstract class IncrementalReverseTask : DefaultTask() {
        @get:Incremental
        @get:PathSensitive(PathSensitivity.NAME_ONLY)
        @get:InputDirectory
        abstract val inputDir: DirectoryProperty

        @get:OutputDirectory
        abstract val outputDir: DirectoryProperty

        @get:Input
        abstract val inputProperty: Property<String>

        @TaskAction
        fun execute(inputChanges: InputChanges) {
            println(
                if (inputChanges.isIncremental) "Executing incrementally"
                else "Executing non-incrementally"
            )

            inputChanges.getFileChanges(inputDir).forEach { change ->
                if (change.fileType == FileType.DIRECTORY) return@forEach

                println("${change.changeType}: ${change.normalizedPath}")
                val targetFile = outputDir.file(change.normalizedPath).get().asFile
                if (change.changeType == ChangeType.REMOVED) {
                    targetFile.delete()
                } else {
                    targetFile.writeText(change.file.readText().reversed())
                }
            }
        }
    }

build.gradle

    abstract class IncrementalReverseTask extends DefaultTask {
        @Incremental
        @PathSensitive(PathSensitivity.NAME_ONLY)
        @InputDirectory
        abstract DirectoryProperty getInputDir()

        @OutputDirectory
        abstract DirectoryProperty getOutputDir()

        @Input
        abstract Property<String> getInputProperty()

        @TaskAction
        void execute(InputChanges inputChanges) {
            println(inputChanges.incremental
                ? 'Executing incrementally'
                : 'Executing non-incrementally'
            )

            inputChanges.getFileChanges(inputDir).each { change ->
                if (change.fileType == FileType.DIRECTORY) return

                println "${change.changeType}: ${change.normalizedPath}"
                def targetFile = outputDir.file(change.normalizedPath).get().asFile
                if (change.changeType == ChangeType.REMOVED) {
                    targetFile.delete()
                } else {
                    targetFile.text = change.file.text.reverse()
                }
            }
        }
    }

| The type of the `inputDir` property, its annotations, and the `execute()` action use `getFileChanges()` to process the subset of files that have changed since the last build. The action deletes a target file if the corresponding input file has been removed.

If, for some reason, the task is executed non-incrementally (by running with `--rerun-tasks`, for example), all files are reported as `ADDED`, irrespective of the previous state. In this case, Gradle automatically removes the previous outputs, so the incremental task must only process the given files.

For a simple transformer task like the above example, the task action must generate output files for any out-of-date inputs and delete output files for any removed inputs.

| A task may only contain a single incremental task action.

### Which inputs are considered out of date?

When a task has been previously executed, and the only changes since that execution are to incremental input file properties, Gradle can intelligently determine which input files need to be processed, a concept known as incremental execution.

In this scenario, the `InputChanges.getFileChanges()` method, available in the `org.gradle.work.InputChanges` class, provides details for all input files associated with the given property that have been `ADDED`, `REMOVED` or `MODIFIED`.

However, there are many cases where Gradle cannot determine which input files need to be processed (i.e., non-incremental execution). Examples include:

- There is no history available from a previous execution.

- You are building with a different version of Gradle. Currently, Gradle does not use task history from a different version.

- An `upToDateWhen` criterion added to the task returns `false`.

- An input property has changed since the previous execution.

- A non-incremental input file property has changed since the previous execution.

- One or more output files have changed since the previous execution.

In these cases, Gradle will report all input files as `ADDED`, and the `getFileChanges()` method will return details for all the files that comprise the given input property.

You can check if the task execution is incremental or not with the `InputChanges.isIncremental()` method.

### An incremental task in action

Consider an instance of `IncrementalReverseTask` executed against a set of inputs for the first time.

In this case, all inputs will be considered `ADDED`, as shown here:

build.gradle.kts

    tasks.register<IncrementalReverseTask>("incrementalReverse") {
        inputDir = file("inputs")
        outputDir = layout.buildDirectory.dir("outputs")
        inputProperty = project.findProperty("taskInputProperty") as String? ?: "original"
    }

build.gradle

    tasks.register('incrementalReverse', IncrementalReverseTask) {
        inputDir = file('inputs')
        outputDir = layout.buildDirectory.dir("outputs")
        inputProperty = project.properties['taskInputProperty'] ?: 'original'
    }

The build layout:

    .
    ├── build.gradle
    └── inputs
        ├── 1.txt
        ├── 2.txt
        └── 3.txt

    $ gradle -q incrementalReverse
    Executing non-incrementally
    ADDED: 1.txt
    ADDED: 2.txt
    ADDED: 3.txt

Naturally, when the task is executed again with no changes, then the entire task is `UP-TO-DATE`, and the task action is not executed:

    $ gradle incrementalReverse
    > Task :incrementalReverse UP-TO-DATE

    BUILD SUCCESSFUL in 0s
    1 actionable task: 1 up-to-date

When an input file is modified in some way or a new input file is added, then re-executing the task results in those files being returned by `InputChanges.getFileChanges()`.

The following example modifies the content of one file and adds another before running the incremental task:

build.gradle.kts

    tasks.register("updateInputs") {
        val inputsDir = layout.projectDirectory.dir("inputs")
        outputs.dir(inputsDir)
        doLast {
            inputsDir.file("1.txt").asFile.writeText("Changed content for existing file 1.")
            inputsDir.file("4.txt").asFile.writeText("Content for new file 4.")
        }
    }

build.gradle

    tasks.register('updateInputs') {
        def inputsDir = layout.projectDirectory.dir('inputs')
        outputs.dir(inputsDir)
        doLast {
            inputsDir.file('1.txt').asFile.text = 'Changed content for existing file 1.'
            inputsDir.file('4.txt').asFile.text = 'Content for new file 4.'
        }
    }

    $ gradle -q updateInputs incrementalReverse
    Executing incrementally
    MODIFIED: 1.txt
    ADDED: 4.txt

| The various mutation tasks (`updateInputs`, `removeInput`, etc) are only present to demonstrate the behavior of incremental tasks. They should not be viewed as the kinds of tasks or task implementations you should have in your own build scripts.

When an existing input file is removed, then re-executing the task results in that file being returned by `InputChanges.getFileChanges()` as `REMOVED`.

The following example removes one of the existing files before executing the incremental task:

build.gradle.kts

    tasks.register<Delete>("removeInput") {
        delete("inputs/3.txt")
    }

build.gradle

    tasks.register('removeInput', Delete) {
        delete 'inputs/3.txt'
    }

    $ gradle -q removeInput incrementalReverse
    Executing incrementally
    REMOVED: 3.txt

Gradle cannot determine which input files are out-of-date when an output file is deleted (or modified). In this case, details for all the input files for the given property are returned by `InputChanges.getFileChanges()`.

The following example removes one of the output files from the build directory. However, all the input files are considered to be `ADDED`:

build.gradle.kts

    tasks.register<Delete>("removeOutput") {
        delete(layout.buildDirectory.file("outputs/1.txt"))
    }

build.gradle

    tasks.register('removeOutput', Delete) {
        delete layout.buildDirectory.file("outputs/1.txt")
    }

    $ gradle -q removeOutput incrementalReverse
    Executing non-incrementally
    ADDED: 1.txt
    ADDED: 2.txt
    ADDED: 3.txt

The last scenario we want to cover concerns what happens when a non-file-based input property is modified. In such cases, Gradle cannot determine how the property impacts the task outputs, so the task is executed non-incrementally. This means that all input files for the given property are returned by `InputChanges.getFileChanges()` and they are all treated as `ADDED`.

The following example sets the project property `taskInputProperty` to a new value when running the `incrementalReverse` task. That project property is used to initialize the task’s `inputProperty` property, as you can see in the first example of this section.

Here is the expected output in this case:

    $ gradle -q -PtaskInputProperty=changed incrementalReverse
    Executing non-incrementally
    ADDED: 1.txt
    ADDED: 2.txt
    ADDED: 3.txt

## Command Line options

Sometimes, a user wants to declare the value of an exposed task property on the command line instead of the build script. Passing property values on the command line is particularly helpful if they change more frequently.

The task API supports a mechanism for marking a property to automatically generate a corresponding command line parameter with a specific name at runtime.

### Step 1. Declare a command-line option

To expose a new command line option for a task property, annotate the corresponding setter method of a property with Option:

    @Option(option = "flag", description = "Sets the flag")

An option requires a mandatory identifier. You can provide an optional description.

A task can expose as many command line options as properties available in the class.

Options may be declared in superinterfaces of the task class as well. If multiple interfaces declare the same property but with different option flags, they will both work to set the property.

In the example below, the custom task `UrlVerify` verifies whether a URL can be resolved by making an HTTP call and checking the response code. The URL to be verified is configurable through the property `url`. The setter method for the property is annotated with @Option:

UrlVerify.java

    import org.gradle.api.tasks.options.Option;

    public class UrlVerify extends DefaultTask {
        private String url;

        @Option(option = "url", description = "Configures the URL to be verified.")
        public void setUrl(String url) {
            this.url = url;
        }

        @Input
        public String getUrl() {
            return url;
        }

        @TaskAction
        public void verify() {
            getLogger().quiet("Verifying URL '{}'", url);

            // verify URL by making a HTTP call
        }
    }

All options declared for a task can be rendered as console output by running the `help` task and the `--task` option.

### Step 2. Use an option on the command line

There are a few rules for options on the command line:

- The option uses a double-dash as a prefix, e.g., `--url`. A single dash does not qualify as valid syntax for a task option.

- The option argument follows directly after the task declaration, e.g., `verifyUrl --url=http://www.google.com/`.

- Multiple task options can be declared in any order on the command line following the task name.

Building upon the earlier example, the build script creates a task instance of type `UrlVerify` and provides a value from the command line through the exposed option:

build.gradle.kts

    tasks.register<UrlVerify>("verifyUrl")

build.gradle

    tasks.register('verifyUrl', UrlVerify)

    $ gradle -q verifyUrl --url=http://www.google.com/
    Verifying URL 'http://www.google.com/'

### Supported data types for options

Gradle limits the data types that can be used for declaring command line options.

The use of the command line differs per type:

`boolean`, `Boolean`, `Property<Boolean>`

  Describes an option with the value `true` or `false`.
  Passing the option on the command line treats the value as `true`. For example, `--foo` equates to `true`.
  The absence of the option uses the default value of the property. For each boolean option, an opposite option is created automatically. For example, `--no-foo` is created for the provided option `--foo` and `--bar` is created for `--no-bar`. Options whose name starts with `--no` are disabled options and set the option value to `false`. An opposite option is only created if no option with the same name already exists for the task.

`Double`, `Property<Double>`

  Describes an option with a double value.
  Passing the option on the command line also requires a value, e.g., `--factor=2.2` or `--factor 2.2`.

`Integer`, `Property<Integer>`

  Describes an option with an integer value.
  Passing the option on the command line also requires a value, e.g., `--network-timeout=5000` or `--network-timeout 5000`.

`Long`, `Property<Long>`

  Describes an option with a long value.
  Passing the option on the command line also requires a value, e.g., `--threshold=2147483648` or `--threshold 2147483648`.

`String`, `Property<String>`

  Describes an option with an arbitrary String value.
  Passing the option on the command line also requires a value, e.g., `--container-id=2x94held` or `--container-id 2x94held`.

`enum`, `Property<enum>`

  Describes an option as an enumerated type.
  Passing the option on the command line also requires a value e.g., `--log-level=DEBUG` or `--log-level debug`.
  The value is not case-sensitive.

`List<T>` where `T` is `Double`, `Integer`, `Long`, `String`, `enum`

  Describes an option that can take multiple values of a given type.
  The values for the option have to be provided as multiple declarations, e.g., `--image-id=123 --image-id=456`.
  Other notations, such as comma-separated lists or multiple values separated by a space character, are currently not supported.

`ListProperty<T>`, `SetProperty<T>` where `T` is `Double`, `Integer`, `Long`, `String`, `enum`

  Describes an option that can take multiple values of a given type.
  The values for the option have to be provided as multiple declarations, e.g., `--image-id=123 --image-id=456`.
  Other notations, such as comma-separated lists or multiple values separated by a space character, are currently not supported.

`DirectoryProperty`, `RegularFileProperty`

  Describes an option with a file system element.
  Passing the option on the command line also requires a value representing a path, e.g., `--output-file=file.txt` or `--output-dir outputDir`.
  Relative paths are resolved relative to the project directory of the project that owns this property instance. See `FileSystemLocationProperty.set()`.

### Documenting available values for an option

Theoretically, an option for a property type `String` or `List<String>` can accept any arbitrary value. Accepted values for such an option can be documented programmatically with the help of the annotation OptionValues:

    @OptionValues('file')

This annotation may be assigned to any method that returns a `List` of one of the supported data types. You need to specify an option identifier to indicate the relationship between the option and available values.

| Passing a value on the command line not supported by the option does not fail the build or throw an exception. You must implement custom logic for such behavior in the task action.

The example below demonstrates the use of multiple options for a single task. The task implementation provides a list of available values for the option `output-type`:

UrlProcess.java

    import org.gradle.api.tasks.options.Option;
    import org.gradle.api.tasks.options.OptionValues;

    public abstract class UrlProcess extends DefaultTask {
        private String url;
        private OutputType outputType;

        @Input
        @Option(option = "http", description = "Configures the http protocol to be allowed.")
        public abstract Property<Boolean> getHttp();

        @Option(option = "url", description = "Configures the URL to send the request to.")
        public void setUrl(String url) {
            if (!getHttp().getOrElse(true) && url.startsWith("http://")) {
                throw new IllegalArgumentException("HTTP is not allowed");
            } else {
                this.url = url;
            }
        }

        @Input
        public String getUrl() {
            return url;
        }

        @Option(option = "output-type", description = "Configures the output type.")
        public void setOutputType(OutputType outputType) {
            this.outputType = outputType;
        }

        @OptionValues("output-type")
        public List<OutputType> getAvailableOutputTypes() {
            return new ArrayList<OutputType>(Arrays.asList(OutputType.values()));
        }

        @Input
        public OutputType getOutputType() {
            return outputType;
        }

        @TaskAction
        public void process() {
            getLogger().quiet("Writing out the URL response from '{}' to '{}'", url, outputType);

            // retrieve content from URL and write to output
        }

        private static enum OutputType {
            CONSOLE, FILE
        }
    }

### Listing command line options

Command line options using the annotations Option and OptionValues are self-documenting.

You will see declared options and their available values reflected in the console output of the `help` task. The output renders options alphabetically, except for boolean disable options, which appear following the enable option:

    $ gradle -q help --task processUrl
    Detailed task information for processUrl

    Path
         :processUrl

    Type
         UrlProcess (UrlProcess)

    Options
         --http     Configures the http protocol to be allowed.

         --no-http     Disables option --http.

         --output-type     Configures the output type.
                           Available values are:
                                CONSOLE
                                FILE

         --url     Configures the URL to send the request to.

         --rerun     Causes the task to be re-run even if up-to-date.

    Description
         -

    Group
         -

### Limitations

Support for declaring command line options currently comes with a few limitations.

- Command line options can only be declared for custom tasks via annotation. There’s no programmatic equivalent for defining options.

- Options cannot be declared globally, e.g., on a project level or as part of a plugin.

- When assigning an option on the command line, the task exposing the option needs to be spelled out explicitly, e.g., `gradle check --tests abc` does not work even though the `check` task depends on the `test` task.

- If you specify a task option name that conflicts with the name of a built-in Gradle option, use the `--` delimiter before calling your task to reference that option. For more information, see Disambiguate Task Options from Built-in Options.

## Verification failures

Normally, exceptions thrown during task execution result in a failure that immediately terminates a build. The outcome of the task will be `FAILED`, the result of the build will be `FAILED`, and no further tasks will be executed. When running with the `--continue` flag, Gradle will continue to run other requested tasks in the build after encountering a task failure. However, any tasks that depend on a failed task will not be executed.

There is a special type of exception that behaves differently when downstream tasks only rely on the outputs of a failing task. A task can throw a subtype of VerificationException to indicate that it has failed in a controlled manner such that its output is still valid for consumers. A task depends on the outcome of another task when it directly depends on it using `dependsOn`. When Gradle is run with `--continue`, consumer tasks that depend on a producer task’s output (via a relationship between task inputs and outputs) can still run after the producer fails.

A failed unit test, for instance, will cause a failing outcome for the test task. However, this doesn’t prevent another task from reading and processing the (valid) test results the task produced. Verification failures are used in exactly this manner by the `Test Report Aggregation Plugin`.

Verification failures are also useful for tasks that need to report a failure even after producing useful output consumable by other tasks.

build.gradle.kts

    val process = tasks.register("process") {
        val outputFile = layout.buildDirectory.file("processed.log")
        outputs.files(outputFile) (1)

        doLast {
            val logFile = outputFile.get().asFile
            logFile.appendText("Step 1 Complete.") (2)
            throw VerificationException("Process failed!") (3)
            logFile.appendText("Step 2 Complete.") (4)
        }
    }

    tasks.register("postProcess") {
        inputs.files(process) (5)

        doLast {
            println("Results: ${inputs.files.singleFile.readText()}") (6)
        }
    }

build.gradle

    tasks.register("process") {
        def outputFile = layout.buildDirectory.file("processed.log")
        outputs.files(outputFile) (1)

        doLast {
            def logFile = outputFile.get().asFile
            logFile << "Step 1 Complete." (2)
            throw new VerificationException("Process failed!") (3)
            logFile << "Step 2 Complete." (4)
        }
    }

    tasks.register("postProcess") {
        inputs.files(tasks.named("process")) (5)

        doLast {
            println("Results: ${inputs.files.singleFile.text}") (6)
        }
    }

    $ gradle postProcess --continue
    > Task :process FAILED

    > Task :postProcess
    Results: Step 1 Complete.
    2 actionable tasks: 2 executed

    FAILURE: Build failed with an exception.

1 | Register Output: The `process` task writes its output to a log file.
2 | Modify Output: The task writes to its output file as it executes.
3 | Task Failure: The task throws a `VerificationException` and fails at this point.
4 | Continue to Modify Output: This line never runs due to the exception stopping the task.
5 | Consume Output: The `postProcess` task depends on the output of the `process` task due to using that task’s outputs as its own inputs.
6 | Use Partial Result: With the `--continue` flag set, Gradle still runs the requested `postProcess` task despite the `process` task’s failure. `postProcess` can read and display the partial (though still valid) result.