<!--
  ~ Copyright 2009 the original author or authors.
  ~
  ~ Licensed under the Apache License, Version 2.0 (the "License");
  ~ you may not use this file except in compliance with the License.
  ~ You may obtain a copy of the License at
  ~
  ~      http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing, software
  ~ distributed under the License is distributed on an "AS IS" BASIS,
  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  ~ See the License for the specific language governing permissions and
  ~ limitations under the License.
  -->
<chapter id="working_with_files">
    <title>Working With Files</title>
    <para>
        Most builds work with files. Gradle adds some concepts and APIs to help you achieve this.
    </para>

    <section id="sec:locating_files">
        <title>Locating files</title>
        <para>You can locate a file relative to the project directory using the
            <apilink class="org.gradle.api.Project" method="file(java.lang.Object)"/> method.
        </para>
        <sample id="resolveFile" dir="userguide/files/file" title="Locating files">
            <sourcefile file="build.gradle"/>
        </sample>
        <para>You can pass any object to the <literal>file()</literal> method, and it will attempt to convert the value
            to an absolute <classname>File</classname> object. Usually, you would pass it a
            <classname>String</classname> or <classname>File</classname> instance. If this path is an absolute path, it is used
            to construct a <classname>File</classname> instance. Otherwise, a <classname>File</classname> instance is
            constructed by prepending the project directory path to the supplied path. The <literal>file()</literal>
            method also understands URLs, such as <literal>file:/some/path.xml</literal>.
        </para>
        <para>Using this method is a useful way to convert some user provided value into an absolute <classname>File</classname>.
            It is preferable to using <literal>new File(somePath)</literal>, as <literal>file()</literal> always evaluates
            the supplied path relative to the project directory, which is fixed, rather than the current working
            directory, which can change depending on how the user runs Gradle.
        </para>
    </section>

    <section id="sec:file_collections">
        <title>File collections</title>

        <para>
            A <firstterm>file collection</firstterm> is simply a set of files. It is represented by the
            <apilink class="org.gradle.api.file.FileCollection"/> interface. Many objects in the Gradle API implement
            this interface. For example, <link linkend="sub:configurations">dependency configurations</link> implement
            <literal>FileCollection</literal>.
        </para>

        <para>
            One way to obtain a <literal>FileCollection</literal> instance is to use the
            <apilink class="org.gradle.api.Project" method="files(java.lang.Object...)"/> method. You can pass this method any number of
            objects, which are then converted into a set of <classname>File</classname> objects. The
            <literal>files()</literal> method accepts any type of object as its parameters. These are evaluated relative
            to the project directory, as per the <literal>file()</literal> method, described in <xref linkend="sec:locating_files"/>.
            You can also pass collections, iterables, maps and arrays to the <literal>files()</literal> method. These are flattened
            and the contents converted to <classname>File</classname> instances.
        </para>

        <sample id="fileCollections" dir="userguide/files/fileCollections" title="Creating a file collection">
            <sourcefile file="build.gradle" snippet="simple-params"/>
        </sample>

        <para>A file collection is iterable, and can be converted to a number of other types using the <literal>as</literal>
            operator. You can also add 2 file collections together using the <literal>+</literal> operator, or subtract one
            file collection from another using the <literal>-</literal> operator.
            Here are some examples of what you can do with a file collection.
        </para>
        <sample id="fileCollections" dir="userguide/files/fileCollections" title="Using a file collection">
            <sourcefile file="build.gradle" snippet="usage"/>
            <test args="-q usage"/>
        </sample>

        <para>You can also pass the <literal>files()</literal> method a closure or a <classname>Callable</classname>
            instance. This is called when the contents of the collection are queried, and its return value is converted
            to a set of <classname>File</classname> instances. The return value can be an object of any of the types
            supported by the <literal>files()</literal> method. This is a simple way to 'implement' the
            <classname>FileCollection</classname> interface.
        </para>
        <sample id="fileCollections" dir="userguide/files/fileCollections" title="Implementing a file collection">
            <sourcefile file="build.gradle" snippet="closure"/>
            <output args="-q list"/>
        </sample>

        <para>
            Some other types of things you can pass to <literal>files()</literal>:
        </para>
        <variablelist>
            <varlistentry>
                <term><classname>FileCollection</classname></term>
                <listitem>
                    <para>These are flattened and the contents included in the file collection.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><classname>Task</classname></term>
                <listitem>
                    <para>The output files of the task are included in the file collection.</para>
                </listitem>
            </varlistentry>
            <varlistentry>
                <term><classname>TaskOutputs</classname></term>
                <listitem>
                    <para>The output files of the TaskOutputs are included in the file collection.</para>
                </listitem>
            </varlistentry>
        </variablelist>

        <para>It is important to note that the content of a file collection is evaluated lazily, when it is needed.
            This means you can, for example, create a <literal>FileCollection</literal> that represents files which
            will be created in the future by, say, some task.
        </para>

    </section>

    <section id="sec:file_trees">
        <title>File trees</title>

        <para>
            A <firstterm>file tree</firstterm> is a collection of files arranged in a hierarchy. For example, a file tree
            might represent a directory tree or the contents of a ZIP file. It is represented
            by the <apilink class="org.gradle.api.file.FileTree"/> interface. The <literal>FileTree</literal> interface
            extends <literal>FileCollection</literal>, so you can treat a file tree exactly the same way as you would a
            file collection. Several objects in Gradle implement the <literal>FileTree</literal> interface, such as
            <link linkend="sec:source_sets">source sets</link>.
        </para>

        <para>
            One way to obtain a <literal>FileTree</literal> instance is to use the
            <apilink class="org.gradle.api.Project" method="fileTree(java.util.Map)"/> method.
            This creates a <literal>FileTree</literal> defined with a base directory, and optionally some Ant-style
            include and exclude patterns.
        </para>

        <sample id="fileTrees" dir="userguide/files/fileTrees" title="Creating a file tree">
            <sourcefile file="build.gradle" snippet="define"/>
        </sample>

        <para>You use a file tree in the same way you use a file collection. You can also visit the contents of the
            tree, and select a sub-tree using Ant-style patterns:
        </para>
        <sample id="fileTrees" dir="userguide/files/fileTrees" title="Using a file tree">
            <sourcefile file="build.gradle" snippet="use"/>
        </sample>
    </section>

    <section id="sec:archive_contents">
        <title>Using the contents of an archive as a file tree</title>

        <para>You can use the contents of an archive, such as a ZIP or TAR file, as a file tree. You do this using
            the <apilink class="org.gradle.api.Project" method="zipTree"/> and
            <apilink class="org.gradle.api.Project" method="tarTree(java.lang.Object)"/> methods. These methods return a <literal>FileTree</literal>
            instance which you can use like any other file tree or file collection. For example, you can use it to expand
            the archive by copying the contents, or to merge some archives into another.
        </para>
        <sample id="fileTrees" dir="userguide/files/fileTrees" title="Using an archive as a file tree">
            <sourcefile file="build.gradle" snippet="archive-trees"/>
        </sample>
    </section>

    <section id="sec:specifying_multiple_files">
        <title>Specifying a set of input files</title>
        <para>Many objects in Gradle have properties which accept a set of input files. For example, the
            <apilink class="org.gradle.api.tasks.compile.JavaCompile"/> task has a <literal>source</literal> property,
            which defines the source files to compile. You can set the value of this property using any of the types
            supported by the <link linkend="sec:file_collections">files()</link> method, which was shown above.
            This means you can set the property using, for example, a <classname>File</classname>, <classname>String</classname>,
            collection, <classname>FileCollection</classname> or even a closure.
            Here are some examples:
        </para>
        <sample id="inputFiles" dir="userguide/files/inputFiles" title="Specifying a set of files">
            <sourcefile file="build.gradle" snippet="set-input-files"/>
        </sample>
        <para>Usually, there is a method with the same name as the property, which appends to the set of files. Again,
            this method accepts any of the types supported by the <link linkend="sec:file_collections">files()</link>
            method.
        </para>
        <sample id="inputFiles" dir="userguide/files/inputFiles" title="Specifying a set of files">
            <sourcefile file="build.gradle" snippet="add-input-files"/>
        </sample>
    </section>

    <section id="sec:copying_files">
        <title>Copying files</title>
        <para>You can use the <apilink class="org.gradle.api.tasks.Copy"/> task to copy files. The copy task is very flexible, and allows
            you to, for example, filter the contents of the files as they are copied, and map to the file names.
        </para>
        <para>To use the <literal>Copy</literal> task, you must provide a set of source files to copy, and a destination directory to copy
            the files to. You may also specify how to transform the files as they are copied. You do all this using a
            <firstterm>copy spec</firstterm>. A copy spec is represented by the <apilink class="org.gradle.api.file.CopySpec"/> interface. The
            <literal>Copy</literal> task implements this interface.
            You specify the source files using the <apilink class="org.gradle.api.file.CopySpec" method="from(java.lang.Object...)"/>
            method. To specify the destination directory, use the <apilink class="org.gradle.api.file.CopySpec" method="into(java.lang.Object)"/>
            method.
        </para>
        <sample id="copy" dir="userguide/files/copy" title="Copying files using the copy task">
            <sourcefile file="build.gradle" snippet="copy-task"/>
            <test args="test"/>
        </sample>
        <para>The <literal>from()</literal> method accepts any of the arguments that the
            <link linkend="sec:file_collections">files()</link> method does. When an argument resolves to a directory,
            everything under that directory (but not the directory itself) is recursively copied into the destination
            directory. When an argument resolves to a file, that file is copied into the destination directory.
            When an argument resolves to a non-existing file, that argument is ignored. If the argument is
            a task, the output files (i.e. the files the task creates) of the task are copied and the task is automatically
            added as a dependency of the <literal>Copy</literal> task.
            The <literal>into()</literal> accepts
            any of the arguments that the <link linkend="sec:locating_files">file()</link> method does. Here is another
            example:
        </para>
        <sample id="copy" dir="userguide/files/copy" title="Specifying copy task source files and destination directory">
            <sourcefile file="build.gradle" snippet="copy-task-2"/>
        </sample>
        <para>You can select the files to copy using Ant-style include or exclude patterns, or using a closure:</para>
        <sample id="copy" dir="userguide/files/copy" title="Selecting the files to copy">
            <sourcefile file="build.gradle" snippet="copy-task-with-patterns"/>
        </sample>
        <para>You can also use the <apilink class="org.gradle.api.Project" method="copy(org.gradle.api.Action)"/> method to copy files. It works the
            same way as the task with some major limitations though. First, the <literal>copy()</literal> is not incremental
            (see <xref linkend="sec:up_to_date_checks" />).
        </para>
        <sample id="copy" dir="userguide/files/copy" title="Copying files using the copy() method without up-to-date check">
            <sourcefile file="build.gradle" snippet="copy-method"/>
        </sample>
        <para>Secondly, the <literal>copy()</literal> method can not honor task dependencies when a task is used as a copy source
            (i.e. as an argument to <literal>from()</literal>) because it's a method and not a task.
            As such, if you are using the <literal>copy()</literal> method as part of a task action, you must explicitly
            declare all inputs and outputs in order to get the correct behavior.
        </para>
        <sample id="copy" dir="userguide/files/copy" title="Copying files using the copy() method with up-to-date check">
            <sourcefile file="build.gradle" snippet="copy-method-with-dependency"/>
        </sample>
        <para>
            It is preferable to use the <literal>Copy</literal> task wherever possible, as it supports incremental building and task dependency inference
            without any extra effort on your part. The <literal>copy()</literal> method can be used to copy files as <emphasis>part</emphasis> of a task's
            implementation. That is, the copy method is intended to be used by custom tasks (see <xref linkend="custom_tasks" />) that need to copy files
            as part of their function. In such a scenario, the custom task should sufficiently declare the inputs/outputs relevant to the copy action.
        </para>
        <section id="sec:renaming_files">
            <title>Renaming files</title>
            <sample id="renameOnCopy" dir="userguide/files/copy" title="Renaming files as they are copied">
                <sourcefile file="build.gradle" snippet="rename-files"/>
            </sample>
        </section>
        <section id="sec:filtering_files">
            <title>Filtering files</title>
            <sample id="filterOnCopy" dir="userguide/files/copy" title="Filtering files as they are copied">
                <sourcefile file="build.gradle" snippet="filter-files"/>
            </sample>
            <para>When you use the <literal>ReplaceTokens</literal> class with the “filter” operation, the result is a template engine that replaces tokens of the form “@tokenName@” (the Apache Ant-style token) with a set of given values. The “expand” operation does the same thing except it treats the source files as
                <ulink url="http://docs.groovy-lang.org/latest/html/api/groovy/text/SimpleTemplateEngine.html">Groovy templates</ulink>
                in which tokens take the form “${tokenName}”. Be aware that you may need to escape parts of your source files when
                using this option, for example if it contains literal “$” or “&lt;%” strings.</para>

            <para>It's a good practice to specify the charset when reading and writing the file, using the <literal>filteringCharset</literal>
                property. If not specified, the JVM default charset is used, which might not match with the actual charset of the files
                to filter, and might be different from one machine to another.
            </para>
        </section>
        <section id="sec:using_the_copyspec_class">
            <title>Using the <classname>CopySpec</classname> class</title>
            <para>Copy specs form a hierarchy. A copy spec inherits its destination path, include patterns, exclude patterns, copy actions,
                name mappings and filters.</para>
            <sample id="nestedCopySpecs" dir="userguide/files/copy" title="Nested copy specs">
                <sourcefile file="build.gradle" snippet="nested-specs"/>
            </sample>
        </section>
    </section>

    <section id="sec:sync_task">
        <title>Using the <literal>Sync</literal> task</title>
        <para>The <apilink class="org.gradle.api.tasks.Sync"/> task extends the <literal>Copy</literal> task. When it
            executes, it copies the source files into the destination directory, and then removes any files from the
            destination directory which it did not copy. This can be useful for doing things such as installing your
            application, creating an exploded copy of your archives, or maintaining a copy of the project's dependencies.
        </para>
        <para>Here is an example which maintains a copy of the project's runtime dependencies in the <filename>build/libs</filename>
            directory.</para>
        <sample id="syncDependencies" dir="userguide/files/sync" title="Using the Sync task to copy dependencies">
            <sourcefile file="build.gradle" snippet="copy-dependencies"/>
            <test args="libs"/>
        </sample>
    </section>

    <section id="sec:archives">
        <title>Creating archives</title>
        <para>
            A project can have as many JAR archives as you want. You can also add WAR, ZIP and TAR archives to your project.
            Archives are created using the various archive tasks:
            <apilink class="org.gradle.api.tasks.bundling.Zip"/>,
            <apilink class="org.gradle.api.tasks.bundling.Tar"/>,
            <apilink class="org.gradle.api.tasks.bundling.Jar"/>,
            <apilink class="org.gradle.api.tasks.bundling.War"/>, and
            <apilink class="org.gradle.plugins.ear.Ear"/>.
            They all work the same way, so let's look at how you create a ZIP file.
        </para>
        <sample id="createZip" dir="userguide/files/archives" title="Creating a ZIP archive">
            <sourcefile file="build.gradle" snippet="zip"/>
        </sample>
        <tip>
            <title>Why are you using the Java plugin?</title>
            <para>The Java plugin adds a number of default values for the archive tasks. You can use the archive
                tasks without using the Java plugin, if you like. You will need to provide values for some additional
                properties.
            </para>
        </tip>

        <para>
            The archive tasks all work exactly the same way as the <literal>Copy</literal> task, and implement the same
            <classname>CopySpec</classname> interface. As with the <literal>Copy</literal> task, you specify the input
            files using the <literal>from()</literal> method, and can optionally specify where they end up in the
            archive using the <literal>into()</literal> method. You can filter the contents of file, rename files, and
            all the other things you can do with a copy spec.
        </para>

        <section id="sec:archive_naming">
            <title>Archive naming</title>

            <para>The format of <filename><replaceable>projectName</replaceable>-<replaceable>version</replaceable>.<replaceable>type</replaceable></filename>
                is used for generated archive file names.
                For example:
            </para>
            <sample id="archiveNaming" dir="userguide/files/archiveNaming" title="Creation of ZIP archive">
                <sourcefile file="build.gradle"/>
                <output args="-q myZip"/>
            </sample>

            <para>This adds a <classname>Zip</classname> archive task with the name <literal>myZip</literal> which produces
                ZIP file <filename>zipProject-1.0.zip</filename>. It is important to distinguish between the name of the archive task
                and the name of the archive generated by the archive task. The default name for archives can be
                changed with the <literal>archivesBaseName</literal> project property. The name of the archive can also be
                changed at any time later on.</para>

            <para>There are a number of properties which you can set on an archive task. These are listed below in <xref linkend="archiveTasksNamingProperties"/>.
                You can, for example, change the name of the archive:
            </para>
            <sample id="zipWithCustomName" dir="userguide/tutorial/zipWithCustomName" title="Configuration of archive task - custom archive name">
                <sourcefile file="build.gradle"/>
                <output args="-q myZip"/>
            </sample>
            <para>You can further customize the archive names:</para>
            <sample id="zipWithArguments" dir="userguide/tutorial/zipWithArguments" title="Configuration of archive task - appendix &amp; classifier">
                <sourcefile file="build.gradle"/>
                <output args="-q myZip"/>
            </sample>

            <table id="archiveTasksNamingProperties">
                <title>Archive tasks - naming properties</title>
                <thead>
                    <tr>
                        <td>Property name</td>
                        <td>Type</td>
                        <td>Default value</td>
                        <td>Description</td>
                    </tr>
                </thead>
                <tr>
                    <td><literal>archiveName</literal></td>
                    <td><classname>String</classname></td>
                    <td>
                        <filename><replaceable>baseName</replaceable>-<replaceable>appendix</replaceable>-<replaceable>version</replaceable>-<replaceable>classifier</replaceable>.<replaceable>extension</replaceable></filename>
                        <para>If any of these properties is empty the trailing <filename>-</filename> is not added to the name.</para>
                    </td>
                    <td>The base file name of the generated archive</td>
                </tr>
                <tr>
                    <td><literal>archivePath</literal></td>
                    <td><classname>File</classname></td>
                    <td><filename><replaceable>destinationDir</replaceable>/<replaceable>archiveName</replaceable></filename></td>
                    <td>The absolute path of the generated archive.</td>
                </tr>
                <tr>
                    <td><literal>destinationDir</literal></td>
                    <td><classname>File</classname></td>
                    <td>Depends on the archive type. JARs and WARs go into <filename><replaceable>project.buildDir</replaceable>/libraries</filename>.
                        ZIPs and TARs go into <filename><replaceable>project.buildDir</replaceable>/distributions</filename>.
                    </td>
                    <td>The directory to generate the archive into</td>
                </tr>
                <tr>
                    <td><literal>baseName</literal></td>
                    <td><classname>String</classname></td>
                    <td><filename><replaceable>project.name</replaceable></filename></td>
                    <td>The base name portion of the archive file name.</td>
                </tr>
                <tr>
                    <td><literal>appendix</literal></td>
                    <td><classname>String</classname></td>
                    <td><literal>null</literal></td>
                    <td>The appendix portion of the archive file name.</td>
                </tr>
                <tr>
                    <td><literal>version</literal></td>
                    <td><classname>String</classname></td>
                    <td><filename><replaceable>project.version</replaceable></filename></td>
                    <td>The version portion of the archive file name.</td>
                </tr>
                <tr>
                    <td><literal>classifier</literal></td>
                    <td><classname>String</classname></td>
                    <td><literal>null</literal></td>
                    <td>The classifier portion of the archive file name,</td>
                </tr>
                <tr>
                    <td><literal>extension</literal></td>
                    <td><classname>String</classname></td>
                    <td>Depends on the archive type, and for TAR files, the compression type as well: <filename>zip</filename>, <filename>jar</filename>,
                        <filename>war</filename>, <filename>tar</filename>, <filename>tgz</filename> or <filename>tbz2</filename>.</td>
                    <td>The extension of the archive file name.</td>
                </tr>
            </table>
        </section>

        <section id="sec:sharing_content_between_multiple_archives">
            <title>Sharing content between multiple archives</title>
            <para>You can use the <apilink class="org.gradle.api.Project" method="copySpec(org.gradle.api.Action)"/> method to share content between archives.</para>
        </section>

        <section id="sec:reproducible_archives">
            <title>Reproducible archives</title>
            <para>Sometimes it can be desirable to recreate archives in a byte for byte way on different machines.
                You want to be sure that building an artifact from source code produces the same result, byte for byte, no matter when and where it is built.
                This is necessary for projects like <ulink url="https://reproducible-builds.org/">reproducible-builds.org</ulink>.
            </para>
            <para>
                Reproducing the same archive byte for byte poses some challenges since the order of the files in an archive is influenced by the underlying filesystem.
                Each time a zip, tar, jar, war or ear is built from source, the order of the files inside the archive may change.
                Files that only have a different timestamp also causes archives to be slightly different between builds.
                All <apilink class="org.gradle.api.tasks.bundling.AbstractArchiveTask"/> (e.g. Jar, Zip) tasks shipped with Gradle include <link linkend="feature_lifecycle">incubating</link> support producing reproducible archives.
            </para>
            <para>
                For example, to make a <literal>Zip</literal> task reproducible you need to set
                <apilink class="org.gradle.api.tasks.bundling.Zip" method="isReproducibleFileOrder()"/> to <literal>true</literal>
                and <apilink class="org.gradle.api.tasks.bundling.Zip" method="isPreserveFileTimestamps()"/> to <literal>false</literal>.
                In order to make all archive tasks in your build reproducible, consider adding the following configuration to your build file:
            </para>
            <sample id="createZip" dir="userguide/files/archives" title="Activating reproducible archives">
                <sourcefile file="build.gradle" snippet="reproducible"/>
            </sample>
        </section>

        <para>Often you will want to publish an archive, so that it is usable from another project. This process is
            described in <xref linkend="artifact_management"/>
        </para>

    </section>

    <section id="sec:properties_files">
        <title>Properties files</title>

        <para>Properties files are used in many places during Java development. Gradle makes it easy to create properties files as a normal part of the build.
            You can use the <apilink class="org.gradle.api.tasks.WriteProperties"/> task to create properties files.
        </para>

        <para>The <literal>WriteProperties</literal> task also fixes a well-known problem with <literal>Properties.store()</literal>
            that can reduce the usefulness of incremental builds (see <xref linkend="sec:up_to_date_checks" />). The standard Java way to write a properties
            file produces a unique file every time, even when the same properties and values are used, because it includes a timestamp in the comments.
            Gradle's <literal>WriteProperties</literal> task generates exactly the same output byte-for-byte if none of the properties have changed.
            This is achieved by a few tweaks to how a properties file is generated:
        </para>
        <itemizedlist>
            <listitem>no timestamp comment is added to the output</listitem>
            <listitem>the line separator is system independent, but can be configured explicitly (it defaults to <literal>'\n'</literal>)</listitem>
            <listitem>the properties are sorted alphabetically</listitem>
        </itemizedlist>
    </section>
</chapter>
