Package com.android.builder.merge

package com.android.builder.merge
The merge package provides the implementation of an incremental merger. The merger reads files from one or more input trees and produces one output tree. The merger produces perfect incremental outputs in the case of incremental input changes. It supports inputs from multiple sources (file system directories and zip files) and supports outputs into multiple forms (file system directories and zip files).

The merger is split into three main concepts discussed in the sections below: the merger, inputs and output.

Main Concepts and Merge Invariant

A fundamental concept in merging is a relative tree. A relative tree is a set of RelativeFile with the restriction that each relative file is uniquely identified by the OS-independent path of the relative file (see RelativeFile.getRelativePath().

Conceptually, a relative tree can be seen as a file system tree. However, in practice, files can come from multiple directories and zips, as long as there are no clashes in OS-independent paths. So, for example, directories x and y in the example below structures could be used to build a relative tree: 

 +---.home
 +---.user
 |   +---.x
 |   |   +---.foo
 |   |   +---.bar
 |   |   +---.subdir
 |   |   |   +---.file1
 |   +---.y
 |   |   +---.extra
 |   |   +---.subdir
 |   |   |   +---.file2
 |   |   |   +---.file3
 |   |   +---.additional

The relative tree would contain files with the following relative paths: 

 foo           -> x /foo
 bar           -> x /bar
 subdir/file1  -> x /subdir/file1
 extra         -> y /extra
 subdir/file2  -> y /subdir/file2
 subdir/file3  -> y /subdir/file3
 additional    -> y /additional

The purpose of the incremental merger is to maintain the merge invariant. The merge invariant is the relation between an ordered list of input relative trees and a single output relative tree. The invariant is expressed as:

  • The set of OS-independent paths in the output relative tree is the union of the set of OS-independent paths in the input relative trees.
  • The contents of each file in the output relative tree is computed by a function based on the list of files in the input relative tree that have the same OS-independent path.

Now consider the following relative tree obtained from zip file z.zip: 

 foo           -> z.zip /foo
 bar1          -> z.zip /bar1
 bar2          -> z.zip /bar2
 sub/sub/deep  -> z.zip /sub/sub/deep
 subdir/file1  -> z.zip /subdir/file1

The only output relative tree that verifies the merge invariant using the previous two input files in order is: 

 foo           -> f(x /foo, z.zip /foo)
 bar           -> f(x /bar)
 bar1          -> f(z.zip /bar1)
 bar2          -> f(z.zip /bar2)
 extra         -> f(y /extra)
 additional    -> f(y /additional)
 subdir/file1  -> f(x /subdir/file1, z.zip /subdir/file1)
 subdir/file2  -> f(y /subdir/file2)
 subdir/file3  -> f(y /subdir/file3)
 sub/sub/deep  -> f(z.zip /sub/sub/deep)

Where f() is the computation function for the merge function.

Incremental Merging and State

The description in the previous section shows how the merge works as a relation between inputs and outputs, but does not describe how the output is actually computed.

A key idea behind the incremental merger is that it is perfectly incremental, that is, if an input relative tree changes, the output relative tree changes only the minimum necessary. Additionally, no useless operations such as rewriting files that don't change are done.

To achieve these goals, the incremental merger only does incremental merges. A full merge is an incremental merge from scratch. The incremental merger maintains knowledge of which files exist in the output and directs the output to change with respect to changed inputs. This has some important implications:

  • Incremental merge will never clean the output.
  • The inputs must be able to tell which changes to relative files have been made.
  • Intermediate state must be saved between merges.

Also, because the merge output function may be dependent on the order of the inputs, each input relative tree is named so, if the order of the inputs changes, the output may need to change even if no files were actually changed.

API

The merge algorithm can be invoked in com.android.builder.merge.IncrementalFileMerger#merge(com.google.common.collect.ImmutableList, com.android.builder.merge.IncrementalFileMergerOutput, com.android.builder.merge.IncrementalFileMergerState). This algorithm requires the previous state of the merge. In a full merge (no previous state) an empty state can be created (see IncrementalFileMergerState.

To invoke the merger, it is necessary to provide the set of inputs and an output. See IncrementalFileMergerInput for a discussion on inputs. See IncrementalFileMergerOutput for a discussion on outputs.