The AGP Upgrade Assistant is a collection of infrastructure, user interfaces and somewhat reusable components for the specific purpose of updating projects consistently in response to a user request to upgrade the version of the Android Gradle Plugin used in that project.
The AGP Upgrade Assistant was introduced in Android Studio 4.2, extending or substantially replacing existing infrastructure which special-cased three actions to take:
The introduction of the AGP Upgrade Assistant in Android Studio 4.2 allowed users to modify their build files if necessary to account for the switch of default in AGP to use Java 8 (rather than the historic default of Java 7), as well as to recommend modifying dependency configurations from the deprecated compile to whichever of api and implementation was appropriate, and to support migration of projects from fabric to Firebase Crashlytics. Without the AGP Upgrade Assistant, each of these steps (or operations like them) would have had to be done manually, prompted by a failure of the project to sync, build, or function correctly once built or deployed.
Since its initial introduction, the AGP Upgrade Assistant has developed in functionality, now offering features such as:
For a variety of reasons, some of which are detailed below, the AGP Upgrade Assistant can not guarantee to successfully update all Android projects, no matter how limited the definition of success might be. For this reason, the Upgrade Assistant's operation has to be as tolerant of failure, and as helpful to the user in the face of failure, as possible.
A build as managed by Gradle is configured, in general, using code in languages which are Turing complete. In order to make even partial success in updating projects achievable, we make no attempt in the AGP Upgrade Assistant to handle non-declarative build configuration beyond a few special cases such as appending constant data to lists and maps. We do not at present attempt to handle
and uses of the AGP Upgrade Assistant in projects making substantial use of these for build configuration will probably fail.
For code in build files and in buildSrc, we could in principle at least detect the use of deprecated APIs, and block upgrades until those usages were removed. (example: our attempt to do this with the removal of the Transform API in AGP 8.0, and similarly the old Variant API in future).
We recognize that there is a significant constituency of users who would like to believe that a tool such as the AGP Upgrade Assistant might Just Work, and we intentionally do not disallow the user from trying other upgrade paths (such as a single-step upgrade-to-latest) even if we are less than optimistic about the chance that everything will operate correctly after such an upgrade. Thus, even if we detect that a user is requesting an upgrade which we consider unlikely to succeed, we allow the user to execute it anyway: it might get them closer. However, if we can prove that the upgrade selected by the user cannot possibly succeed, or that we do not have enough information to be able to make that determination, we do have a mechanism for blocking an upgrade
The AGP Upgrade Assistant assumes that:
After an upgrade, we deem the upgrade successful if the project successfully syncs (imports, loads into the IDE) without additional user intervention.
In particular, we make no analogous statement about the project successfully building, deploying, or passing any accompanying tests; while we hope that changes to allow the project to sync will tend to increase the chance that the project will build and deploy successfully, we recognize that in the presence of semantic changes in the Android Gradle Plugin, and version changes of Gradle itself and other gradle plugins, can lead to differences in the build.
We impose a single “time” variable, based on the ordering of versions of the Android Gradle plugin.
Many other relevant software artifacts also have versions that might have to be examined or changed in order to effect an upgrade, such as Gradle itself, other Gradle plugins, or environmental software such as the JDK used to run Gradle. Changes or thresholds in those versions are mapped, directly or indirectly, to the ordering defined by Android Gradle plugin versions; we make no attempt to model the complete space of possible upgrade states and outcomes, instead encoding required versions of those artifacts for any given Android Gradle plugin version, and assuming that if the project is using a later version of those artifacts pre-upgrade than the system thinks are required post-upgrade, they continue to be valid. (One consequence of this is that we do not attempt to lower the versions of any software used by a project.)
AGP's versioning scheme is partly-Semantic, in the sense that major changes in behaviour are reserved for major version number increments, while patch-level increments (after the .0 of any release series) are intended to include safe bug-fix changes only.
This implies that some upgrades involve less risk than others. Upgrades to newer patch-level versions should be almost risk-free; upgrades within a given major series will involve some behavior changes, while changes to build files required by upgrades between major series might be substantial, and there might be subtle semantic changes that are not immediately obvious.
The Upgrade Assistant takes the opinionated view that under normal circumstances we do not recommend upgrading through more than one major version number change at once, and we only recommend upgrading the major version number from the last version in the previous series. This is only a recommendation; the Upgrade Assistant allows the user to select any published version that is not known to be incompatible with the version of Android Studio it is running it.
The default version recommendation means that upgrading a project to the latest version can take multiple steps. At present, the User Interface does not handle multiple steps within one logical session; once one upgrade is complete, as far as the Upgrade Assistant is concerned, it is in the past and not retrievable through any more specialized means than Undo. It might be nice to revisit some of the possibilities for a more integrated handling of multi-step upgrades.
The Upgrade Assistant is implemented as a Refactoring Processor, albeit one with a number of extra bells and whistles and substantial internal structure. We have had to override some sections of the implementation of the BaseRefactoringProcessor in order to provide some small amounts of custom behaviour beyond that afforded through extensions.
Some of the internal structure resides in the use of component refactoring processors: each set of modifications to build configuration files for one conceptual change should be contained within one component processor.
Historically, the AGP Version upgrade processor itself was somewhat privileged in the architecture; however, this special status is now limited to the context of presenting information to the user.
For a given project with a particular AGP version, each component computes its necessity; for many processors dealing with some change in AGP behaviour, this will involve comparing whether the AGP version is before, within or after a region, bounded below by the version in which a new behaviour was introduced, and above by the version in which the old behaviour was removed. These processors can be said to have an “active region” in version number space.
All component processors are handled individually, and no component should depend on another component being enabled or active; nor should they in general depend on their usages being handled before or after those of a different component, except: if the version number interval for one processor‘s active region precedes another’s, the processors can rely on the fact that all of the earlier processor's actions will be taken before any of those of the later processor.
Note: currently the above statement about ordering is only informally true, by virtue of the order of the componentRefactoringProcessors defined in AgpUpgradeRefactoringProcessor; it would be good to implement more of Allen's (1983) logic and either enforce this by construction or validate it. It is important in order to offer some guarantees about the composability of upgrades, that the result of A→B and B→C be the same in textual form as A→C -- the expectation is that any ordering of processors should give the same semantics, but automated tests are easier to implement in terms of equality of outputs than equality of Gradle Dsl semantics.
Note: this is shortly to be a problem that needs to be resolved. We already have active processors which interact with properties in the packagingOptions block of android:
AndroidManifestExtractNativeLibsToUseLegacyPackagingAndroidManifestUseEmbeddedDexToUseLegacyPackagingRefactoringProcessorMigratePackagingOptionsToJniLibsAndResourcesRefactoringProcessorBut as of AGP 8.0.0-beta02, packagingOptions itself is deprecated in favour of packaging, and we will also need to perform that migration. Making sure that this happens without loss (and without writing nonsense) will not be completely trivial.
It is sometimes important for the usages generated by a single component to be processed in order; for example, to move Dsl properties to a different place in the hierarchy, adding the property at the new place must be done while the value of the old place is still available in the Dsl model. (Note that this could be achieved by storing the value in the creation usage, or by using an atomic move operation, but it can be convenient to use the ordering of usages within a component).
Under normal operation, the usage order is preserved. However, executing the proposed refactoring from the Preview (usage view) window returns the active usage set to the AgpUpgradeRefactoringProcessor in arbitrary order, through some internal (to the IntelliJ platform) use of an unordered set. We keep a list of the usages previewed so as to be able to restore the original order of the previewed usages.
A number of component upgrades involve fairly simple operations on the build Dsl, corresponding to renaming or reparenting individual properties, deleting them (either with some other mechanism for backwards-compatibility, or as a final removal of something no longer supportable), or changing the preferred way of referring to properties (for example, migrating from one overloaded operator to separate operators for different-typed values). Component processors to do these operations can be built from re-usable metacomponents, specified as fields in the PropertiesOperationsRefactoringInfo data class; individual move, rewrite or delete operations can also be used as part of the implementation of general component processors.
Individual components can block an upgrade. If an individual component detects that the state of the project is such that it should be active, but that it cannot successfully alter the project to lead to a successful upgrade, then it should block, by returning a list of BlockReason objects from an override of the blockProcessorReasons() member function.
If the processor is not mandatory for the upgrade, the user can disable it and proceed with upgrading using any remaining processors. However, if the processor is mandatory for the upgrade, the user is not offered the ability to override the block through the Upgrade Assistant interface; they must make the necessary modifications to remove the block before the Upgrade Assistant will allow its managed upgrade process.
In an ideal world, we would be able to re-use each component refactoring processor directly to provide an Action to perform the changes to the build for the corresponding behaviour change externally to the Upgrade Assistant, for example as Refactoring menu items. This has not been explored fully as yet.
Most of the Component Refactoring Processors make changes to build files, and most (all?) of them do so using the facilities of the gradle-dsl model. This model provides a view of the build configuration, mostly independent of how that configuration is expressed, and allows changes to be made to build and configuration files whether they are expressed in Groovy, Kotlin or Toml.
The changes to the build model made by the processors accumulate within the build model, with later queries to the model reflecting earlier changes, but are only written out to file near the end of the Refactoring Process, in the extending performPsiSpoilingRefactoring() method.
Modifications to project files using other mechanisms (through VirtualFiles, or even through standard Java IO) might be needed, if changes are needed to files that are not covered by the gradle-dsl model. Any such changes require the system to be notified of the changes for consistency. Individual component processors can do this by adding the PsiFile corresponding to files changed in this way to their otherAffectedFiles list; the IDE view of those files will be updated at the end of the operation of the processor, performing any postponed operations and committing documents as necessary.
When we detect a declared incompatibility between the project's declared version of the Android Gradle Plugin and the versions of AGP supported by the running Android Studio, we pop up a modal dialog informing the user of this, and allowing them to run an upgrade with very limited customizeability. The limited ability of the user to tweak the upgrade, coupled with the fact that we declare ourselves incompatible between preview versions of AGP/Studio, despite often being compatible and Sync at least partly working, suggests that we should perhaps revisit this flow in the light of the better scope for compatibility we have between AGP and Studio versions.
The modal interface is in some sense required for incompatible versions of AGP and Studio, because if the versions are truly incompatible then we cannot trust very much of the project information that we have: most of the operations of the Upgrade Assistant do not depend on valid (Sync) models, though the precondition really ought to be that they should be obtainable. On the other hand, the incompatibility between different preview versions of AGP and Studio is at least partly artificial, and could perhaps be relaxed if we implemented producer/consumer model versioning.
More generally, there is the possibility in the future that AGP and Android Studio release cycles will decouple at least somewhat. If they do, some thought needs to go into how Android Studio should treat versions of AGP that are released after themselves -- there is no realistic way of knowing what changes a newer version of AGP might require.
In the more expected case of the AGP Upgrade Assistant being used when an upgrade is merely recommended (or strongly recommended) rather than forced, we provide a tool window interface rather than a modal one.
This tool window interface allows the user to customize several things about the upgrade, with consistency maintained by various validators and UI triggers:
the target AGP version: we do not enforce an upgrade to a particular version. While the initial target is a recommended version (with somewhat complicated rules for generating that recommendation; see above for a prose description and computeGradlePluginUpgradeState() for the details), the user can override this recommendation, taking larger or smaller steps as they wish. The selection box enforces the well-formedness of the AGP version string and the constraint that upgrades should go forwards in version space only.
for the project‘s current AGP version and the user’s specified target version, a number of upgrade components will be: preconditions (pre-upgrade), required (upgrade) or recommended (applicable post-upgrade) steps. (Others will be irrelevant to that combination of current and target version, and yet others will in principle be applicable to that combination but would have no effect on the particular project). For those upgrade components, the checkbox tree maintains consistency: preconditions can only be unselected if the required components are unselected, which itself can only happen if post-upgrade steps are unselected. In the case of “upgrading” to the same version, where there are no required steps, recommended steps are selected by default.
if all components is unselected, the upgrade is blocked, or there is nothing to do, the run and preview (see below) buttons are automatically disabled; they are re-enabled if the user unblocks the upgrade or selects a component.
Users would like to have a view of changes that will be made to their project, usually expressing that as wanting a “diff view” or something. Preview, or Show Usages, is not that, but it is nevertheless interesting. The Show Usages view (available both from the tool window and -- more problematically, given that it is not modal -- from the modal forced-upgrade dialog) represents the changes that are to be made similarly to the usages in other refactoring previews (e.g. a method or class rename), with some of the same affordances: including the ability by users to remove individual usages from the action to be taken. This allows fine-grained semantic customization of the operation of the processor, but on a meaningful-change by meaningful-change basis, rather than line-by-line: if a given usage's refactoring operation does multiple things, they will all happen together or not at all.
We want to make it easy for users to revert changes made by the Upgrade Assistant, because of our acknowledgment that 100% success is unachievable.
The Upgrade Assistant integrates with Undo. The overall processor ensures that Sync runs after the refactoring is complete; it also adds a BasicUndoableAction to the UndoManager that ensures that Sync runs after Undo (and Redo) operations.
The overall processor also manages any undo or redo hooks needed by component processors, making sure to call them appropriately. At the moment the only component processor making use of this is the one updating the project JDK to meet compatibility requirements of AGP.
Note that it seems fairly easy to confuse Undo involving changes from a single Action (such as a refactoring) in multiple files.
We also provide a revert operation using the Local File History mechanism, presented to the user as a button in the post-upgrade report. Using it might not completely restore the state of the project, as for example changes to e.g. the project JDK might not be captured in Local File History; we might need to integrate better with the hooks implemented for Undo.
A few things related to the Upgrade Assistant need updating when we are close to a release of Android Studio and/or AGP, mostly related to data and testing where the AGP Upgrade Assistant is (close to) the source of historical truth.
AGP and Studio share a common view of the version of Gradle required for that version of AGP, through SdkConstants.GRADLE_LATEST_VERSION. This means that for a release of Studio, the Upgrade Assistant will automatically be aware of the version of Gradle required for upgrades to the latest version of AGP.
However, the Upgrade Assistant also supports upgrades to versions of AGP that are not the current, each of which might have its own required version of Gradle; the Upgrade Assistant by design updates Gradle, if necessary, to the minimal version required by the requested version of AGP. A table listing compatibility requirements is maintained in CompatibleGradleVersion, for use in the Upgrade Assistant but also in project templates and in tests; this table should be essentially the same as the user-facing documentation at the Android Gradle Plugin release notes.
More infrequently, Android Studio and/or AGP change their distribution or requirements regarding other external software: for example, the SDK build tools, the NDK, or the JDK.
Optional or required upgrades to the JDK are handled by a dedicated component refactoring processor, and changes to the table required versions need to be made when a new version of the JDK is required by the system. At the moment that table is local to the processor, in the CompatibleJdkVersion enum class.
At present, changes in required versions of the SDK build tools or the NDK are not handled by the AGP Upgrade Assistant.
The integration tests for the AGP Upgrade Assistant are, for infrastructural reasons, run in shards each specialized to the current version of AGP for that test. This is because it is relatively expensive to start a Gradle daemon process, but once it has been started it can be shared between a number of tests if each uses the same version of AGP. We verify that the upgrade succeeds (in our terms, which implies syncing successfully), but rather than running sync on the post-upgrade state of the project (which would require starting more Gradle processes, slowing the tests down and using up our infra capacity) we verify that the post-upgrade state as described by build-related files is equal to golden files.
When a stable (or RC, or possibly beta) version of Studio is released, it should be integrated into the OldAgpTests and AgpVersionSoftwareEnvironment mechanisms in tests, and prebuilts made available, so that integration tests can be run of the Upgrade Assistant's (on studio-main) ability to upgrade projects to versions of AGP on the beta channel.
Tests of the upgrade process from canary to canary are currently done partly-manually. Automating them would be good, but ideally we should not have to upload prebuilds of every AGP canary. Perhaps we should upload the first AGP alpha of every series, and verify that the Upgrade Assistant can upgrade to the latest alpha / beta (but note that this might require a change in behaviour of the Upgrade Assistant recommendations to accommodate the fact that the test environment will not treat the “latest” version as having been published).
James F. Allen, “Maintaining Knowledge about Temporal Intervals”. Communications of the ACM 26:11, pp.832—843 (1983)
(Google internal) Index of AGP Upgrade Assistant documents