Upgrade the ICU on Android to the new upstream version. The new upstream versions can be found at https://github.com/unicode-org/icu/releases.
The below contains the steps and commands in order to upgrade the ICU version in the AOSP.
Create a new branch in AOSP
aosp/icu67
in the external/icu
project forked from aosp/master
Configure the versions and temp directory
2a. Customize the following environment variables. The following example targets the ICU 71.1 and CLDR 41.0 version.
export ICU_VERSION=71 export ICU_MINOR_VERSION=1 export CLDR_VERSION=41 export ICU_UPGRADE_BUG=1234567890 # buganizer bug # Initially empty directory to store upstream source export UPSTREAM_CLDR_GIT=/media/user/disk/icu-git/cldr export UPSTREAM_ICU_GIT=/media/user/disk/icu-git/icu
2b. Build the clean source of AOSP
source build/envsetup.sh lunch sdk_phone_x86_64-userdebug m
Copy the CLDR sources into the upstream-release-cldr
branch
3a. Copy sources
export CLDR_UPSTREAM_BRANCH=release-${CLDR_VERSION} cd ${ANDROID_BUILD_TOP}/external/cldr git fetch aosp upstream-release-cldr git branch ${CLDR_UPSTREAM_BRANCH} --track aosp/upstream-release-cldr git checkout ${CLDR_UPSTREAM_BRANCH} git clone https://github.com/unicode-org/cldr.git ${UPSTREAM_CLDR_GIT} git --git-dir=${UPSTREAM_CLDR_GIT}/.git --work-tree=${UPSTREAM_CLDR_GIT} fetch git --git-dir=${UPSTREAM_CLDR_GIT}/.git --work-tree=${UPSTREAM_CLDR_GIT} checkout ${CLDR_UPSTREAM_BRANCH} rm -rf * cp -r ${UPSTREAM_CLDR_GIT}/* . git clean -dfX # Remove ignored files git add -A git commit -F- <<EOF Copy upstream ${CLDR_UPSTREAM_BRANCH} Bug: ${ICU_UPGRADE_BUG} Test: n/a EOF # Upload this CL to upstream-release-cldr branch repo upload --cbr .
3b. Merge the upstream sources with patches in aosp/master
export CLDR_BRANCH=cldr${CLDR_VERSION}-master git branch ${CLDR_BRANCH} --track aosp/master git checkout ${CLDR_BRANCH} git merge ${CLDR_UPSTREAM_BRANCH} -m " Merge CLDR ${CLDR_VERSION} in upstream-release-cldr into aosp/master Bug: ${ICU_UPGRADE_BUG} Test: external/icu/tools/updatecldrdata.py "
3c. Resolve any merge conflicts with the Android-specific patches. Continue creating the merge commit
git merge --continue
3d. Upload the CL to master branch
repo upload --cbr .
Copy ICU upstream sources into external/icu
cd ${ANDROID_BUILD_TOP}/external/icu export ICU_BRANCH=icu${ICU_VERSION} export UPSTREAM_RELEASE_TAG=release-${ICU_VERSION}-${ICU_MINOR_VERSION} git fetch aosp ${ICU_BRANCH} git branch ${ICU_BRANCH} --track aosp/${ICU_BRANCH} git checkout ${ICU_BRANCH} # Clone the upstream CLDR repo locally to ${UPSTREAM_ICU_GIT} test -d ${UPSTREAM_ICU_GIT} || git clone https://github.com/unicode-org/icu.git ${UPSTREAM_ICU_GIT} git --git-dir=${UPSTREAM_ICU_GIT}/.git --work-tree=${UPSTREAM_ICU_GIT} fetch git --git-dir=${UPSTREAM_ICU_GIT}/.git --work-tree=${UPSTREAM_ICU_GIT} checkout ${UPSTREAM_RELEASE_TAG} find icu4j/ -type f,d ! -regex ".*/\(Android.mk\|Android.bp\|adjust_icudt_path.mk\|liblayout-jarjar-rules.txt\|.gitignore\|AndroidTest.xml\)" -delete find icu4c/ -type f,d ! -regex ".*/\(Android.mk\|Android.bp\|.gitignore\|AndroidTest.xml\)" -delete cp -r ${UPSTREAM_ICU_GIT}/icu4j . cp -r ${UPSTREAM_ICU_GIT}/icu4c . git checkout HEAD -- icu4c/.gitignore icu4j/.gitignore # Android has extra .gitignores. Use our version. rm -r tools/cldr cp -r ${UPSTREAM_ICU_GIT}/tools/cldr tools/cldr git add -A git commit -F- <<EOF Copy ICU ${UPSTREAM_RELEASE_TAG} into aosp/${ICU_BRANCH} Copy the files with the following commands: find icu4j/ -type f,d ! -regex ".*/\(Android.mk\|Android.bp\|adjust_icudt_path.mk\|liblayout-jarjar-rules.txt\|.gitignore\|AndroidTest.xml\)" -delete find icu4c/ -type f,d ! -regex ".*/\(Android.mk\|Android.bp\|.gitignore\|AndroidTest.xml\)" -delete cp -r \${UPSTREAM_ICU_GIT}/icu4j . cp -r \${UPSTREAM_ICU_GIT}/icu4c . git checkout HEAD -- icu4c/.gitignore icu4j/.gitignore rm -r tools/cldr cp -r \${UPSTREAM_ICU_GIT}/tools/cldr tools/cldr EOF
Apply Android-specific patches into external/icu
5a. Cherry-pick the patches from the last staging branch. For example using the following query for ICU 71 patches
git cherry-pick <first_patch_in_the_chain>~1..<last_patch_in_the_chain>
5b. Cherry-pick the patches since the ICU upgrade
Regenerate and commit the artifacts
6a. Update icu source data files
croot external/icu tools/updatecldrdata.py git add -A git commit -F- <<EOF Regenerated source data files with Android CLDR patches Source data files updated using: tools/updatecldrdata.py Test: n/a EOF
6b. Update icu binary data files
tools/updateicudata.py git add -A git commit -F- <<EOF Regenerated binary data files with Android CLDR patches Binary data files updated using: tools/updateicudata.py Test: n/a EOF
6c. Pin the public API surface temporarily
./tools/srcgen/generate_allowlisted_public_api.sh git add -A git commit -F- <<EOF Pin the current API list Test: ./tools/srcgen/generate_allowlisted_public_api.sh EOF
6d. Regenerate android_icu4j/
tools/srcgen/generate_android_icu4j.sh git add -A git commit -F- <<EOF Regenerate android_icu4j/ from icu4j/ android_icu4j files updated using: tools/srcgen/generate_android_icu4j.sh Test: n/a EOF
find android_icu4j/ -name *.orig -delete # Please manually update and resolve conflict of the java doc in android_icu4j/ git commit -amend # Regenerate the patch files ./tools/srcgen/javadoc_patches/create_patches.sh git add -A && git commit -F- <<EOF Regenerate java doc patches Test: n/a EOF
6e. Re-genereate libandroidicu/ sources
./tools/icu4c_srcgen/generate_libandroidicu.py git add -A && git commit -F- <<EOF Regenerate libandroidicu The command: ./tools/icu4c_srcgen/generate_libandroidicu.py Test: n/a EOF
6f. Regenerate libicu/ sources
./tools/icu4c_srcgen/generate_ndk.py git add -A && git commit -a -F- <<EOF Regenerate libicu.so and ICU4C CTS headers The command: ./tools/icu4c_srcgen/generate_ndk.py Test: n/a EOF
git diff HEAD~1 ./libicu/ndk_headers/
# Manually update doxygen doc in the .h headers in libicu/ndk_headers ./tools/icu4c_srcgen/doc_patches/create_patches.sh git add -A && git commit -F- <<EOF Regenerate patches in libicu headers EOF
6g. [Not required for every ICU release] Increment Distro major version
6h. Generate time zone files
cd $ANDROID_BUILD_TOP/system/timezone repo start ${ICU_BRANCH} . ./update-tzdata.py git add -A git commit -F- <<EOF Regenerate data files for ICU ${ICU_VERSION} upgrade Binary data files updated using: system/timezone/update-tzdata.py This updates the ICU version number inside of icu_tzdata.dat but doesn't change the timezone data itself. Bug: ${ICU_UPGRADE_BUG} Test: n/a EOF # If only the build time in icu4c/source/data/misc/zoneinfo64.txt, this step isn't needed. cd $ANDROID_BUILD_TOP/external/icu git add -A git commit -F- <<EOF Regenerate tz-related data files Data files updated using: system/timezone/update-tzdata.py Bug: ${ICU_UPGRADE_BUG} Test: n/a EOF
6i. Update the version numbers in the METADATA
git commit
the file change6j. Regenerate frameworks/base/libs/androidfw/LocaleDataTables.cpp
croot frameworks/base ./tools/localedata/extract_icu_data.py $ANDROID_BUILD_TOP > libs/androidfw/LocaleDataTables.cpp git commit -a -F- <<EOF Regenerate LocaleDataTables.cpp due to ICU ${ICU_VERSION} upgrade The command: ./tools/localedata/extract_icu_data.py \$ANDROID_BUILD_TOP > libs/androidfw/LocaleDataTables.cpp Bug: ${ICU_UPGRADE_BUG} Test: atest FrameworksCoreTests:android.text.format EOF
Build and run test
7a. Build by m droid cts
7b. Run the device tests by atest CtsIcu4cTestCases CtsIcuTestCases CtsLibcoreTestCases CtsLibcoreOjTestCases CtsBionicTestCases CtsTextTestCases minikin_tests -- --abi x86_64 # the primary ABI
7c. Run the host-side test by
ant check
make CINTLTST_OPTS=-w INTLTEST_OPTS=-w check
(Currently, it has some failing tests. No of failures?)Upload the CLs to gerrit for code reviews from aosp/icu${ICU_VERSION}
in external/icu
and aosp/upstream-release-cldr
in external/cldr
repo upload --cbr -o uploadvalidator~skip --no-verify .
aosp/icu*
branch to aosp/mastercd $ANDROID_BUILD_TOP/external/icu repo start icu${ICU_VERSION}-master . git merge --no-ff icu${ICU_VERSION} -m " Merge branch aosp/icu${ICU_VERSION} into aosp/master Bug: ${ICU_UPGRADE_BUG} Test: atest CtsIcu4cTestCases CtsIcuTestCases CtsLibcoreTestCases CtsLibcoreOjTestCases CtsBionicTestCases CtsTextTestCases minikin_tests "
Upload and submit changes from external/icu, external/cldr, libcore, frameworks/base, system/timezone
10a. repo upload --cbr -o uploadvalidator~skip --no-verify .
10b. Code review the diff in android_icu4j/src/main/tests/android/icu/extratest/expected_transliteration_id_list.txt
After submitting all the CLs to aosp/master, expose the new stable ICU4J APIs to Android SDK
rm tools/srcgen/allowlisted-public-api.txt ./tools/generate_android_icu4j.sh # Modify Icu4jTransform.java to allowlist more classes if needed. Check the error message for the details m update-api droid git commit -a -F- <<EOF Expose the new stable APIs from ICU4J ${ICU_VERSION} According to the upstream API coverage report external/icu/icu4j/coverage-exclusion.txt, the methods should have API coverage running in the existing CtsIcuTestCases. Bug: ${ICU_UPGRADE_BUG} Test: atest CtsIcuTestCases EOF
Hi, Libcore and ICU team, ICU <version> just landed Android AOSP. https://android.googlesource.com/platform/external/icu/+/master/README.version as well as Android S (or Android 12). https://googleplex-android.googlesource.com/platform/external/icu/+/sc-dev/README.version Note: - Contains bug fixes / build changes with a small set of API methods added. - Unicode stays at version 14, and no new version has been published yet.