android16-6.12 June 2025 release 17 Artifacts: https://ci.android.com/builds/submitted/13859059/kernel_aarch64/latest
FROMGIT: rcu: Fix delayed execution of hurry callbacks
We observed a regression in our customer’s environment after enabling
CONFIG_LAZY_RCU. In the Android Update Engine scenario, where ioctl() is
used heavily, we found that callbacks queued via call_rcu_hurry (such as
percpu_ref_switch_to_atomic_rcu) can sometimes be delayed by up to 5
seconds before execution. This occurs because the new grace period does
not start immediately after the previous one completes.
The root cause is that the wake_nocb_gp_defer() function now checks
"rdp->nocb_defer_wakeup" instead of "rdp_gp->nocb_defer_wakeup". On CPUs
that are not rcuog, "rdp->nocb_defer_wakeup" may always be
RCU_NOCB_WAKE_NOT. This can cause "rdp_gp->nocb_defer_wakeup" to be
downgraded and the "rdp_gp->nocb_timer" to be postponed by up to 10
seconds, delaying the execution of hurry RCU callbacks.
The trace log of one scenario we encountered is as follow:
// previous GP ends at this point
rcu_preempt [000] d..1. 137.240210: rcu_grace_period: rcu_preempt 8369 end
rcu_preempt [000] ..... 137.240212: rcu_grace_period: rcu_preempt 8372 reqwait
// call_rcu_hurry enqueues "percpu_ref_switch_to_atomic_rcu", the callback waited on by UpdateEngine
update_engine [002] d..1. 137.301593: __call_rcu_common: wyy: unlikely p_ref = 00000000********. lazy = 0
// FirstQ on cpu 2 rdp_gp->nocb_timer is set to fire after 1 jiffy (4ms)
// and the rdp_gp->nocb_defer_wakeup is set to RCU_NOCB_WAKE
update_engine [002] d..2. 137.301595: rcu_nocb_wake: rcu_preempt 2 FirstQ on cpu2 with rdp_gp (cpu0).
// FirstBQ event on cpu2 during the 1 jiffy, make the timer postpond 10 seconds later.
// also, the rdp_gp->nocb_defer_wakeup is overwrite to RCU_NOCB_WAKE_LAZY
update_engine [002] d..1. 137.301601: rcu_nocb_wake: rcu_preempt 2 WakeEmptyIsDeferred
...
...
...
// before the 10 seconds timeout, cpu0 received another call_rcu_hurry
// reset the timer to jiffies+1 and set the waketype = RCU_NOCB_WAKE.
kworker/u32:0 [000] d..2. 142.557564: rcu_nocb_wake: rcu_preempt 0 FirstQ
kworker/u32:0 [000] d..1. 142.557576: rcu_nocb_wake: rcu_preempt 0 WakeEmptyIsDeferred
kworker/u32:0 [000] d..1. 142.558296: rcu_nocb_wake: rcu_preempt 0 WakeNot
kworker/u32:0 [000] d..1. 142.558562: rcu_nocb_wake: rcu_preempt 0 WakeNot
// idle(do_nocb_deferred_wakeup) wake rcuog due to waketype == RCU_NOCB_WAKE
<idle> [000] d..1. 142.558786: rcu_nocb_wake: rcu_preempt 0 DoWake
<idle> [000] dN.1. 142.558839: rcu_nocb_wake: rcu_preempt 0 DeferredWake
rcuog/0 [000] ..... 142.558871: rcu_nocb_wake: rcu_preempt 0 EndSleep
rcuog/0 [000] ..... 142.558877: rcu_nocb_wake: rcu_preempt 0 Check
// finally rcuog request a new GP at this point (5 seconds after the FirstQ event)
rcuog/0 [000] d..2. 142.558886: rcu_grace_period: rcu_preempt 8372 newreq
rcu_preempt [001] d..1. 142.559458: rcu_grace_period: rcu_preempt 8373 start
...
rcu_preempt [000] d..1. 142.564258: rcu_grace_period: rcu_preempt 8373 end
rcuop/2 [000] D..1. 142.566337: rcu_batch_start: rcu_preempt CBs=219 bl=10
// the hurry CB is invoked at this point
rcuop/2 [000] b.... 142.566352: blk_queue_usage_counter_release: wyy: wakeup. p_ref = 00000000********.
This patch changes the condition to check "rdp_gp->nocb_defer_wakeup" in
the lazy path. This prevents an already scheduled "rdp_gp->nocb_timer"
from being postponed and avoids overwriting "rdp_gp->nocb_defer_wakeup"
when it is not RCU_NOCB_WAKE_NOT.
Fixes: 3cb278e73be5 ("rcu: Make call_rcu() lazy to save power")
Co-developed-by: Cheng-jui Wang <cheng-jui.wang@mediatek.com>
Signed-off-by: Cheng-jui Wang <cheng-jui.wang@mediatek.com>
Co-developed-by: Lorry.Luo@mediatek.com
Signed-off-by: Lorry.Luo@mediatek.com
Tested-by: weiyangyang@vivo.com
Signed-off-by: weiyangyang@vivo.com
Signed-off-by: Tze-nan Wu <Tze-nan.Wu@mediatek.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Neeraj Upadhyay (AMD) <neeraj.upadhyay@kernel.org>
Bug: 433659824
Bug: 434811509
Bug: 435082773
(cherry picked from commit 463d46044f04013306a4893242f65788b8a16b2e
https://git.kernel.org/pub/scm/linux/kernel/git/neeraj.upadhyay/linux-rcu.git rcu.merge.22.07.2025)
Change-Id: I4a0cedfd3fe35f609262ba204f5fea2be2d5f9de
Signed-off-by: Tze-nan Wu <Tze-nan.Wu@mediatek.com>
(cherry picked from commit d64d8b7dab32e1356e13ee75a26a6bd386d7cbc1)
BEST: Make all of your changes to upstream Linux. If appropriate, backport to the stable releases. These patches will be merged automatically in the corresponding common kernels. If the patch is already in upstream Linux, post a backport of the patch that conforms to the patch requirements below.
EXPORT_SYMBOL_GPL() require an in-tree modular driver that uses the symbol -- so include the new driver or changes to an existing driver in the same patchset as the export.LESS GOOD: Develop your patches out-of-tree (from an upstream Linux point-of-view). Unless these are fixing an Android-specific bug, these are very unlikely to be accepted unless they have been coordinated with kernel-team@android.com. If you want to proceed, post a patch that conforms to the patch requirements below.
scripts/checkpatch.plUPSTREAM:, BACKPORT:, FROMGIT:, FROMLIST:, or ANDROID:.Change-Id: tag (see https://gerrit-review.googlesource.com/Documentation/user-changeid.html)Bug: tag.Signed-off-by: tag by the author and the submitterAdditional requirements are listed below based on patch type
UPSTREAM:, BACKPORT:UPSTREAM:.(cherry picked from commit ...) line important patch from upstream
This is the detailed description of the important patch
Signed-off-by: Fred Jones <fred.jones@foo.org>
- then Joe Smith would upload the patch for the common kernel as
UPSTREAM: important patch from upstream
This is the detailed description of the important patch
Signed-off-by: Fred Jones <fred.jones@foo.org>
Bug: 135791357
Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
(cherry picked from commit c31e73121f4c1ec41143423ac6ce3ce6dafdcec1)
Signed-off-by: Joe Smith <joe.smith@foo.org>
BACKPORT: instead of UPSTREAM:.UPSTREAM:(cherry picked from commit ...) line BACKPORT: important patch from upstream
This is the detailed description of the important patch
Signed-off-by: Fred Jones <fred.jones@foo.org>
Bug: 135791357
Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
(cherry picked from commit c31e73121f4c1ec41143423ac6ce3ce6dafdcec1)
[joe: Resolved minor conflict in drivers/foo/bar.c ]
Signed-off-by: Joe Smith <joe.smith@foo.org>
FROMGIT:, FROMLIST:,FROMGIT:(cherry picked from commit <sha1> <repo> <branch>). This must be a stable maintainer branch (not rebased, so don't use linux-next for example).BACKPORT: FROMGIT: important patch from upstream
This is the detailed description of the important patch
Signed-off-by: Fred Jones <fred.jones@foo.org>
- then Joe Smith would upload the patch for the common kernel as
FROMGIT: important patch from upstream
This is the detailed description of the important patch
Signed-off-by: Fred Jones <fred.jones@foo.org>
Bug: 135791357
(cherry picked from commit 878a2fd9de10b03d11d2f622250285c7e63deace
https://git.kernel.org/pub/scm/linux/kernel/git/foo/bar.git test-branch)
Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
Signed-off-by: Joe Smith <joe.smith@foo.org>
FROMLIST:Link: tag with a link to the submittal on lore.kernel.orgBug: tag with the Android bug (required for patches not accepted into a maintainer tree)BACKPORT: FROMLIST: FROMLIST: important patch from upstream
This is the detailed description of the important patch
Signed-off-by: Fred Jones <fred.jones@foo.org>
Bug: 135791357
Link: https://lore.kernel.org/lkml/20190619171517.GA17557@someone.com/
Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
Signed-off-by: Joe Smith <joe.smith@foo.org>
FROMLIST: tag to try to hide this fact. Use the ANDROID: tag as described below as this must be considered as an Android-specific submission, not an upstream submission as the community will not accept these changes as-is.ANDROID:ANDROID:Fixes: tag that cites the patch with the bug ANDROID: fix android-specific bug in foobar.c
This is the detailed description of the important fix
Fixes: 1234abcd2468 ("foobar: add cool feature")
Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
Signed-off-by: Joe Smith <joe.smith@foo.org>
ANDROID:Bug: tag with the Android bug (required for android-specific features)