An advanced multi-display support is requested for certain Android form-factors so that user(s) can type text on each display at the same time without losing software keyboard focus in other displays (hereafter called “multi-client scenario”). This is not possible in Android IMEs built on top of InputMethodService
class. The assumption that a single IME client can be focused at the same time was made before Android IME APIs were introduced in Android 1.5 and many public APIs in InputMethodService
have already relied heavily on that assumption (hereafter called “single-client scenario”). Updating InputMethodService
class to support multi-client scenario is, however, quite challenging because:
InputMethodService
, which is already hard to maintain,Thus the first decision we made was that to support such special multi-display environments a new type of IME (hereafter called “multi-client IME”) needs to be designed and implemented rather than reusing InputMethodService
public class. On top of this decision, following decisions were also made:
InputMethodService
, multiple multi-client IMEs cannot be enabled. The system pre-installs only one multi-client IME.MultiClientInputMethodManagerService
(MCIMMS) for multi-client IME scenario and use it instead of InputMethodManagerService
(IMMS) when a certain runtime flag is enabled at the device boot time. This means:For multi-client IME to properly work, an internal boolean resource com.android.internal.R.bool.config_perDisplayFocusEnabled
needs to be true
. Since this value cannot be overridden at the run time, you may need to rebuild the system image to enable per-display focus mode.
As for multi-client IME mode itself, you can enable multi-client IME mode just by setting a valid component name that supports multi-client IME protocol to the system property persist.debug.multi_client_ime
, as long as android.os.Build.IS_DEBUGGABLE
returns true
and you can have root access. Reboot is required for this to take effect.
# Build and install a sample multi-client IME make -j MultiClientInputMethod adb install -r $OUT/system/priv-app/MultiClientInputMethod/MultiClientInputMethod.apk # Enable multi-client IME for the side-loaded sample multi-client IME adb root adb shell setprop persist.debug.multi_client_ime com.example.android.multiclientinputmethod/.MultiClientInputMethod adb reboot
To disable multi-client IME on non-supported devices again, just clear persist.debug.multi_client_ime
as follows. Reboot is still required for this to take effect.
# Disable multi-client IME again adb root adb shell "setprop persist.debug.multi_client_ime ''" adb reboot
There is a sample multi-client IME in development/samples/MultiClientInputMethod/
.
Neither forward nor backward compatibility is guaranteed in multi-client IME APIs. The system integrator is responsible for making sure that both the system and pre-installed multi-client IME are compatible with each other every time the system image is updated.
VrManager#setVrInputMethod()
system API is not supported.InputMethodManager#getEnabledInputMethodSubtypeList()
InputMethodManager#getCurrentInputMethodSubtype()
InputMethodManager#setCurrentInputMethodSubtype()
InputMethodManager#getShortcutInputMethodsAndSubtypes()
InputMethodManager#setAdditionalInputMethodSubtypes()
InputMethodManager#getLastInputMethodSubtype()
Settings.Secure#SELECTED_INPUT_METHOD_SUBTYPE
InputMethodManager#showInputMethodPicker()
InputMethodManager#showInputMethodAndSubtypeEnabler()
InputMethodManager#setInputMethod()
InputMethodManager#setInputMethodAndSubtype()
InputMethodManager#switchToLastInputMethod()
InputMethodManager#switchToNextInputMethod()
InputMethodManager#shouldOfferSwitchingToNextInputMethod()
Settings.Secure#DEFAULT_INPUT_METHOD
Settings.Secure#ENABLED_INPUT_METHODS
false
.InputMethodManager#isFullscreenMode()
InputMethodService.Insets#touchableRegion
)InputConnection#commitContent()
API is silently ignored.adb shell dumpsys
does not include any log from MCIMMS yet.In order to override persist.debug.multi_client_ime
device property, an explicit root permission is needed.
Multi-client IME must be pre-installed since it is considered as part of the system component. This is verified by checking ApplicationInfo.FLAG_SYSTEM
bit. This security check can be bypassed when Build.IS_DEBUGGABLE
is true
so that IME developers can easily side-load their APKs during development phase.
public final class MultiClientInputMethodManagerService { ... @Nullable private static InputMethodInfo queryInputMethod(Context context, @UserIdInt int userId, @Nullable ComponentName componentName) { ... if (! && (si.applicationInfo.flags & ApplicationInfo.FLAG_SYSTEM) == 0) { Slog.e(TAG, imeId + " must be pre-installed when Build.IS_DEBUGGABLE is false"); return null; }
services/core/java/com/android/server/inputmethod/MultiClientInputMethodManagerService.java
Sometimes MCIMMS needs to issue certain types of identifiers to the multi-client IME so that the IME can later specify to which entity or resource it intends to access. A good example is the IME client identifier. Multi-client IME definitely need to be able to specify which IME client to be interacted with for certain operations. The problem is that MCIMMS cannot simply pass IInputMethodClient
to the multi-client IME as an ID because it would allow the IME to make IPC calls to the IME client. For this kind of situations, we usually use Binder
object just as a non-spoofable token. For instance, IMMS creates another ‘Binder’ token then pass it to the IME, instead of directly passing ‘IWindow’ Binder token.
public class InputMethodManagerService extends IInputMethodManager.Stub implements ServiceConnection, Handler.Callback { ... @GuardedBy("mMethodMap") private final WeakHashMap<IBinder, IBinder> mImeTargetWindowMap = new WeakHashMap<>(); ... @GuardedBy("mMethodMap") @NonNull InputBindResult attachNewInputLocked(@StartInputReason int startInputReason, boolean initial) { ... final Binder startInputToken = new Binder(); final StartInputInfo info = new StartInputInfo(mCurToken, mCurId, startInputReason, !initial, mCurFocusedWindow, mCurAttribute, mCurFocusedWindowSoftInputMode, mCurSeq); mImeTargetWindowMap.put(startInputToken, mCurFocusedWindow); ... } ... @BinderThread private void reportStartInput(IBinder token, IBinder startInputToken) { if (!calledWithValidToken(token)) { return; } synchronized (mMethodMap) { final IBinder targetWindow = mImeTargetWindowMap.get(startInputToken); if (targetWindow != null && mLastImeTargetWindow != targetWindow) { mWindowManagerInternal.updateInputMethodTargetWindow(token, targetWindow); } mLastImeTargetWindow = targetWindow; } }
services/core/java/com/android/server/inputmethod/InputMethodManagerService.java
However, in MCIMMS, for certain cases we decided to use a simple integer token, which can be spoofable and can be messed up if integer overflow happens. This is because:
Binder
token.Currently the following IDs are implemented as integer tokens: