Sound Trigger Middleware

Overview

Sound Trigger Middleware is a system service that exposes sound trigger functionality (low-power detection of acoustic events) to applications and higher-level system service.

It has the following roles:

• Isolating the soundtrigger HAL from potentially untrusted clients.
• Enforcing correct behavior of the clients.
• Enforcing correct behavior of the HAL and attempting to recover from failures.
• Enforcing permissions for using soundtrigger functionality.
• Serializing access to the HAL.
• Logging soundtrigger usage in a comprehensive and consistent manner.
• Generating a dumpsys report including current state and history of operations.
• Providing a standard interface regardless which version of the HAL is implemented and gracefully degrading operation whenever necessary.

Structure

The service implementation can be divided into three main layers:

• The “bottom layer” is concerned with HAL compatibility - making all HAL versions look and behave the same.
• The “middle layer” is concerned with the business logic of the service.
• The “top layer” is concerned with exposing this functionality as a System Service and integrating with other parts of the system.

HAL Compatibility Layer

This layer implements the ISoundTriggerHal interface, which is the version-agnostic representation of the sound trigger HAL driver. It has two main implementations, SoundTriggerHw2Compat and SoundTriggerHw3Compat responsible for adapting to V2.x and V3 HAL drivers, respectively, including supporting their respective minor-version differences.

This layer also includes several ISoundTriggerHal decorators, such as SoundTriggerHalWatchdog that enforces deadlines on calls into the HAL, and SoundTriggerHalEnforcer which enforces that the HAL respects the expected protocol.

The decorator-based design is an effective tool for separation of aspects and modularity, thus keeping classes relatively small and focused on one concern. It is also very effective for testability by following dependency injection principles.

Business Logic Layer

This layer also uses a decorator-based design for separation of concerns. The main interface being decorated is ISoundTriggerMiddlwareInternal, which closely follows the external-facing AIDL interface, ISoundTriggerMiddlewareService.

Each of the decorators serves a focused purpose: for example, SoundTriggerMiddlwarePermission deals with enforcing permissions required for the various methods, SoundTriggerMiddlewareLogging logs all API usage, SoundTriggerMiddlewareValidation enforces correct usage of the protocol and isolates client errors from internal server errors.

At the bottom of this decorator stack is SoundTriggerMiddlewareImpl / SoundTriggerModule, which are the adapter between ISoundTriggerHal and ISoundTriggerMiddlwareInternal, introducing the notion of having separate client sessions sharing the same HAL.

Service Layer

This layer ties everything together. It instantiates the actual system service and the decorator stack. It also provides concrete connections to the Audio service (for negotiating sessions shared between Audio and Sound Trigger and for notifications about audio recording) and to the various HAL factories.

This is the only layer that makes strong assumptions about the environment instead of relying on abstractions.

Error Handling and Exception Conventions

We follow conventions for usage of exceptions in the service, in order to correctly and consistently distinguish the following cases:

1. The client has done something wrong.
2. The service implementation has done something wrong.
3. The HAL has done something wrong.
4. Nobody has done anything wrong, but runtime conditions prevent an operation from being fulfilled as intended.

The SoundTriggerMiddlewarePermission class would reject any calls from unauthorized clients, responding with the appropriate exception.

The SoundTriggerMiddlewareValidation class does much of this separation. By validating the client‘s data and state, it would throw a relevant RuntimeException exception to the client without passing the requests down to the lower layers. Once that is done, any exception thrown from the underlying implementation can be assumed to be not the client’s fault. If caught, they will be classified according to the following rule:

• If they are RecoverableExceptions, they represent category #4 above, and will be presented to the client as ServiceSpecificExceptions with the same error code.
• Otherwise, they are considered an internal error (including HAL malfunction) and will be presented to the client as ServiceSpecificException(Status.INTERNAL_ERROR).

Internally, we would throw RecoverableException whenever appropriate. Whenever a HAL malfunctions, SoundTriggerHalEnforcer is responsible for rebooting it and throwing an exception. A HAL death is considered a valid failure mode, and thus result in RecoverableException(Status.DEAD_OBJECT), which ends up as a ServiceSpecificException(Status.DEAD_OBJECT) on the client side.

Notes About Thread Synchronization

This component has some tricky thread synchronization considerations due to its layered design and due to the fact that it is involved in both in-bound and out-bound calls from / to external components.

The following mutexes need to be considered:

• Typically, a one or more mutexes that exist in every layer of the sound trigger middleware stack to serialize access to its internal state or to external components.
• Audio Policy Service lock. This one is external - it should be assumed to be held whenever we're inside the ExternalCaptureStateTracker.setCaptureState() call stack AND to be acquired from within our calls into AudioSessionProvider.acquireSession() / AudioSessionProvider.releaseSession().

To avoid potential deadlocks, a strict locking order must be ensured whenever nesting locks. The order is:

• Upper layers of the stack, starting from the top (i.e. may not attempt to acquire a higher-layer mutex while a lower-layer mutex is being held) until ISoundTriggerHw2.
• Audio Policy Service lock.
• Lower layers of the stack, starting from ISoundTriggerHw2 all the way down to the HAL.

In order to enforce this order, some conventions are established around when it is safe for a module to call another module, while having its local mutex(es) held:

• Most calls (see exceptions below) originating from SoundTriggerMiddlewareService simply propagate down the decorator stack. It is legal to call into the next layer down while holding a local mutex. It is illegal to invoke a callback with a local mutex held.
• Callbacks propagate from the lower layers up to the upper layers. It is legal to hold a local mutex within a callback, but not while call to an upper layer.
• In order to be able to synchronize, despite the asynchronous nature of callbacks, stopRecognition() and unloadModel() work differently. They guarantee that once they return, the callbacks associated with them will no longer be called. This implies that they have to block until any pending callbacks are done processing and since these callbacks are potentially holding locks of higher-order mutexes, we must not be holding a local mutex while calling down. The proper sequence for these calls is:
  • Obtain the local lock if needed. Update/check local state as necessary.
  • Call the respective method of the delegate (“downwards”). Once it returns, not more callbacks related to this operation will be called.
  • Obtain the local lock if needed. Update local state as necessary. Assume that state might have changed while the lock has been released.
  • Release the local lock.
  • Invoke any synchronous callbacks if needed.
• Calling from SoundTriggerMiddlewareImpl / SoundTriggerModule into the audio policy service via acquireSession() / releaseSession() while holding the local lock is legal.
• setCaptureState() calls, originating from Audio Policy Service, into the lower layers of the stack may call into the HAL (specifically, they must invoke stopRecognition(), but must not block on callbacks. For this reason, SoundTriggerHw2ConcurrentCaptureHandler, which is the recipient of these calls, features a buffer and an additional thread, which allows the actual stopping to be synchronous, as required, without having to block the call upon higher layers processing the callbacks.