Change Detector enables Always-on Mobile Context

Context is defined as the inference of user or environment from sensor measurements on nearby devices. Enabling a context-aware platform requires always-on sensing, and low-power is important if the platform is mobile. Therefore various aspects of sensor-detected context on a mobile platform need to be addressed:

  • Sensor Choice: One of the lowest power sensors in a smart phone is the accelerometer which can be sampled at 50Hz using less than 0.02 mWh of energy. This is a negligible fraction of the total cellphone battery which generally has a capacity of more than 5000 mWh, but the information needs to be acquired and interpreted as discussed next.
  • Sensor Acquisition: Sampling sensors on the main CPU requires a wake lock on the application processor (AP), which would consume at least 180 mWh of energy. This is not sustainable and points to requiring sensor sampling on an embedded processor. However, even a Cortex M3 uses over 50 mWh of energy and hence a very power-efficient core is needed to sustain always-on sampling.
  • Context Change Detector: Once the sensors are sampled, some processing needs to occur to determine the context. Therefore, to avoid constantly waking the AP for no reason, the embedded processor should also detect whether a context has changed.
  • Context Classification: Once a change is detected, the sensor data is interpreted to classify which context occurred. Since contexts do not happen very often throughout the day, this could run on the AP which has the benefit of having access to other sub-systems beyond sensors.

The above shows the Context Change Detector is at the heart of a low-power context-aware system. At Sensor Platforms, we have developed just such a system. An important point to saving power is for the embedded processor to notify the AP only on context change rather than on motion change.

The following two figures show our change detector in action. The first shows the case where the user is sitting with the phone in-pocket, then the user gets up and starts walking. Our change detector extracts relevant segments, shown as shaded blue rectangles, and notifies any subscribers of potential context changes. Note that even though the phone undergoes significant motion while the user is walking, the Context Change Detector is able to tell that it is the same context and not continually wake the AP. Step counting is also tracked with this Change Detector.

The second figure shows a smart phone initially on a table. The phone rings, the user picks it up, talks for a few seconds, and then starts walking while continuing the conversation. Again, notice how only activity changes are broadcast, but recurrent or repeated motions during the same activity are quickly recognized and notifications stopped.

On July 11th 2014, Audience completed the acquisition of Sensor Platforms. Audience believes the combination of its Advanced Voice and Multisensory Processing with Sensor Platforms’ technology places the combined company in a unique position to deliver compelling solutions based on the fusion of voice and motion.

Multisensory user interface with context awareness will bring fresh differentiation and end-user value to the market for smartphones, wearables and other mobile devices. Sensor Platforms has developed key technologies and software infrastructure in this space, and the combination of our engineering teams will enable us to rapidly scale our capabilities in context awareness-based user interface.

Audience welcomes the Sensor Platforms team and thanks all of its partners for their continued support during this transition.