In recent years, breakthroughs from the field of deep learning have transformed how sensor data (e.g., images, audio, and even accelerometers and GPS) can be interpreted to extract the high-level information needed by bleeding-edge sensor-driven systems like smartphone apps and wearable devices. Today, the state-of-the-art in computational models that, for example, recognize a face, track user emotions, or monitor physical activities are increasingly based on deep learning principles and algorithms. Unfortunately, deep models typically exert severe demands on local device resources and this conventionally limits their adoption within mobile and embedded platforms. As a result, in far too many cases existing systems process sensor data with machine learning methods that have been superseded by deep learning years ago.

In this tutorial, I will survey the state of progress towards solving a collection of open research problems that collectively aim to understand the role deep learning will play in various forms of resource constrained computing moving forward. Predominately, I will discuss the two central themes of this area. First, the study of how deep learning can be used to model human behavior and context. Early modeling findings are showing exciting gains in recognition accuracy and robustness for key learning problems (e.g., activity recognition, context models) that underpin sensor systems. And second, the approaches being developed to reduce (and regulate) the system resources deep models consume to levels acceptable for constrained platforms. Already, at least at inference-time, efficiency results are demonstrating the feasibility of adopting deep learning on familar embedded processors and commodity wearables. 



Nic Lane holds dual academic and industrial appointments as a Senior Lecturer (Associate Professor) at University College London (UCL), and a Principal Scientist at Nokia Bell Labs. At UCL, Nic is part of the Digital Health Institute and UCL Interaction Center, while at the Bell Labs he leads DeepX -- an embedded focused deep learning unit at the Cambridge location that is part of the broader Pervasive Sensing and Systems department. Before moving to England, Nic spent four years at Microsoft Research based in Beijing. There he was a Lead Researcher within the Mobile and Sensing Systems group (MASS). Nic's research interests revolve around the systems and modeling challenges that arise when computers collect and reason about people-centric sensor data. At heart, Nic is an experimentalist and likes to build prototype next-generation of wearable and embedded sensing devices based on well-founded computational models. His work has received multiple best paper awards, including two from ACM UbiComp (2012 and 2015). Nic's recent academic service includes serving on the PC for leading venues in his field (e.g., UbiComp, MobiSys, SenSys, WWW, CIKM), and this year he will act as PC-chair of HotMobile 2017. Nic received his PhD from Dartmouth College in 2011.