Short Course: RF Sensing for Wireless AI Perception: Theories, Algorithms, and Applications - Part 3 of 6

The past decade has witnessed the conceptualization and rapid development of wireless sensing, i.e., sensing using RF signals such as Wi-Fi, mmWave, UWB, RFID, etc. Remarkable advances have been achieved in both academia and the industry. Take Wi-Fi sensing as an example. Wireless sensing has turned Wi-Fi devices from a pure communication platform to a ubiquitous sensing infrastructure. Wi-Fi sensing leverages ambient Wi-Fi signals to analyze and interpret environmental contexts, achieving human sensing in a wireless, contactless, and sensorless way without using cameras, wearables, or dedicated sensors. Wi-Fi sensing is revolutionizing many fields like healthcare, home/robot automation, elderly care, smart cars with applications such as presence detection, vital sign monitoring, sleep monitoring, fall detection, gait recognition, and gesture control, just to name a few. To date, Wi-Fi sensing has been successfully commercialized as world-changing products (e.g., Verizon Home Awareness, Wiz SpaceSense, LinkSys Aware, all partnered with Origin AI, an industry leader in Wi-Fi sensing), and there are still many more innovative applications to further explore. This sensing capability adds a brand new dimension to the functions, capabilities, and applications of all Wi-Fi systems, inaugurating a new WLAN sensing protocol called IEEE 802.11bf and spawning an emerging paradigm of integrated sensing and communication, an increasingly hot topic and the next big move in wireless communication systems. Besides Wi-Fi, other RF technologies, especially millimeter-wave (mmWave) signals have also been extensively exploited for non-contact sensing, fostering applications of pose estimation, imaging, vital sign monitoring, etc., using low-cost, compact mmWave radars. In the broad context of machine perception, wireless sensing technologies extend conventional modalities beyond the visible spectrum (color or infrared images) to the RF spectrum (Wi-Fi, mmWave, 5G, UWB, etc), enabling a new Wireless AI Perception that works in absolute darkness, through occlusions, and with privacy protection. Over the years, a range of theories, models, methods, and applications have been proposed and developed to achieve these technological advances. Regardless of the underlying modalities, wireless sensing mostly leverages channel information, commonly represented as the Channel Impulse Response (CIR) or Channel State Information (CSI), which characterizes how RF 1 signals interact with the environment and the targets therein during propagation. While the wireless channel has been well modeled for wireless communication, sensing with channel measurements on commodity devices like off-the-shelf Wi-Fi or compact indoor mmWave radars involves various new challenges. Many conventional techniques used in radar signal processing and wireless communications do not directly apply and need to be upgraded. For example, channel parameter estimation of Time-of-Flight, Angle-of-Arrival, Doppler Speed, etc., becomes difficult, if not prohibitive, due to the limited bandwidth, small antenna arrays, significant phase offsets on commodity Wi-Fi and/or compact radar devices. Therefore, in addition to further development of traditional sensing techniques, brand new theories and methods have been proposed, such as statistical electromagnetic approaches. On the other hand, as deep learning demonstrates impressive effectiveness in various domains, deep wireless sensing (i.e., wireless sensing with deep learning) also attracts numerous attention, and remarkable advances have been achieved to tailor data processing, feature extraction, and model design, particularly for the unique characteristics of wireless data, e.g., complex, high-dimensional, time-frequency domains, non-visible, etc. As witnesses to, and also important participants of such historical advancements, we believe that it is the right time and of great interest to deliver this increasingly hot topic of wireless sensing to the signal processing community. Many design principles of related algorithms are signal-processing based, but wireless sensing has not been systematically introduced or embraced by the broader signal-processing community at ICASSP. Wireless sensing is a suite of increasingly popular cross-disciplinary research areas that such sister communities as biomedical engineering, computer vision, communications, and ubiquitous computing have been embracing with open arms in recent years. Yet the key building blocks touch multiple technical areas of the signal processing community—it is truly “Signal Processing Inside” and a perfect fit for the ICASSP audience. This short course will raise more awareness of the exciting research and development opportunities in the signal processing community on wireless sensing, stimulate discussions and explorations, and help the signal processing community play strong roles in this emerging area. The vital roles of signal processing to be discussed in the tutorial can help students appreciate the importance of signal processing through timely and appealing examples, fostering an opportunity to engage signal-processing researchers and industry technologists in this promising yet technically challenging area. The goal of the course is to introduce (1) the concepts, fundamental principles, promising applications, and major challenges of wireless sensing, with a focus on Wi-Fi sensing and mmWave sensing; (2) how signal processing techniques are tailored to extract contexts of interest from often noisy and weak RF sensing signals, e.g., extremely weak breathing/heartbeat motions under NLOS conditions and multipath-rich environments. This includes algorithms and techniques adapted from traditional channel parameter estimation methods to fit the sensing signals as well as statistical sensing approaches designated for wireless sensing; (3) how to apply deep learning techniques and design new RF-tailored neural networks for the unique wireless sensing data, including aspects of data representation, data augmentation, and generation, feature extraction, model design, etc.