Multi-Connectivity and Edge Computing for Ultra-Low-Latency Lifelike Virtual Reality

We explore a novel multi-user mobile VR system for streaming scalable 8K 360-degree video at high reliability and immersion fidelity, and low interactive latency, via a synergistic integration of scalable 360-degree tiling, dual-band millimeter wave (mmWave) and Wi-Fi transmission, and edge computing. High rate directed mmWave links are studied to send VR viewport-specific high-quality enhancement layers of the 360-degree content to the individual users, while Wi-Fi broadcast of the base layer of the entire 360-degree panorama is sent to all users, to augment the system's reliability. The viewport-specific enhancement layers can comprise compressed and raw 36-degree tiles, decoded first at the edge server. We explore the joint optimization of the mmWave access point to user association, the choice of 360-degree tiles to be transmitted decompressed, the allocation of mmWave data rate across the compressed tiles in a viewport-specific enhancement layer, and the allocation of computing resources at the edge server and user devices. Our objective is to maximize the minimum delivered VR immersion fidelity across all users, given transmission, latency, and computing constraints. We demonstrate that our framework can enable a significant improvement in immersion fidelity (8 dB to 10 dB) and spatial resolution (8K vs. 4K), over MPEG-DASH that uses Wi-Fi transmission only. We also show that an increasing number of raw 360-degree tiles are sent, as the mmWave link rate or the edge server/user computing power increase, exploring rigorously here the fundamental interplay between computing and communication capabilities, end-to-end system latency, and delivered VR immersion fidelity.