The BRIDGE Framework: Robustness and Resilience in Decentralized Learning

In many applications, machine learning involves managing data across multiple devices without the availability of a central server, necessitating a decentralized learning approach. In such settings, nodes are susceptible to failures from malfunctions or cyberattacks, which can undermine traditional learning algorithms. This paper addresses the robustification of decentralized learning amidst Byzantine failures, where nodes can arbitrarily deviate, threatening system stability. Prior works have utilized ad-hoc methods akin to robust statistics; however, we propose a formal integration of robust statistical principles into the learning process for a more systematic approach. We introduce BRIDGE, a scalable Byzantine-resilient decentralized machine learning framework, designed to fortify resilience and offer structured analysis against Byzantine behaviors. BRIDGE comes with algorithmic and statistical convergence guarantees for both strongly convex and select nonconvex problems. Our experiments validate BRIDGE's scalability and effectiveness, underscoring its robustness and showcasing the benefits of incorporating robust statistics into decentralized learning systems formally.