Detecting and Interpreting Changes in Scanning Behavior in Large Network Telescopes

Network telescopes or “Darknets” receive unsolicited Internet-wide traffic, thus providing a unique window into macroscopic Internet activities associated with malware propagation, denial of service attacks, network reconnaissance, misconfigurations and network outages. Analysis of the resulting data can provide actionable insights to security analysts that can be used to prevent or mitigate cyber-threats. Large network telescopes, however, observe millions of nefarious scanning activities on a daily basis which makes the transformation of the captured information into meaningful threat intelligence challenging. To address this challenge, we present a novel framework for characterizing the structure and temporal evolution of scanning behaviors observed in network telescopes. The proposed framework includes four components. It (i) extracts a rich, high-dimensional representation of scanning profiles composed of features distilled from network telescope data; (ii) learns, in an unsupervised fashion, information-preserving succinct representations of these scanning behaviors using deep representation learning that is amenable to clustering; (iii) performs clustering of the scanner profiles in the resulting latent representation space on daily Darknet data, and (iv) detects temporal changes in scanning behavior using techniques from optimal mass transport. We robustly evaluate the proposed system using both synthetic data and real-world Darknet data. We demonstrate its ability to detect real-world, high-impact cybersecurity incidents such as the onset of the Mirai botnet in late 2016 and several interesting cluster formations in early 2022 (e.g., heavy scanners, evolved Mirai variants, Darknet “backscatter” activities, etc.). Comparisons with state-of-the-art methods showcase that the integration of the proposed features with the deep representation learning scheme leads to better classification performance of Darknet scanners.