Rethinking Anomaly Detection: From Theory to Practice

The field of anomaly detection is rapidly evolving with numerous algorithms and applications. However, the fundamental principles behind anomaly detection are less frequently studied. This thesis aims to evaluate algorithm performance, assess the reliability of autoencoders, and introduce a novel application in power grid load estimation.

A comprehensive comparative study on real-world data identifies that only a few algorithms are necessary for effective anomaly detection. Specifically, $k$-nearest neighbors and extended isolation forest perform well in detecting common anomalies.

Autoencoders are increasingly used for anomaly detection, especially in computer vision due to their feature extraction capabilities. However, we analyze the underlying assumptions of autoencoder-based anomaly detection and conclude that these assumptions do not hold in practice, making autoencoders unreliable for this purpose.

Applying anomaly detection to the energy sector, we utilize statistical process control and binary segmentation to identify measurement errors and power rerouting in load estimation. By filtering these anomalies, we improve the reliability of power grid load estimates in the Netherlands. Enhancing load estimation through interpretable methods helps reduce unused grid capacity, providing necessary flexibility for the ongoing energy transition.