Scalable, Asynchronous, Distributed Eigen-Monitoring of Astronomy Data Streams

This telescope will take repeated images of the night sky every 20 s, thereby generating 30 terabytes of calibrated imagery every night that will need to be co-analyzed with other astronomical data stored at different locations around the world.

Event detection, classification, and isolation in such data sets may provide useful insights to unique astronomical phenomenon displaying astrophysically significant variations: quasars, supernovae, variable stars, and potentially hazardous asteroids. However, performing such data mining tasks is a challenging problem for such high-throughput distributed data streams. In this paper, we propose a highly scalable and distributed asynchronous algorithm for monitoring the PCs of such dynamic data streams and discuss a prototype web-based system PADMINI (Peer-to-Peer Astronomy Data Mining) which implements this algorithm for use by the astronomers.