LOBSTER: A European platform for passive network traffic monitoring

Over the past few years we have been witnessing a large number of new programs and applications which generate prolific amounts of questionable, if not illegal, traffic that dominates our networks. Hoping from one port to another and using sophisticated encoding mechanisms, such applications have managed to evade traditional monitoring tools and confuse system administrators. In this paper we present a concerted European effort to improve our understanding of the Internet through the LOBSTER passive network traffic monitoring infrastructure. By capitalizing on a novel Distributed Monitoring Application Programming Interface which enables the creation of sophisticated applications on top of commodity hardware, LOBSTER empowers a large number of researchers and system administrators into reaching a better understanding of the kind of traffic that flows through their networks. We have been running LOBSTER for more than a year now and we have deployed close to forty sensors in twelve countries in three continents. Using LOBSTER sensors •; we have captured more than 600,000 sophisticated cyberat-tacks which attempted to masquerade themselves using advanced polymorphic approaches •; we have monitored the traffic of entire NRENs making it possible to identify the magnitude (as well as the sources) of file-sharing (peer to peer) traffic.