Skip content, jump to navigation.

Time Machine

The time machine is a joint project of the Technische Universität Berlin, the Technische Universität München, and the ICSI (University of California Berkeley). It is open-source and published under the BSD license.

TU München TU Berlin ICSI

Outline

There are times when it would be extraordinarily convenient to record the entire contents of a high-volume network traffic stream, in order to later "travel back in time" and inspect activity that has only become interesting in retrospect. Two examples are security forensics—determining just how an attacker compromised a given machine—and network trouble-shooting, such as inspecting the precursors to a fault after the fault.

To perform this task efficiently, the packets are first stored in a ring buffer in the memory (RAM), later the packets are copied to (hard) disk. This allows the time machine to smoothen capture bandwidth peaks in memory and store huge amounts of traffic on disk, covering several days of network traffic. The time machine is designed to work in Gbps environments.

Since it is not feasible to capture the complete load of a fully utilized Gbps link to disk, the time machine utilizes a mechanism called "connection cutoff" to reduce the the amount of data to process. This "connection cutoff" only records the first X bytes of every monitored connection (identified via the 5-tupel of source and destination IP and Port and the transport protocol). Indeed this approach it does not impair the analysis capabilities (unless the cutoff is set to low) because most of the "interessting" data is located in the first few packets of a connection. The effiency of this approach comes from leveraging the heavy-tailed nature of network traffic: because the bulk of the traffic in high-volume streams comes from just a few connections.

To take full advantage of this recording it is import to be able to quickly locate certain packets. For example one might be interested in all packets of a specific connection or all packets from one IP address. This is achieved by indexing stored packets. The indexes to create can be specified, for example one could create indexes for the connection 5-tupel, for IP address pairs, for IP addresses, etc. One can than issue a queries for a specific index to the time machine and the time machine will lookup the query in its index and will return all stored packets matching the query.

It is planned to add a feature that will enable the time machine to directly interact with the Bro intrusion detection system (www.bro-ids.org). Thus the Bro system can request certain packets or connections from the time machine.

up top

Documentation

up top

Download

Please note, that the current release of the time machine is in an early development stage. Bug reports and comments on the functionality and handling of the time machine and its documentation are appreciated. Please do not hesitate to send an email with your question or comment to tm@lists.net.t-labs.tu-berlin.de.

Developer release: Download tm-20061220-0.tar.gz

Most notable changes since 20061111:

(Be sure to subscribe to tm-announce.)

previous releases:

If you are experiencing packet losses, you might perhaps want to take a look at our recommendations for best packet capturing systems.

up top

Users Mailinglist

For up-to-date Informations on the Time Machine project, new versions, and improvments please be sure to subscribe to:

tm-announce mailinglist subscription page

up top

Contributors

Authors:

Contributors:

All of us can be reached via the time machine list: tm@lists.net.t-labs.tu-berlin.de

up top

Publications

Conferences and Workshops

  1. Gregor Maier, Robin Sommer, Holger Dreger, Anja Feldmann, Vern Payson, Fabian Schneider. Enriching Network Security Analysis with Time Travel. In SIGCOMM '08: Proceedings of the 2008 conference on Applications, technologies, architectures, and protocols for computer communications, (Location: Seattle, WA, USA), ACM Press, New York, NY, USA, August 2008. (To appear) details
  2. Stefan Kornexl, Vern Paxson, Holger Dreger, Anja Feldmann, Robin Sommer. Building a Time Machine for Efficient Recording and Retrieval of High-Volume Network Traffic. In IMC '05: Proceedings of the 5th ACM SIGCOMM Internet Measurement Conference, (Location: Berkeley, CA, USA), Pages 267-272, ACM Press, New York, NY, USA, 2005. details

Misc (Posters, Talks, etc.)

  1. Gregor Maier, Anja Feldmann, Fabian Schneider, Robin Sommer, Vern Paxson, Holger Dreger. Time Machine. Poster at Deutsche Telekom Laboratories, Off-Site Meeting, August 2007. details
  2. Gregor Maier, Stefan Kornexl, Anja Feldmann, Vern Paxson, Robin Sommer, Fabian Schneider, Bernhard Ager, Holger Dreger. Time Machine. Poster at Deutsche Telekom Laboratories, Annual Review Meeting, February 2007. details

Thesis

  1. Holger Dreger. Operational Network Intrusion Detection: Resource-Analysis Tradeoffs. PhD Thesis Technische Universität München, Munich, Germany, 2007. details

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

It has been automatically generated using the bib2html program.