$Id: TM_HOWTO 160 2006-12-19 02:02:57Z gregor $ TIME MACHINE CONFIGURATION The Time Machine is customized by a configuration file which contains a number of user configuration options. This includes general options such as logfile names, capture device settings as well as the configuration of storage classes with the respective parameters. By default, the configuration file is called `tm.conf' and is read from the current working directory. This can be overridden by the -c command line option (see TIME MACHINE COMMAND LINE OPTIONS section below). The configuration file is structured in sections. Options for a section are grouped by curly brackets and are separated by semicolons. There is exactly one main section, and a number of class configuration sections. For better orientation, see the example tm.conf file in the TM package. The indexes are currently NOT configureable through the config file. Currently connection4, connection3, connection2 and ip indexes are defined and configured. If you want to change this edit Storage.cc and search for 'INDEX CONFIGURATION' The configuration sections and options are as follows. * main section General settings for the Time Machine. The following options are available in the main section: device "" Directs the Time Machine to start capturing packets from the specified device. read_tracefile "" Instead of fetching packets from a capture interface, the Time Machine can also read a trace file in libpcap format and act like these packets come from the network. The packets from the file are read in as fast as they can be processed. filter "" Applies a global filter to the packets to be captured. This filter is evaluated before any classification takes place. It is specified in tcpdump(1) format. bro_connect_str ":" Instructs the Time Machine to connect to a Bro running on , TCP port via the Bro communication protocol. The Bro connection is established by the Time Machine upon TM startup. Note that at this time it is not possible to have Bro set up the connection. TM uses the Broccoli library for communcation with Bro. console 0|1 Determine whether a command line interface should be displayed on the controlling terminal of the tm process. This CLI can be used to issue TM commands (see below). daemon 0|1 Run the timemachine as a daemon in background mode. Incompatible with console. workdir "" Determine the working directory where all class storage files and index files should be kept. This directory must exist upon TM startup. indexdir "" Path, absolute or relative to `workdir' (see above), where the Berkeley DB files of the disk indexes will reside. This directory must exist upon TM startup. It is a performance gain to place the index database files on different disk than the class storage files. logfile "" Direct log output to the named file. log_interval Specify the frequency of statistical output to the logfile (log interval in seconds). conn_timeout Determine the inactivity timeout in seconds for connections before they are deleted from the TM's connection table. max_subscriptions Maximum number of subscriptions to allow. 0 for no limit. queryfiledir "" Path, absolute or relative to `workdir' (see above), where query result files will be created. This directory must exist upon TM startup. rmtconsole 0|1 Start a remote console listener. This will listen on a network socket for incomming connections. When connected a remote user can issue timemachine commands as could be done from the locale console. I.e. use telnet to connect. Please note, that this connection is not authenticated and not encrypted. Use with care. rmtconsole_port The local port to listen for incomming remote console connections. Default is 42042. rmtconsole_listen_addr The IP address to listen for incomming remote console connections. Default is 127.0.0.1. Hostnames cannot be used. A value of 0.0.0.0 will listen on all interfaces. tweak_capture_thread priority|scope|no This option can only be used on FreeBSD. It will tweak thread scheduling parameters for the capture thread (see Documentation/TUNIG) priority ... Increase the capture thread's priority to realtime priority scope ...... Set the capture thread's contentio scope to system scope. no ......... Do tweak scheduling parameters (default). * class section A class section in the configuration file is started by class "" and followed by a set of options grouped by curly brackets, individually separated by semicolons. generally can be expressed by suffixes `K' or `k', `M' or `m' and `G' or `g' for Kilo, Mega and Giga, respectively. The following options are known to this section: filter "" Define the filter that is used to determine the packets that go to this class. Exactly as the main section filter, this is in tcpdump filter string format. precedence Whenever a packet matches two or more classes with the same filter string (see above), the highest class precedence number determines which class the packet goes to. cutoff |no For this class, stop recording when more than bytes have been transmitted by any single connection. The keyword `no' disables cutoff for this class. disk Allocate disk storage of bytes for this class. Files for this storage are kept in `workdir' (see above). filesize Any of the files that make up the disk storage is bytes in size. mem Allocate RAM storage of bytes in size. pkts_to_disk 2 The moment packets are to be evicted from the RAM buffers to disk, this number determines how many packets to move at a single step. CONNECTION TABLE The Time Machine knows of connections. A connection herein is defined as a flow of packets characterized by the 5-tuple of (layer 4 protocol, source ip, source port, destination ip, destination port); for protocols other than TCP and UDP source and destination ports are not applicable (they are set to zero in the connection's identifier). Connections are bidirectional, i.e. packets in the `forward' and `return' direction are accounted to a single connection. Associated with every connection is a set of variables that keep track of the state of the connection: number of bytes, number of packets transmitted by the connection, and timestamp of last packet of the connection. A connection table entry is also optionally associated a flag to suspend the connection size cutoff, and is optionally marked to be subscribed to a TM client. The TM keeps state of connections in a connection table in RAM. Old connections are evicted from this table when their last packet arrival time is more than a configured timeout in the past (see configuration option `conn_timeout' above). The eviction of connections can be inhibited for a selective connection by issuing the command suspend_timeout (see TIME MACHINE COMMANDS below). SUBSCRIBING **CURRENTLY NOT IMPLEMENTED / TESTED** A Time Machine user can subscribe for a connection. This means it requests to be delivered all future packets for this connection without having to query explicitly for them. Subscription is automatically requested whenever the upper boundary of a query time range (see `query' command above) is given as 0 (zero). A subscription is valid as long as the connection's state is kept in the TM's connection table (see CONNECTION TABLE above). TIME MACHINE COMMANDS User commands can be issued to the Time Machine either on the local console (the controlling terminal) line interface of the tm process ( see also `console' option in CUSTOMIZATION above) or by connecting to the remote console e.g. usign telnet (telnet localhost 42042). In future versions one will also be able to issue commands by sending Bro communication protocol messages containing the command. The TM knows of the following commands: query [ ] ::= feed tag | to_file "" ::= index "" ::= start end | mem_only Query the indexes for the given index key. The result of a query can either be send to a remote bro system or to a file in the local filesystem. The index to query is specified by the keyword index followed by the name of the index. This name corresponds to the name that is returned by the getIndexNameStatic() method. Examples are connection4, connection3, ip, etc. Finally the key to search is specified by . The sematics of the key spec is defined by the index itself. For example a valid spec for connection4 would be "tcp 1.2.3.4:80 5.6.7.8:88" The enables one to restrict the search. When mem_only is specified, only the index entries stored in RAM are searched and only packets from the memomry ringbuffer are returned. The timestamps enable you, to specify a timespan. Only packets falling in this timespan will be returned. The timespan has not been tested yet!!! Examples # query to_file "file1.pcap" index connection4 "tcp 1.2.3.4:80 5.6.7.8:1025" # query to_file "file1a.pcap" index connection4 "tcp 1.2.3.4:80 5.6.7.8:1025" mem_only # query to_file "file2.pcap" index connection3 "tcp 1.2.3.4 5.6.7.8:1025" # query to_file "file2a.pcap" index connection3 "tcp 1.2.3.4 5.6.7.8:1025" start 1163668495 end 1163669900 # query to_file "file3.pcap" index connection2 "1.2.3.4 5.6.7.8" # query to_file "file4.pcap" index ip "1.2.3.4" After issuing these queries the specified files will be present in the queries directory containing the packets matching the query **CURRENTLY NOT IMPLEMENTED / TESTED** For a query specified by a connection, end may be supplied 0 (zero) or -1. For 0 the query automatically subscribes to this connection. For -1 the connection is being subscribed to and additionally the connection's inactivity timeout is suspended, i.e. the connection will not be evicted from TM's connection table when it reaches the configured inactivity timeout (see conn_timeout configuration option in TIME MACHINE CONFIGURATION). suspend_cutoff " : :" unsuspend_cutoff " : :" Suspend resp. remove the supension of the cutoff for one connection. If a connection is supended, all packets will get recorded and the cutoff value is ignored suspend_timeout " : :" Inhibit the eviction of the specified connection from the connection table (as described in the section CONNECTION TABLE above). unsuspend_timeout " : :" Remove the `suspend_timeout' flag on the connection so that it will get evicted from the connection table as soon as the regular timeout mechanism comes into effect (also see the section CONNECTION TABLE above). Example: # supend_cutoff "tcp 1.2.3.4:80 7.8.9.1:1042" bro_connect ":" connect to Bro on , TCP port using the Bro communication protocol. Not yet supported bro_connect (re)connect to the Bro configured in the TM configuration file. Not supported. show conn "tcp 1.2.3.4:80 7.8.9.1:1042" Display information available on the specified conneciton in the TM's connection table (see CONNECTION TABLE above). show conn sample Display a sample of the newest and oldest connections from the TM's connection table (see CONNECTION TABLE above). TIME MACHINE COMMAND LINE OPTIONS The Time Machine accepts the following command line options. Command line options override the according configuration file settings. -i Directs the Time Machine to start capturing packets from the specified device. Cf. `device' configuration directive in TIME MACHINE CONFIGURATION section. -r Read packets from specified tracefile rather than Cf. `read_tracefile' configuration directive in TIME MACHINE CONFIGURATION section. -f Apply global tcpdump(1) filter. Cf. `filter' configuration directive in TIME MACHINE CONFIGURATION section. -c Read configuration file (see TIME MACHINE CONFIGURATION section above) from specified file rather than from tm.conf in the current working directory, which is the default.