Model of data handling for in-depth analysis of network traffic

The article suggests a new object model of data for in-depth analysis of network traffic. In contrast to the model used by most existing network analyzers, such as Wireshark or Snort, the core of our model supports data streams reassembling and next processing of them. Analysis continues even in case of loss of individual packets. The model supports both stateless and statefull network protocols. State of protocol machine may be stored in a special memory location related to each connection of relevant type. The article stated the requirements for network traffic analysis tools. A high speed data processing in resource-limited environment is the main requirement for online systems. Offline analyzer operates with a network trace of the fixed size, so the processing speed is not so important. It becomes possible to visualize the data structure disassembled. Offline analyzer also traces how network streams formed from packets. The model provides an interface for parsers implemented in the form of dynamic link libraries. It also provides a convenient universal mechanism for binding parsers so one can develop parsers independently. This is achieved through the use of special functions (recognizers) allowing for the data itself to determine which parser should be used. It is crucial for parsers to be compatible with both online and offline analyzers. Our model also provides processing of modified, e.g. compressed or encrypted, data. Unlike Snort the model supports nested tunneling protocols. Actually it forms the basis of the infrastructure for in-depth analysis of network traffic.