💾 Archived View for 0x80.org › gemlog › 2021-06-21-ipfix.gmi captured on 2021-12-03 at 14:04:38. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
Internet Protocol Flow Information Export (IPFIX) is an IETF protocol, as well as the name of the IETF working group defining the protocol. It was created based on the need for a common, universal standard of export for Internet Protocol flow information from routers, probes and other devices that are used by mediation systems, accounting/billing systems and network management systems to facilitate services such as measurement, accounting and billing. The IPFIX standard defines how IP flow information is to be formatted and transferred from an exporter to a collector.[1] Previously many data network operators were relying on Cisco Systems' proprietary NetFlow technology for traffic flow information export.
quoted from wikipedia. I wanted to experiment a little with it and found the following library
https://github.com/DominoTree/rs-ipfix
I found it lacking some features from IPFIX RFC and it panicked on some tests, so I decided to contribute to the library and fork the code.
https://github.com/q6r/rs-ipfix
Did many changes to the library most notably
Support custom fields definitions
IPFIX supports custom fields such as when using Gigamon, nprobe, sonicwall, alcatel, ..etc these devices export custom fields, and the
spec defines how to handle those with the various cases they have, the library implements the ability to support any required types to
parse them correctly.
Can parse variable size fields
Some custom fields by the mentioned devices are exported with variable size such as HTTP_POST, and others by nprobe defined in spec section-7.
Minimal memory usage
I have minimized the memory usage by reusing memory instead of cloning unless necessary.
Major redesign and performance gains
Redesigned the code to be more performant, and minimized the code by using nom-derive and moving to a newer version of nom
Due to how the state is handled in the parser it can't be multi-threaded, mainly options template and templates states. They can be lifted outside of the paresr and wrapped in a RWLock, and other thread-friendly containers to then make the code threaded/concurrent/...etc and increase the performance.
Benchmark done in Freebsd 13.0-Release, Intel(R) Core(TM) i5-10300H CPU @ 2.50GHz, 8 ncpu
lolcalhost :: ~/src/rs-ipfix ‹main› » cargo bench
Finished bench [optimized] target(s) in 0.02s
Running unittests (target/release/deps/rsipfix-9fa70b6ad5fb7f7d)
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
Running unittests (target/release/deps/parse-b022c9e09e73d70a)
running 1 test
test parse_data ... bench: 39,335 ns/iter (+/- 670)
test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured
Crate can be found in