Pflua and high performance packet filtering

| | Comments ()

Time to write other post! This time I will comment on one of our most recent projects here in Igalia, a high performance packet filtering toolkit written in Lua.

Several weeks ago I received a phone call coming from Juan. Andy was looking for some mate ready to jump in a new opportunity related to high performance networking, hypervisors, packet filtering and LuaJIT. Hey! this mix sounded great so I joined Andy and we went ahead.

Six weeks later, and with Diego joining the project too, one first implementation (Pflua) of the libpcap packet filtering language (pflang), together with the proper testing code and benchmarking (Pflua-bench) went live.

Along those weeks, I hacked in bindings/FFI implementation, performance/benchmarking, testing stuff and kernel-space to user-space code adaptation (Linux BPF JIT wrapped as a dynamic library!). With this post I will share a quick overview of the project and the proper links to explore it in detail.

As mentioned, Pflua implements the libpcap packet filtering language, which we allude as 'pflang' for short. Pflua is a high performance packet filtering toolkit implemented in LuaJIT (a tracing compiler for the Lua language). Together with Pflua we developed Pflua-bench too, a benchmarking implementation of pflang.

Pflua and Pflua-bench were developed for Snabb Gmbh, the company behind the Snabb Switch network appliance toolkit. You can read on this project or getting in touch with Luke and other Snabb hackers in the snabb-devel forum. They are working in very interesting and challenging use cases where virtualization and Software Defined Networking (SDN) are pulling more and more networking into servers. At the same time, user-space networking software is out-performing kernel-space software too.

In this point, you could be interested in the inner technical details for Pflua and Pflua-bench. If so, I would recommend to read the last post of my colleague Andy. He introduces the project with a great compiler hacker perspective. If you are in a hurry I would highlight the following points:

  • Pflua implements two compilation pipelines or execution engines. It is able to generate Lua code starting from a pflang expression or starting from Berkeley Packet Filter VM. With the first engine you reach great flexibility to craft complex/expert filters. Moreover, your final filters in Lua will be free from some limitations and constraints in BPF, such as extra bound checks or converting to host byte order.
  • Pflua-bench compares 5 pflang implementations: the user-space BPF interpreter from libpcap (libpcap), the old Linux kernel-space BPF compiler (linux-bpf), the new Linux kernel-space BPF compiler (linux-ebpf), BPF bytecode cross-compiled to Lua (bpf) and pflang compiled directly to Lua (Pflua). You can see our benchmarking results and comparative analysis here.
  • Pflua seems to be an acceptable implementation of pflang and, in many circumstances, Pflua is the fastest pflang implementation by a long shot.

As mentioned, Pflua was developed for Snabb Gmbh around an Open Source virtualized Ethernet networking stack and it has the right potential to become one high performance packet filtering toolkit in SDN solutions (forwarding and control planes).

We are incubating this project in Igalia. Feel free to follow the development and drop us a mail if you want to support this project or you are just using it!


comments powered by Disqus