Feedback for VPN Benchmark

[Qubasa]

Hey @LeeSmet, @maximevanhees

I created an open source benchmark for Peer-to-Peer VPNs and wanted to reach out for feedback.
You can find the current test results here: https://vpnbench.clan.lol/cross-profile

In particular I wanted to know if there are any mistakes with the feature matrix and the Mycelium Overview.

Thanks for taking a look :-)

[LeeSmet]

Hi, nice project :)

From a quick look I don't see any errors about the feature matrix.

I'm interested in how you've run the tests, particularly the UDP one since that should just work (tm).

[Qubasa]

Thank you, happy you like it :-)

For the UDP test I'm running it with iperf3 the exact command can be found here:

https://github.com/Qubasa/vpn-bench/blob/main/pkgs/vpn-bench/vpn_bench/iperf3.py#L42

however I have to note, that iperf3 UDP mode is buggy / has somekind of buffer / integer overflow as can be seen in the logs:

however it is still an interesting benchmark, because it means mycelium does something quite different with the UDP packets such that it triggers this bug (in comparison to other mesh VPNs).

In higher impairment profiles, more VPNs start to fail the UDP benchmark and it starts to become inconsistent which VPNs fail and which not, but the baseline is pretty stable.

[marek22k]

Is it also true that mycelium has implemented neither PFS nor replay protection?

[LeeSmet]

Currently keys are static, and we maintain the same secret. Adding ratcheting is something which is something I'd like to do in the future, though without adding too much overhead.

There is indeed no replay protection, it's assumed that the protocols handle this in the overlay, as that is technically also possible on the regular internet

[marek22k]

Currently keys are static, and we maintain the same secret. Adding ratcheting is something which is something I'd like to do in the future, though without adding too much overhead.

That's great to hear!

There is indeed no replay protection, it's assumed that the protocols handle this in the overlay, as that is technically also possible on the regular internet

The same can be said about encryption. I thought the idea behind overlay networks such as Yggdrasil or Mycelium was precisely to take this responsibility away from the protocol and, as IPsec was supposed to do, provide native security for all protocols. I always thought that replay protection was a fundamental component of a VPN or similar software.