Performance tests say ‘maybe’.

Tests captured the number of GET, SET, INCR and MSET ‘requests per second’ vs ‘number of clients’. Tests were initiated using the ‘redis-benchmark’ utility running on the same machine as the instance of Redis/Fredis.net being tested.  See ‘Testing Setup’ below for further details

Obviously Microsoft Redis trounces both OSX Redis (3.0.7) and Fredis.net. However, Fredis.net is in the same ballpark as OSX Redis, which is not insignificant when comparing a program written in a managed, functional language against one with a reputation for speed written in C. I need to be careful about what I say I’m comparing, these tests also compare IO completion ports + Windows VS Kqueue + OSX. Also, Fredis.net uses a simple .net Dictionary to store string data (Redis strings are really byte arrays), Redis functionality not implemented by Fredis.net may require something more sophisticated than a simple Dictionary but which comes with a cost. What can be said is that you are not shooting yourself in the foot by choosing a managed functional language for performance critical applications.

Fredis.net does consume more CPU that either Microsoft Redis or Redis on OSX, it also consumes much more memory, this can be seen when running the performance tests. As an experiment I added a GC.Collect call in Fredis.net FlushDB processing after which memory use was comparable to either version of Redis (I removed the GC.Collect after this experiment, manually triggering GC collections is usually the dumb thing to do).

Future Fredis.net development

The next version of Fredis.net will, probably, use SocketAsyncEventArgs, which should improve Fredis.net performance, and maybe CPU and memory usage/GC pressure.

Testing setup

Tests were run on a late 2013 MacBook Pro, with 16 GB of ram and a Core i7-4960HQ CPU. Microsoft Redis andFredis.net were run on Windows 10 running via bootcamp. OSX Redis was run on El Capitan. Both Microsoft and official Redis had persistence disabled.

A key space (the number of distinct keys) of 64 and a message size of 1K were chosen for no other reason than they were neither very small nor large. Tests were run for 1, 2, 4, 8 … 1024 clients.

Output from a script like the one below was piped to a text file, which was in turn processed by a small F# program which produced charts using FSharp.Charting to generate the graphs.

redis-cli flushdb

echo “redis-benchmark -r 64 -d 1024 -t ping,set,get,incr,mset -n 100000 -q -c 1”

redis-benchmark -r 64 -d 1024 -t ping,set,get,incr,mset -n 100000 -q -c 1

redis-cli flushdb

echo “redis-benchmark -r 64 -d 1024 -t ping,set,get,incr,mset -n 100000 -q -c 2”

redis-benchmark -r 64 -d 1024 -t ping,set,get,incr,mset -n 100000 -q -c 2