Testing ZFS vs ext4 with sysbench (Google Cloud)
The results:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
+----------------------------------------------------------------------------+ | EXT4 | +-----------------------------+---------------+---------------+--------------+ | | read_write | update_index | read_only | +-----------------------------+---------------+---------------+--------------+ | SQL statistics: | | | | | queries performed: | | | | | read: | 390488 | 0 | 665266 | | write: | 111564 | 210258 | 0 | | other: | 55783 | 0 | 95038 | | total: | 557835 | 210258 | 760304 | | transactions: | | | | | total | 27891 | 210258 | 47519 | | tps | 15.49 | 116.81 | 26.4 | | qps: | 309.84 | 116.81 | 422.33 | | | | | | | Latency (ms): | | | | | min: | 57.5 | 0.73 | 2.75 | | avg: | 516.36 | 68.49 | 303.06 | | max: | 1943.78 | 624.89 | 794.55 | | 95th percentile: | 746.32 | 150.29 | 475.79 | +-----------------------------+---------------+---------------+--------------+ | ZFS | +-----------------------------+---------------+---------------+--------------+ | | read_write | update_index | read_only | +-----------------------------+---------------+---------------+--------------+ | SQL statistics: | | | | | queries performed: | | | | | read: | 642236 | 0 | 1148574 | | write: | 183472 | 294325 | 0 | | other: | 91741 | 0 | 164082 | | total: | 917449 | 294325 | 1312656 | | transactions: | | | | | total | 45867 | 294325 | 82041 | | tps | 25.48 | 163.51 | 45.57 | | qps: | 509.6 | 163.51 | 729.17 | | | | | | | Latency (ms): | | | | | min: | 46.81 | 2.61 | 2.95 | | avg: | 313.97 | 48.92 | 175.53 | | max: | 1045.32 | 332.37 | 589.83 | | 95th percentile: | 458.96 | 116.8 | 287.38 | +-----------------------------+---------------+---------------+--------------+ |
The ZFS numbers shows between 1.5x to 2x better QPS and TPS and better overall latency. While this is good and all, I am not entirely sure whether O_DIRECT is the best flush method on Google storage. I would assume that GCP persistent disks are POSIX compliant and if that is the case, I’d also need to test O_DSYNC.
- Measure encryption performance against LUKS.