KVM storage caching modes perfomance comparison

In my everyday work i need perfomance rich KVM VMs on my workstation. I already use virtio drivers for network and storage. But what about tune it more?

First, I've moved my VMs from qcow images to raw LVM volumes.

Next, I've switced disk cache mode to 'unsafe' as most fastest. I don't care about data integrity if smth goes wrong.

But decided to compare different modes. Detailed description about cache modes could be found here.

Cache modes differ mostly by used or not "page cache" of host operating sytem and used or not "write cache" of host hardware disk (See table 1).

Table 1

	Disk Write Cache (Host)	Page Cache (Host)
writethrough	×	✔
none	✔	×
writeback	✔	✔
unsafe	Used, but ignores transfer operations	Used, but ignores transfer operations

I've done disk perfomance testing with different cache modes on raw LVM volume. Used bonnie++ benchmark. 5 tests for every cache mode. Full results here.

Table 2

	Write (sequental block), MBs	Read (sequental block), MBs
writethrough	35	2337 (page cache)
none	49	103
writeback	45	2381 (page cache)
unsafe	54	2257 (page cache)

Most fastest mode is "unsafe". But do not use "unsafe" mode if you need data integrity.
Optimal mode is "writeback", it has good write perfomance and used page cache, that increases read perfomance a lot.
Safest mode is writethrough(by default in libvirt), but slowest.

Resume:
unsafe cache mode gave me +57% disk perfomance.

Specs:
Disk: Seagate Barracuda 500G (ST500DM002-1BC142)
MB: ASUS P8H67-M EVO
CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
OS: ARCH linux 3.11.6-1-ARCH x86_64 (libvirt 1.1.4-1, qemu 1.6.1-2)