Performance

Performance testing was done with NFS & SMB copying from Windows 7 computer. Which was a mistake... NFS on Windows 7 is not a very working solution. I was getting around 35MB/s transfer speeds with NFS and SSD as ZIL device. I almost decided to dump the whole thing.. but then..

I changed to use a  Linux box and voila, speed was 110MB/s constantly with NFS, but then the SMB was still not so good, around 120MB/s at start and it went down to 45MB/s in the end of the transfer on Windows 7 machine. So I decided to change the whole fs to always use sync, so that it would go through ZIL. This gave it a constant speed of 80MB/s which isn't still very good, but manageable.

Direct writing to the fs was tested with both, striped and mirrored ZIL, with sync always on. Speeds were 333MB/s with mirrored ZIL and 512MB/s with striped ZIL, which would be something we are looking for.

Reading speed has not been tested yet, I am waiting for another Infiniband card, since this one is not supported by the OS.

So as a conclusion:

• Do not use Windows 7 as NFS client
• Keep all writes synchronized (zfs set sync=always <fs>)
• Max write speed depends on your ZIL speed (my setup, 333MB/s - 512MB/s)
• SMB writes are not as fast as I hoped, but we'll see after changing to Infiniband

Some more later..