Solaris / Openindiana trashed here comes Linux

It seemed to be so that the Solaris / Openindiana solution was not stable enough to bring this NAS to life.... here comes Linux

I chose to test the NAS by installing basic Ubuntu from mini installation (12.10) and upgrade Kernel to 3.7.6

Here's the list what I have in it now

• Linux Ubuntu 12.10 with Kernel 3.7.6
• Infiniband & opensm in connected mode
• EnhanceIO (hd cache doing caching for software RAID6) (https://github.com/stec-inc/EnhanceIO)
• 2x SSD RAID1 for cache device (55GB of usable cache)
• 4x 2TB in RAID6 storage (ext4) (3.6TB of storage)
• Samba for Windows connectivity
• NFS for Virtual Machines
• Webmin and Ajenti for web management

I have tested speeds directly on the NAS and through Infiniband. Below is a test from Bonnie++ and directly dd:ing to the disk

Version 1.96

Sequential Output

Sequential Input

Random Seeks

Sequential Create

Random Create

Size

Per Char

Block

Rewrite

Per Char

Block

Num Files

Create

Read

Delete

Create

Read

Delete

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

K/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

/sec

% CPU

bb

31G

1129

98

427376

26

240289

28

5761

99

664467

48

+++++

+++

16

+++++

+++

+++++

+++

+++++

+++

+++++

+++

+++++

+++

+++++

+++

Latency

15154us

396ms

2729ms

7362us

32064us

2667us

Latency

259us

347us

628us

269us

10us

396u

Writes

time dd if=/dev/zero of=test.file bs=10M count=10000

10000+0 records in

10000+0 records out

104857600000 bytes (105 GB) copied, 304.926 s, 344 MB/s

real    5m5.893s

user    0m0.076s

sys     1m13.032s

time dd if=/dev/zero of=test.file bs=10M count=3000


3000+0 records in
3000+0 records out
31457280000 bytes (31 GB) copied, 79.0217 s, 398 MB/s

real    1m19.040s
user    0m0.008s
sys     0m18.900s

time dd if=/dev/zero of=test.file bs=10M count=1000

1000+0 records in

1000+0 records out

10485760000 bytes (10 GB) copied, 21.2707 s, 493 MB/s

real    0m21.273s

user    0m0.000s

sys     0m6.500s

Reads

time dd if=test.file of=/dev/null bs=10M count=10000

10000+0 records in

10000+0 records out

104857600000 bytes (105 GB) copied, 233.494 s, 449 MB/s

real    3m53.500s

user    0m0.028s

sys     2m17.232s

time dd if=test.file of=/dev/null bs=10M count=3000

3000+0 records in

3000+0 records out

31457280000 bytes (31 GB) copied, 54.6859 s, 575 MB/s

real    0m54.692s

user    0m0.016s

sys     0m28.104s

time dd if=test.file of=/dev/null bs=10M count=1000

1000+0 records in

1000+0 records out

10485760000 bytes (10 GB) copied, 16.5363 s, 634 MB/s

real    0m16.553s

user    0m0.012s

sys     0m9.480s

I'll add some Infiniband speeds later

T