NAS Performance - SPEC SFS 2014

Evaluation of the system as a storage node on the network can be done using multiple approaches. As a simple NAS accessed from a single client, Intel NASPT would work very well. There are other artificial benchmarking programs such as IOMeter and IOZone (all of which are used in our dedicated NAS reviews). However, when it comes to file servers used in business scenarios, business metrics make more sense. For example, a database administrator might wonder how many simultaneous databases could be sustained on a given machine? An administrator in a software company might want to know how many simultaneous software builds could be processed on the machine if it were to be used as a storage node. SPEC SFS 2014 allows us to evaluate systems based on such business metrics.

Prior to discussing about the various business scenarios, let us take a look at the test setup (including details of the testbed and how the file server itself was configured.

Solution Under Test Bill of Materials

  • Advatronix Nimbus 400 (ASRock Rack C2550D4I with 16GB RAM)
  • AnandTech NAS Testbed (64GB RAM, 1GB to each member VM)
  • Netgear GSM7352S Ethernet Switch

Component Software

  • Advatronix Nimbus 400 system running Windows Server 2012 R2 Essentials
  • Load generators running on AnandTech NAS Testbed (10x Windows 7 VMs in a Windows Server 2008 R2 Hyper-V Installation)

Storage and File-Systems

  • Advatronix Nimbus 400 - 4x OCZ Vector 128GB SSDs : Storage Spaces with Parity Space
  • AnandTech NAS Testbed - NTFS partitions created at OS install time on OCZ Vertex 4 64GB SSDs

Transport Configuration

  • Advatronix Nimbus 400 - 2x 1GbE LAN Ports in 802.11ad LACP to Netgear GSM7352S
  • AnandTech NAS Testbed - 11x 1GbE LAN Ports to Netgear GSM7352S (1x management, 1x to each of 10 VMs)
  • All SMB benchmark traffic flowed through the Netgear GSM7352S network switch

The four business metrics that we will be looking at today include:

  • Database
  • Software Build
  • Video Data Acquisition (VDA)
  • Virtual Desktop Infrastructure (VDI)

The database and software build categories are self-explanatory. The VDA profile refers to usage of a storage node as a recording target for streaming video (usually from IP cameras). The VDI profile refers to the number of virtual desktops / virtual machines that can be supported using the file server as a storage node for the virtualization infrastructure.

Database

The following graphs show the requested and achieved op rates for the database workload. Note that beyond four databases, the gap between them is more than 10% - this automatically means that the storage system is unable to support more than four databases concurrently. In all the workloads, it is the latency which decides the suitability and not the bandwidth available.

Database Workload - Op Rates

Database Workload - Latency and Bandwidth

The SPEC SFS 2014 benchmark also provides a summary file for each workload which contains data additional to whatever is graphed above. The summary for the database workload is available here

Software Build

The software build workload simulations suggest that handling more than 7 concurrent software builds off the NAS would not be a good experince,

Software Build Workload - Op Rates

Software Build Workload - Latency and Bandwidth

The report summary for the software build workload is available here

Video Data Acquisition

The Advatronix Nimbus 400 has no trouble with handling up to 10 data sources in the video data acquisition benchmark profile.

VDA Workload - Op Rates

VDA Workload - Latency and Bandwidth

The report summary for the VDA workload is available here

Virtual Desktop Infrastructure

VDI again presents a big challenge. While the 8-bay U-NAS build could not handle even one virtual desktop, the Nimbus 400 starts off OK. However using a Windows Storage Spaces volume for VM data store proves to be a stumbling block when attempting to run two or more virtual machines.

VDI Workload - Op Rates

VDI Workload - Latency and Bandwidth

The report summary for the VDI workload is available here

Performance Metrics - Storage Subsystem Miscellaneous Aspects and Concluding Remarks
Comments Locked

18 Comments

View All Comments

  • lwatcdr - Wednesday, August 12, 2015 - link

    I would really like to see some data using FreeNAS and Windows as well as Ubuntu. With the cost of drivers so low both of the NAS systems offer a huge amount of data for home or business.
  • wintermute000 - Thursday, August 13, 2015 - link

    nowhere near enough free RAM for Freenas. 1Gb per Tb is the recommendation. With modern drives @ even RAIDZ1, you do the math
  • wintermute000 - Thursday, August 13, 2015 - link

    Sorry, no idea why but I read it as 4Gb not 4x4Gb, my bad
  • Brutalizer - Sunday, August 16, 2015 - link

    For zfs, it is recommened to use 1GB ram per 1TB disk space - only if you use deduplication. If not, 4GB in total is enough. Zfs has a very efficient disk cache, if you only have 2GB ram in your server you will not any disk cache, which is no big deal actually. Myself used 1GB ram server for a year with solaris and zfs for a year without problems. Lot of ignorance about zfs. Try it out yourself with 2-4 GB ram server and see it will work fine.
  • DanNeely - Wednesday, August 12, 2015 - link

    Aside from the front panel having USB3, this case looks identical to one I bought from Chenbro a few years ago for my DIY nas. I'd be a bit concerned about the quality. The plastic locking half of the handle on one of the drive sleds popped when I pulled it out a month or two ago to add an additional drive to my setup. The metal half was still usable to pull the drive out and it appears to be held in place securely from the rear; but the normal latch mechanism is obviously not working any more.
  • Anonymous Blowhard - Wednesday, August 12, 2015 - link

    I'm concerned about the presence of a Marvell SATA controller + FreeBSD based OS like FreeNAS, since there's been many reports of drives performing poorly or dropping out of ZFS pools under high I/O.
  • bobbozzo - Friday, August 14, 2015 - link

    Remove USB card and insert IBM 1015 RAID card. Hope cabling is compatible.
  • SirGCal - Wednesday, August 12, 2015 - link

    I personally have two 8-drive, one RAID6 and one RAIDZ2, both running Ubuntu. Both of them also run swifter then this. Curious.
  • Ratman6161 - Wednesday, August 12, 2015 - link

    Data is a little stilted because the Asrock is using an 8C 2750 vs the 4 C in the Advantronix - so anything CPU sensitive is not really fair - particularly since the Advantronix is available with the 8C CPU.
    That said, I sort of doubt many people will be running DB's on this sort of machine. And the other tests seem to indicate that the faster CPU doesn't really buy you anything.

    And...4 SSD's in a RAID 5? The cost per GB for doing things that way is very high compared to spinning disks and if its being used in a home setting the performance of the SSD's is not needed. comparing prices online I could get 4x WD Black 750 GB drives for almost $100 cheaper than the 4 x 128 GB Vectors. Take a look at the read performance of the two units. Theoretically the Asrock with an 8 drive array should get better read speed than the Nimbus 400 with only 4. But it doesn't leading me to believe that a lot of the SSD's performance is wasted. Spinning disks is probably the most cost effective way to go with these.
  • lwatcdr - Wednesday, August 12, 2015 - link

    Encryption uses up a good amount of CPU time.

Log in

Don't have an account? Sign up now