Isilon IQ and S series rocks

Just finished about a month of resiliency and performance testing of the Isilon OneFS-based clustered storage. In our lab we had six IQ9000 nodes, six 5400S nodes and an accelerator node with dual 10Gbit interfaces. I tested NFS, CIFS, FTP, HTTP, and even got a hold of their beta code to test iSCSI. While I can’t share any benchmarks (NDA) I have to say, their gear is awesome. It’s super-bulletproof. You can powerfail drives, nodes, or the back-end Infiniband switches and the thing just keeps on rocking. The selectable parity can even allow you to take more than one node failure and keep trucking along.

The  GUI is not a huge DHTML crap-infested monstrosity like some I could mention (ahem rhymes with Gun – as in what you want when you use it). It gets the job done, shows you the stats you really would care about like “How full is my cluster in total?” or “How much traffic am I seeing to all nodes?” etc.. The command line is also very functional. Most of the commands start with ‘isi’ like “isi stat” that will show you the cluster status.

One rather dramatic display of simplicity is when you setup a new node. You just connect it to the infiniband and within two keystrokes on the node’s front panel you have the new node join the cluster. The cluster expands the space available and get’s you going right away.

~ by aliver on December 18, 2009.

4 Responses to “Isilon IQ and S series rocks”

  1. We’re currently in the process of evaluating Isilon’s solution (as a Netapp house).

    I have some (as yet unfounded) concerns about random IO performance with their architecture.

    Did you happen to encounter any such issues, or re-run your performance benchmarks on their later hardware/OS?

    • I did run FIO, CrystalDiskMark, IOMeter and some simple “dd” style tests on their 6.0 beta. Random I/O was very respectable and on par with NetApp. It’s especially good on 10Gbit Ethernet. Single stream sequential performance has some room for improvement, but doesn’t trail the competition. I’d love to see 500MB/s on a 5 node cluster, but that’s tough to achieve without major tuning. It’s more like 350MB/s. That still keeps up with the pack. Now I believe that they are at an RC level with 6.0. The iSCSI stats were even better and the SMB/CIFS code is much better and gets you out from underneath the Samba permissions hassles you’ll have with 5.x. I’d highly recommend deploying 6.x if you are in an Windows heavy environment. If you are mainly NFS and iSCSI then you should be fine with 5.x. I work in a very NetApp heavy shop and I’ve been a NetApp customer for about 12 years. I know their products very well. I can tell you that Isilon will free you from the infernal aggregate space irritation and gives me nearly 30% better raw to usable ratios because of the lack of aggregate/volume/snap reserves you will no longer need to allocate. You still want to have at least 10% free on your cluster, but any modern filesystem needs some free space. OneFS is no different than WAFL in this respect. The storage primitives underneath are MUCH different and heavily advantage Isilon. NetApp has been mostly good to me over the years. I don’t want to badmouth them. However, they are suffering from a lot of legacy design issues, nowadays. They have really failed to bring the Spinnaker / GX technology to bear on the pain that customers feel over subdividing the snot out of their precious filer space. True that OnTap 8.1 will make that a lot better by giving you much bigger aggregates and (if you can start over on a new cluster) giving you a global namespace. However, without the “CoralFS” technology from the GX line allowing you to stripe your aggregates and scale horizontally, they are going to still be playing catch up to Isilon and others who started with the true scale-out cluster technology.

  2. That’s a lot of great info – thank you.

    Your view on where the vendors are at the moment is very much in line with my thinking; Certainly Netapp building on their legacy architecture is a concern to me, though I’m unsure as to how much (if any) difference 8.1 will bring from a fundamental architecture point of view. We’re currently running GX (with Coral) so I have a (perhaps unfair) tendency to judge 8.1 by the failings I’ve seen in GX before seeing a production system. Coral was a great idea, but I wonder whether it’s suited to NFS (without P-NFS) given the cluster tax associated with the additional latency of Ethernet when accessing remote data.

    It may prove sensible for them to ‘take a breather’ from file striping across nodes so they can focus no working on feature parity with their traditional OS?

    I can see Netapp’s rationale for doing what they’re doing both technically and commercially but wonder whether the difficulties they have had with the integration of Spinnaker have done them more harm than is first apparent.

    Let’s hope not, as they’re a great company that generally serve us very well.

    We’ll be testing Isilon with the latest OS because we need some of the bells and whistles it brings. However, Isilon’s challenge seems to be the opposite of Netapp’s; building features on top of their architecture (rather than changing architecture and keeping features). I’ve no idea how good the combined entity of EMC/Isilon will be at doing that, or whether any implementation details of their scale out platform will get in the way of those additions.

    You’d like to think that EMC’s expertise in feature rich NAS devices would combine well with Isilon’s knowledge of scale out storage. However, things never seem to work out quite so simply – Spinnaker is a prime example of that!

    • Thanks for the insightful comment. I agree with you about Isilon having the opposite challenge to NetApp. They don’t have vfilers, dedupe, or anything like SnapLock. There are a raft of other features they lack, as well. Most are on their roadmap, but they too will need to deliver them rather than just talk about them. Here’s hoping the EMC sugar-daddy relationship will benefit them in the same way it has done for Data Domain, and that they retain the same level of autonomy. I vacillate on what NetApp should do to freshen up; they certainly aren’t incapable of doing great things.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: