StarWind versus EqualLogic/hardware SAN

[robnicholson]

I suspect this is a simple question with a complex answer. We're in the market for a SAN as we a) need more disk space, b) want to improve flexibility in virtualisaion and expansion and c) want to mitigate against hardware failure for better business continutity.

So we've got quotes in for EqualLogic and Xiotech. Both very good with some great reviews & feedback from users. But they are not what you would exactly call cheap! In fact, "taking the micky" might pop up.

We looked at DataCore and liked the product but once again, not the price.

I happened upon StarWind whilst researching the project. Even tried the trial on a spare VM with 8TB eSATA disk enclosure. It was simple to use which I loved. Suspect there is a lot of extra complexity in the SAN world that's there to confuse.

But as for the price - well, it's wonderful. Not got UK pricing from you yet but saw some US prices and rough back of envelope calculation gives me a PowerEdge R710, dual CPU, 8GB RAM, Windows 2008, 8 x 600GB 10k 600GB (~4TB usuable with RAID) for £6000. Add in the most expensive StarWind license and I doubt I'd be spending more than £10k.

Compare this with the £25k+ that I'm been quoted for EqualLogic and Xiotech and it's not surprised I'm interested in StarWind. Heck, for £25k, I could have the same set-up in our UK and USA offices.

But... I worry about the old adage of "You get what you pay for"...

So after waffling on there for a while, this is an open and I hope honest question:

How does would StarWind on something like the above (suspect you'll say more RAM but heck, that adds another £1k) compare with hardware SANS like EqualLogic & Xiotech on the following?

1. Performance (way top of list)
2. Software reliability
3. Hard reliability (appreciate this is mainly outside of your control but others may know)

Cheers, Rob.

PS. Also love the idea that we can have a split 4TB SAS fast drives but also mix with a cheap-as-chips eSATA disk enclosure if we get the right StarWind license.

[robnicholson]

On a similar subject - iSCSI performance. I've read more than I want to on this subject over the past few weeks but I'd sort of come to the conclusion that for our set-up (100 office type users), that the potential speed increase of fibre over iSCSI just wasn't worth the extra costs and loss of flexibity (i.e. with iSCSI, almost any PC could be immediately connected to SAN without having to worry about £££ HBA).

General wisdom was that in most instances, it was the speed of the disk system that would be the bottleneck, not how fast the files can be transferred over 1gbit/s Ethernet.

But they I came across a post on here that said:

1) Cheap high capacity SATA drives. SAS is not going to help much as network is the bottleneck here.
2) As much RAM as you can install. We'll use every free byte as a cache.
3) Two HA nodes with Write-Back cache configured. HA makes multi-gigabyte WB cache safe to use.
4) Multiple RAID sets. To ensure you can rebuild them separately.

No problem with 2-4 but #1 surprises me.

Cheers, Rob.

[Max (staff)]

I think this is not about your case,
1 GbE is capable of 120Mbps and if you have 100 clients you need multiple RAID nests which will be able to fullfill 1GbEx100clients with about 80/20 random/sequential load. This is an approximate number which is VERY overestimated (the possibility of a simultaneous 100% link use from 100 clients is miserable ) but still you will not be able to make this with 1 or 2 SATA based RAIDs

[anton (staff)]

1) Never had any business with Xiotech but I can confirm both Equallogic and DataCore are providers of a very mature technology and excellent service. Had a pleasure to have a phone conversation with Paula Long (being ex-EQL at that time) and she had really impressed me!

Both companies are between good and perfect but both are actually far far away from SMB and entry level Enterprise market you probably belong to. Can tell you why exactly in private but not here

2) I would not consider StarWind as "el cheapo" solution. You always need to calculate hidden costs. With EQL you get everything in one package "all inclusive" style: hardware, software and installation labor. With StarWind you need to pick up proper hardware, pay for it, pick up proper OS, pay for it, pick up StarWind SKU, install everything & configure and pay for StarWind. So to StarWind costs you need to add:

a) Hardware expenses

b) OS license

and

c) Human time

Are they cons or pros? Let's find out! You do get quite a lot of benefits however compared to EQL and other entirely SAN solution:

a) Hardware expenses. You can deal with the hardware you're familiar with. Have positive experience with Dell? Go Dell! Putting SuperMicro boards into all your servers during last 20 years? Go SM! Excellent flexibility and a real bargain prices on hard. 2TB hard disk still takes $100 from your budget and not $500 your SAN vendor is willing to charge you for "just because" it can do it. So picking up custom hardware would result getting more RAM, disk space and faster CPU cycles compared to entirely hardware solution SAN vendor sells under it's name.

BENEFITS: FLEXIBILITY, PERFORMANCE and PRICE.

b) OS license. You can do nothing with EQL firmware but you can perfectly re-cycle your Windows Server 2008 R2 license you've paid for once. Even more you can put say auxiliary Exchange Server on the same machine running StarWind and they would co-exist just fine. Something you can never do with EQL as it's SAN dedicated thing. Good news: StarWind can run on top of Hyper-V so you don't need to pay for OS license if you don't want to. And more good news: we'll represent Linux-based version soon. After all b) starts to get less and less important now.

BENEFITS: OS LICENSE RE-CYCLE OR RE-USE, NO PAID OS AT ALL.

c) Human time. You get ready box from EQL and you need to configure everything yourself going StarWind way. Is it good or not? Well... It depends! It's time consuming but you finally get trained engineer who understands the way our SAN works. Does it worth extra time? It's you who should decide. For me it's a benefit

BENEFITS: SAN ADMINISTRATOR IS DEALING WITH KNOWN SOFTWARE AND HARDWARE AND NOT BLACK BOX HE HAS NO CLUE WHAT TO DO WITH IF IT DOES NOT BOOT.

Let's now shoot down your list:

1) Performance. Spending less money you can always build machine running circles around EQL solution. We do scale up to 2 nodes nearly linearly now, would add 3 quite soon and grid architecture close to Fall 2011. I don't see any movements in this directly from EQL now...

2) Software we use as a backbone is Windows OS. It has MILLIONS more instances running compared to EQL firmware.

3) The same about dedicated hardware (MIPS-based AFAIR) EQL use and COTS x86 we deal with. See single Intel server mobo you'd probably pick up for your SAN appliance is tested MUCH harder then any proprietary hardware on Earth.

I would comment about performance and my vision answering your second query below...

I suspect this is a simple question with a complex answer. We're in the market for a SAN as we a) need more disk space, b) want to improve flexibility in virtualisaion and expansion and c) want to mitigate against hardware failure for better business continutity.

So we've got quotes in for EqualLogic and Xiotech. Both very good with some great reviews & feedback from users. But they are not what you would exactly call cheap! In fact, "taking the micky" might pop up.

We looked at DataCore and liked the product but once again, not the price.

I happened upon StarWind whilst researching the project. Even tried the trial on a spare VM with 8TB eSATA disk enclosure. It was simple to use which I loved. Suspect there is a lot of extra complexity in the SAN world that's there to confuse.

But as for the price - well, it's wonderful. Not got UK pricing from you yet but saw some US prices and rough back of envelope calculation gives me a PowerEdge R710, dual CPU, 8GB RAM, Windows 2008, 8 x 600GB 10k 600GB (~4TB usuable with RAID) for £6000. Add in the most expensive StarWind license and I doubt I'd be spending more than £10k.

Compare this with the £25k+ that I'm been quoted for EqualLogic and Xiotech and it's not surprised I'm interested in StarWind. Heck, for £25k, I could have the same set-up in our UK and USA offices.

But... I worry about the old adage of "You get what you pay for"...

So after waffling on there for a while, this is an open and I hope honest question:

How does would StarWind on something like the above (suspect you'll say more RAM but heck, that adds another £1k) compare with hardware SANS like EqualLogic & Xiotech on the following?

1. Performance (way top of list)
2. Software reliability
3. Hard reliability (appreciate this is mainly outside of your control but others may know)

Cheers, Rob.

PS. Also love the idea that we can have a split 4TB SAS fast drives but also mix with a cheap-as-chips eSATA disk enclosure if we get the right StarWind license.

[anton (staff)]

"Video killed the radio star!" (c) The Bugglers

With 10 GbE becoming a mainstream (and 40 GbE and 100 GbE being under development) there's no sense to invest into FC if you had never did it before. It does make sense if you have existing FC infrastructure and trained personnel. But as an initial investment I would not call this a wise decision.

Why are you surprised? I/O makes or breaks the whole system. If done locally. But we do push huge amounts of data over the wire everything starts getting pretty much complicated. Assume you have fixed budget (not very difficult to imagine, yeah?). You can either:

1) Have single node with redundant PSUs, multiple fans, battery-powered expensive cached SAS RAID card, SAS drives of a limited capacity in RAID5/6 and also limited size system RAM. Backed up with high Ah UPS.

*OR*

2) Have multiple nodes w/o redundant PSUs (cheaper), passive cooling (less chances to fail), no RAID card (cheaper), cheap high capacity SATA drives in RAID1 or 10 of a higher total capacity and more system RAM. Backed up with a pair of cheaper low Ah UPSes.

So in second case you'll have doubled systems (less chances to fail), doubled network I/O paths (less chances to fail and FASTER), distributed write-back cache of a many gigabytes (RAID on-board cache is SLOWER and is limited in size compared to system RAM) and no write-penalty RAID1 or 10 instead of bad write RAID5/6. As a result you're going to have more storage of a higher performance for less money. Sounds like "win-win" situation, doesn't it?

If you have unlimited budget you can of course go SAS everywhere. So for doubled $ per TB of storage you're going to have extra couple of % for MBps and IOps. Maybe.

On a similar subject - iSCSI performance. I've read more than I want to on this subject over the past few weeks but I'd sort of come to the conclusion that for our set-up (100 office type users), that the potential speed increase of fibre over iSCSI just wasn't worth the extra costs and loss of flexibity (i.e. with iSCSI, almost any PC could be immediately connected to SAN without having to worry about £££ HBA).

General wisdom was that in most instances, it was the speed of the disk system that would be the bottleneck, not how fast the files can be transferred over 1gbit/s Ethernet.

But they I came across a post on here that said:

1) Cheap high capacity SATA drives. SAS is not going to help much as network is the bottleneck here.
2) As much RAM as you can install. We'll use every free byte as a cache.
3) Two HA nodes with Write-Back cache configured. HA makes multi-gigabyte WB cache safe to use.
4) Multiple RAID sets. To ensure you can rebuild them separately.

No problem with 2-4 but #1 surprises me.

Cheers, Rob.

[robnicholson]

>I would not consider StarWind as "el cheapo" solution. You always need to calculate hidden costs. With EQL you get everything in one package "all inclusive" style: hardware, software and installation labor. With StarWind you need to pick up proper hardware, pay for it, pick up proper OS, pay for it, pick up StarWind SKU, install everything & configure and pay for StarWind. So to StarWind costs you need to add:

I appreciate there are additional (wouldn't quite say hidden) costs with StarWind that you don't get with EQL in terms of specificying and purchasing the hardware but equally EQL providers usually stick on a day's consultancy to install the EQL. We can specify and order a Dell PowerEdge in a few hours. Installation in the rack is as intrusive as installing EQL so no difference there. Installing Windows 2008 Server takes an hour, StarWind takes about 15 minutes! Learning StarWind interface = same as learning EQL interface.

>OS license

That £7k figure I might have quoted versus £25k for EQL included the Windows server licenses.

>not $500 your SAN vendor is willing to charge you for "just because" it can do it.

That, in a nutshell, is the key point I think.

>Performance. Spending less money you can always build machine running circles around EQL solution. We do scale up to 2 nodes nearly linearly now, would add 3 quite soon and grid architecture close to Fall 2011. I don't see any movements in this directly from EQL now...

I kind of like the idea that we can upgrade our hardware on our schedule and for relatively little cost. Trashing the RAID array with your SAN is the real sticking point as backing up and restoring that data in a small time window. However, as we could stick in a eSATA disk enclosure as temporary storage (don't forget, we have replication DR to USA incase it all goes pointy up), transfer to that, upgrade the RAID and copy back whilst still mainly live is very attractive.

With that 4TB EQL, it's "buy a new box" time if we need to expand.

>Software we use as a backbone is Windows OS. It has MILLIONS more instances running compared to EQL firmware.

But probably millions more bugs

Cheers, Rob.

[robnicholson]

>"Video killed the radio star!" (c) The Bugglers

Thumbs up for performance at lower prices. SAN really does seem to be last bastion of old mainframe pricing

>With 10 GbE becoming a mainstream (and 40 GbE and 100 GbE being under development) there's no sense to invest into FC if you had never did it before. It does make sense if you have existing FC infrastructure and trained personnel. But as an initial investment I would not call this a wise decision.

Good point and hadn't notice things like this:

http://www.ebuyer.com/product/264853?ut ... m=products

The additional 1Gbit card we were adding to the PowerEdge is £120. Wonder if the budget could stretch to 10Gbit...

Cheers, Rob.

[robnicholson]

Hmm, whilst 10Gbe cards are getting cheaper, switches are still pretty stiff.

Regards, Rob.

[anton (staff)]

You can start with building 10 GbE backbone. Just connect 2 or 3 StarWind nodes with each other keeping 1 GbE uplinks to your hypervisor servers. With combined MPIO for multiple nodes it should provide enough of bandwidth.

Hmm, whilst 10Gbe cards are getting cheaper, switches are still pretty stiff.

Regards, Rob.

[WillDental]

In our Development area, we just put in a Dell PowerConnect 8024F for just over $8,000 vs the usual $32,000 for the Cisco blade in a Cisco 6509. All 10 GbE. Had to upgradeto current code to get jumbo 9216 working well, but all is good ow . . .

[anton (staff)]

Interesting... Did you deploy StarWind already or have any plans to use it in the future?

[robnicholson]

That's one hell of a development department Whilst still fiendisly expensive (£5.5k in UK) for a 24 port switch, it does still make StarWind plus 10Gbe still way cheaper than the hardware SAN world. Can EqualLogic (iSCSI) or Xiotech (fibre) match 10GBe yet and if so, what kind of money are we looking at?

Cheers, Rob.

[Max (staff)]

I'm not sure about 5.5k here, you can also get a decent one for ~3K on ebay. Offers pop up from time to time.