Anyone with HA and 5.7 ?

[camealy]

I would love to hear some real world feedback and experiences before taking the plunge. 5.6 has been very stable, but HA is still slow.

Thanks!

Kurt

[Kevin Brown]

We're all waiting for you Kurt!

Regards
Kevin

[oxyi]

LOL, great, just upgraded my HA to 5.7...

I will run a IOmeter again, since I ran it just before on 5.6, now upgraded to 5.7, we shall see...
my setup is slow, so let see how slow it can get with 5.7

[kmax]

Upgrade went fine. No issues.

Sync is much faster....around 2-3x of previous versions.

[oxyi]

HMMMM !!

Upgrade is fine. NO issues whatsoever.
Sync is indeed much faster, did a fast sync.

Speed wise is the part that's WTF to me now..

Since I had IOmeter opened before I upgraded, right after I did it, I did a test again..

This is based on RealLife 60%Rand 65%Read.

Before on 5.6
IO per sec - 1230.07
MB per sec 10.08

After 5.7
IO per sec - 306.52
MB per sec 2.51

I know my setup is slow, but this doesn't seem right to me... it looks like I am getting 1/4 of performance compared to before.
I will do IOmeter again, and report back.

[camealy]

yikes

[oxyi]

Can someone please confirm that I am just hallucinating ?

I just did it again.

I/O per second 488.43
MB per sec 4.00

attached the zipfile contained the CSV files from IOmeter, can anyone tell if anything changed ?
results1.csv is the first one I did back at 15th.

[kmax]

Not sure what to say...however I created a simple 50 GB HA target (with 1 GB write-back cache) and attached it to one my hyper-v nodes just for testing purposes, no CSV or anything like that. Connected multipath to it using 2 1Gb interfaces round robin. The starwind servers have 10Gb CX4 interfaces to the switches.

First, did a simple file copy from the local node (which can sustain 300 MB/sec reads). It copied the files at the max...meaning both iSCSI network interfaces were at 99% sustained so about 200MB/sec for the duration of the copy.

Fired up IO meter. Did 8 worker threads, all in one access pattern, 8 worker threads. Showing 9000 IOPs. 123 MB/sec...which makes sense considering the iSCSI interfaces are hovering around 50% utilization each.

I guess the only other interesting thing I can add is the fact that the Starwind nodes are showing 7,000 IOP's a piece. Should be around 4,500-5,000 because I know from experience that my cluster usually averages a measly 500 IOPs in normal operation.

[oxyi]

Kmax, so are you saying that your performance is about the same ?

Can anyone else do more testing, I am curious to find out whether it's my environment

[anton (staff)]

This one sucks. You should have all numbers at least DOUBLED when sliding away from 5.6 to 5.7 so I guess something goes wrong. Please drop us a message so somebody from techies would jump on your head to take a look what's really broken. Thank you!

HMMMM !!

Upgrade is fine. NO issues whatsoever.
Sync is indeed much faster, did a fast sync.

Speed wise is the part that's WTF to me now..

Since I had IOmeter opened before I upgraded, right after I did it, I did a test again..

This is based on RealLife 60%Rand 65%Read.

Before on 5.6
IO per sec - 1230.07
MB per sec 10.08

After 5.7
IO per sec - 306.52
MB per sec 2.51

I know my setup is slow, but this doesn't seem right to me... it looks like I am getting 1/4 of performance compared to before.
I will do IOmeter again, and report back.

[anton (staff)]

200 MB/sec for long write is a good number. If you've filled write back cache entirely i mean.

I/O meter results are not that good. What's your I/O pattern?

With 5.7 HA performance should be nearly doubled. So with 4.5-5K IOps per node in a properly configured Round Robin (switch node I/Os number is not that large) you should expect 9-9.5K IOps for 2 node config. What you actually have I think

Not sure what to say...however I created a simple 50 GB HA target (with 1 GB write-back cache) and attached it to one my hyper-v nodes just for testing purposes, no CSV or anything like that. Connected multipath to it using 2 1Gb interfaces round robin. The starwind servers have 10Gb CX4 interfaces to the switches.

First, did a simple file copy from the local node (which can sustain 300 MB/sec reads). It copied the files at the max...meaning both iSCSI network interfaces were at 99% sustained so about 200MB/sec for the duration of the copy.

Fired up IO meter. Did 8 worker threads, all in one access pattern, 8 worker threads. Showing 9000 IOPs. 123 MB/sec...which makes sense considering the iSCSI interfaces are hovering around 50% utilization each.

I guess the only other interesting thing I can add is the fact that the Starwind nodes are showing 7,000 IOP's a piece. Should be around 4,500-5,000 because I know from experience that my cluster usually averages a measly 500 IOPs in normal operation.

[kmax]

Yeah, I'm maxing out the iSCSI connections with bluk reads/writes. No problems there. And the sync channel will do over 350MB/a sec now where before it was around 120. It is getting into the area of what my storage is capable of...considering I'm running this while 20 other VMs are running against the SAN.

Don't have numbers for 5.6 besides sync channel performance, but everything seems fine.

[anton (staff)]

Sync channel should push around 700 MB/sec (assuming you're on 10 GbE side) and link utilization around 80-90%. Could you please verify it on heavy dedicated write load with maximum I/O packet size?

Yeah, I'm maxing out the iSCSI connections with bluk reads/writes. No problems there. And the sync channel will do over 350MB/a sec now where before it was around 120. It is getting into the area of what my storage is capable of...considering I'm running this while 20 other VMs are running against the SAN.

Don't have numbers for 5.6 besides sync channel performance, but everything seems fine.

[oxyi]

Hi Anton,

Raw image without formatted, I can achieve 117MB per sec.

With formatted image, if I let IOmeter created a 4GB testing file on the drive, I would only get 3-6MB per sec.

What's the difference ? I can copy file into my iSCSI drive with blazing speed( bc of cache) but within the drive, if I would to copy stuffs out and make another copy of it on the same drive - transfer rate is only 3MB-6MB, which coincided with the same result of IOmeter.

Something went wrong ?

[kmax]

Can someone (probably should be Starwind) create a standard .icf file in iometer and upload it so we have something common to run?