Missing a trick - deduplicated device

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
tman
Posts: 3
Joined: Thu Nov 13, 2014 12:47 pm

Thu Nov 13, 2014 12:56 pm

I'm brand new to Starwind, and have just installed the iSCSI SAN & NAS (free edition) tool. I'm trying to create a single, deduplicaed image for mounting on a remote host. I can create a device, but cannot see the option to create a deduplicated device. Maybe I'm being stupid, I don't know. I've looked at the Starwind docs, and it says, 'Create a new device', then select 'Hard Disk > Advanced Virtual > Deduplicated disk device'. Well, all I see when I select 'Add Device' is 'High Availability Device', 'Virtual Hard Disk', 'Raw Device', 'Virtual CD/DVD' and 'Virtual Tape'. Am I totally missing something here?

Also, the exact version of the software I've downloaded is Starwind iSCSI SAN & NAS v6.0.6399 (build 20140213). Is this what I should be using?

Thanks in advance. Just need some guidance here.
thefinkster
Posts: 46
Joined: Thu Sep 18, 2014 7:15 pm

Thu Nov 13, 2014 4:31 pm

Deduplication is only available on Thin provisioned images with LSFS enabled. There is a settings box for LSFS settings that allows you to check deduplication on/off.

I think in build 7354 (latest release); the LSFS bug still exists in that the SPSPX files keep growing on each synchronization to the point you'll run out of space. You'll also need to set aside enough space (I think maximum of 3x) for LSFS at this point.

StarWind guys, what ETA for getting LSFS fixed up and working so we can have the performance gains that LSFS theoretically brings to the table?
tman
Posts: 3
Joined: Thu Nov 13, 2014 12:47 pm

Thu Nov 13, 2014 4:47 pm

Thanks. I appreciate the reply. Should this be in the Free edition too? When comparing the free/paid version differences, dedup shows as being in the free version. Still can't see the option though. You talk about a later build than I've got, yet I download the latest version from the link on the Starwind website. Is there another link I should have used?

The dedup requirements actually sound a bit heavy. 3x the space for dedup - is that based on the LUN size? I have a single 12TB LUN (Virtual HD / Snapshot & CDP device) assigned to a host currently as I couldn't find the dedup device option, and had to urgently create the LUN. If you say there are bugs with dedup, would it still be better to use the dedup features on the host mounting the iSCSI LUN (it's WS2012R2, so has dedup built in)
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Tue Nov 18, 2014 10:18 am

Should this be in the Free edition too?
Yes, it should
When comparing the free/paid version differences, dedup shows as being in the free version.
Can you drop me the link where you saw that please?
You talk about a later build than I've got, yet I download the latest version from the link on the Starwind website. Is there another link I should have used?
The latest build is 7354. You can always check it here: http://www.starwindsoftware.com/customer-page
The dedup requirements actually sound a bit heavy. 3x the space for dedup - is that based on the LUN size?
Dedul is included into Log-Structured File System (LSFS) device, which logs all the writes, thus consuming more space.

I hope that helped.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
tman
Posts: 3
Joined: Thu Nov 13, 2014 12:47 pm

Thu Nov 27, 2014 10:16 am

Looks like I'd actually installed SW6.0, which doesn't have the dedup feature. Is there a straightforward upgrade path to v8.0, or do I need to do an uninstall/re-install (I'd rather not have to if possible).

Thanks
Mordock
Posts: 4
Joined: Thu Oct 09, 2014 7:19 pm

Sat Nov 29, 2014 8:13 pm

So let me get this straight. Thin provisioning uses LSFS, a logging technology that logs all writes and never deletes anything, thus the lsfs files grow until they fill up the disk and the LUN crashes. So "Thin Provisioning" actually uses more disk space than Thick Provisioning. (I have a 30GB LUN that is now using 150GB of disk space). Dedup requires LSFS, so it also requires more disk space than simply using Thick Provisioning.

WHY IS THIS EVEN AN OPTION IF IT DOESN'T WORK AT ALL?????
thefinkster
Posts: 46
Joined: Thu Sep 18, 2014 7:15 pm

Mon Dec 01, 2014 1:01 pm

As far as I know, this is a bug in all v8 builds (up to 7354).

Can StarWind support confirm?
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Wed Dec 03, 2014 11:23 am

Looks like I'd actually installed SW6.0, which doesn't have the dedup feature.
Actually v6 has dedup feature as weel:) It works on old engine but still. What I`d recommend you to do is to us v8 non-LSFS devices. That is temporary work-around obviosuly.
Is there a straightforward upgrade path to v8.0, or do I need to do an uninstall/re-install (I'd rather not have to if possible).
Unforrtunatelly you`ll need to remove everything first and after recreate everything from scratch
So let me get this straight. Thin provisioning uses LSFS, a logging technology that logs all writes and never deletes anything, thus the lsfs files grow until they fill up the disk and the LUN crashes. So "Thin Provisioning" actually uses more disk space than Thick Provisioning. (I have a 30GB LUN that is now using 150GB of disk space). Dedup requires LSFS, so it also requires more disk space than simply using Thick Provisioning.
I`ll gladly comment that. The reason is pretty obvious - it shouldn`t work in this way. LSFS should "clean up" the disk space by removing unused data blocks. The problem hasn`t appeared in our test-lab before the release, and it appeared after v8 was released. We need a month to remove this issue from our software.
as far as I know, this is a bug in all v8 builds (up to 7354).
That is correct. As I mentioned above - next build that will fix it will be updated within a month.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Wed Dec 03, 2014 5:03 pm

Gentlemen,

Good news! We got the fix in beta build. Please refer to this thread, and let us know if that helped:
http://www.starwindsoftware.com/forums/ ... =17&t=3947
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Fri Dec 19, 2014 7:40 pm

Hello everyone! May I wonder if you guys had any chance to test the beta build? Do you have any results to share please?
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
Post Reply