Unsolved
This post is more than 5 years old
56 Posts
0
1420
Expanding symchg functionality
Using symchg, it's possible to measure the amount of changed tracks on a device. This can be very useful when you want to measure the impact Thin Provisioning would have, without having to actually implement it. 5773 code isn't even required: symchg has been available for ages.
The only issue is that symchg measures single tracks, and the extent size of Virtual Provisioning is 12 tracks. This could cause the measurements to not accurately reflect the amount of actual disk space a device would use, were it a thin device.
The extreme example would be an application that writes every thirteenth track, and leaves the other tracks alone. symchg would report only that only 1/12th of all tracks were ever written, but a thin device would have allocated the maximum amount of space for that device.
If it would be possible to expand the functionality of symchg so that an optional "extent size" argument could be given, symchg could be used to accurately measure the effects that Thin Provisioning would have had. In case of an extent size of 12, symchg would group 12 tracks together, and report whether the group was ever written to or not.
Any opinions? Would it be useful to issue an enhancement request for this? Or am I overlooking something?
The only issue is that symchg measures single tracks, and the extent size of Virtual Provisioning is 12 tracks. This could cause the measurements to not accurately reflect the amount of actual disk space a device would use, were it a thin device.
The extreme example would be an application that writes every thirteenth track, and leaves the other tracks alone. symchg would report only that only 1/12th of all tracks were ever written, but a thin device would have allocated the maximum amount of space for that device.
If it would be possible to expand the functionality of symchg so that an optional "extent size" argument could be given, symchg could be used to accurately measure the effects that Thin Provisioning would have had. In case of an extent size of 12, symchg would group 12 tracks together, and report whether the group was ever written to or not.
Any opinions? Would it be useful to issue an enhancement request for this? Or am I overlooking something?
xe2sdc
2 Intern
2 Intern
•
2.8K Posts
0
November 2nd, 2008 08:00
Davidy8s
154 Posts
0
November 3rd, 2008 13:00
But isn't the example you are citing an actual use of only "one" of the hypothetical 13 tracks due to V.P.?
I'm with Stefano here, and will defer to those with more knowledge on the subject but my reading of VP is that you are fronting nnn tracks to allow an allocation to proceed but the acutal usage is much lower normally. When the normal exceeded, V.P. can satisfy the extra requested storage.
We may have to poke around a bit for this one.
Stefano? I think we can give Jurjen the nod for question of the week, don't you?
Dave Yates
EMC TSE3
Benevolent Host S/W & Mainframe Forum Moderator
Jurjen_Oskam
56 Posts
0
November 5th, 2008 07:00
Suppose I create a normal, non-thin device of 9 GiB. On current Symmetrix arrays, this device consists of 147456 tracks of 64 KiB each. (Depending on its configuration, Solutions Enabler can translate this to tracks of 32 KiB each, because older Symmetrix arrays had track sizes of 32 KiB. I'll use track sizes of 64 KiB for this example though.) When a host issues a write to this device, that write will land on one of the 147456 tracks. DeltaMark will set a flag for that lucky track, to indicate that the track has been written to. It doesn't matter how small the write was, or whether the write was in the beginning of the track, the middle, or the end. Even if just one byte was changed in a track, the flag gets set. (It is possible for one host write to land on *more* than one track. In that case, DeltaMark sets the flag for all tracks that were written to.)
Now suppose I have 5773 code, and I create a Thin Device (tdev) with a maximum size of 9GiB. Initially, this device won't take any real disk space. (Not exactly true, but good enough for this example.) This 9GiB is not divided in chunks of 64 KiB each, as the "real" device above was. Instead, it's divided in 12288 chunks of 768KiB each. (Note that 12 times 64 equals 768. A thin device is divided in chunks of 12 tracks.) When a host issues a write to the thin device, that write will land on one of the 12288 chunks (extents). Since this is a thin device, the real disk space wasn't allocated yet. However, the host issued a write, which means it's now time to dynamically allocate some real disk space to store the data the host wants to write. A thin device always allocates real disk space in chunks of 768 KiB, even if the host only wrote, say, 8 KiB. After this one write, one of the 12288 chunks of the thin device is now allocated. If another write is done to the thin device and it happens to land on the already allocated chunk, it's simple: the data can be stored in the already allocated space without allocating any more real disk space. If a write happens to land in a non-allocated chunk, 768 KiB of real disk space is first allocated for that chunk before the data can be stored. (A chunk that was once allocated is never given back during the lifetime of the thin device.)
Using DeltaMark, it's possible to gather statistics from normal, non-thin devices to see how they would behave if the device were a thin device. Just look at the total number of flags set on a device: this tells you how much tracks were actually written to, and therefore tells you how much real disk space would have been allocated were it a thin device. The problem with this is that DeltaMark uses chunks of 64 KiB, and Thin Provisioning uses chunks of 768 KiB. Suppose a host only issues writes to every thirteenth track. DeltaMark will report that of the 147456 tracks, only 147456/12 = 12288 were ever written to. This indicates that only 1/12th of the device was ever written to. However, if that same device were a thin device, each of those 12288 tracks is part of a different extent of 768 KiB. This means all 12288 extents of 768 KiB each would have been allocated, and my 9 GiB thin device would take up 9 GiB of real disk space.
If it were possible to give symchg an "extent size" argument, you could tell symchg to divide a device not in chunks of 64 KiB, but in chunk sizes of your own choosing (probably in tracks, or multiples of the track size). With an "extent size" argument of 12 tracks, you could let DeltaMark report exactly what would be allocated were a device a thin device because a DeltaMark "chunk" would exactly match a thin device "chunk."
I hope this is a better description.
xe2sdc
2 Intern
2 Intern
•
2.8K Posts
0
November 5th, 2008 09:00
(a lot of lines below) --S-- you can use the "
OMG I can only say you did your homeworks !!!!!!
I'm still trying to understand everything you wrote !!! I'll have a look at other threads in the meanwhile, just to relax my mind
Catch you later, Jurjen...
Message was edited by: Benevolent Host S/W & Mainframe Forum Moderator
David Yates
Davidy8s
154 Posts
0
November 5th, 2008 12:00
You again win the Host S/W prize for question of the day. I've done a bit of digging, and now know a bit more. Even so, I've rubbed the SE Developer genie bottle once again and let him out
This is very much similar to the allocation of mainframe DASD when not allocated with release.
BTW, the expanded example was quite helpful, thank you. We're looking into it. My tentative read is that your logic is sound here and worth investigating. Let's see what we come up with.
Best regards,
Dave Yates
EMC TSE3
Benevolent Host S/W & Mainframe Forum Moderator
&
infrequent editor of Stefano's NUMEROUS posts
xe2sdc
2 Intern
2 Intern
•
2.8K Posts
0
November 5th, 2008 13:00
Message was edited by: Benevolent Host S/W &
Mainframe Forum Moderator
David Yates
I prefer ----8<-----8<-----8<-----8<---- :-)
@>--'--,--'-- A rose for you, Dave
Davidy8s
154 Posts
0
November 6th, 2008 06:00
xe2sdc
2 Intern
2 Intern
•
2.8K Posts
0
November 6th, 2008 06:00
xe2sdc
2 Intern
2 Intern
•
2.8K Posts
0
November 6th, 2008 06:00
xe2sdc
2 Intern
2 Intern
•
2.8K Posts
0
November 6th, 2008 06:00
Davidy8s
154 Posts
0
November 6th, 2008 06:00
Jurjen, I forwarded your inquiry up the SE chain of support to get some thoughts on your idea.
Stefano, I think I'll give that ascii rose to Jurjen as a prize for his "questions of the day" !
Best regards,
Dave Yates
EMC TSE3
Benevolent Host S/W & Mainframe Forum Moderator
Jurjen_Oskam
56 Posts
0
November 6th, 2008 11:00
But back on topic: thanks for asking the SE team! I'm very curious about what they think of it. I think it would be valuable to be able to invoke symchg against a standard, non-thin non-virtual device, and be able to calculate what the disk usage would have been if it were a thin virtual device, so I hope they'll agree.
Regards,
Jurjen
Jurjen_Oskam
56 Posts
0
November 17th, 2008 22:00
xe2sdc
2 Intern
2 Intern
•
2.8K Posts
0
November 18th, 2008 06:00
Davidy8s
154 Posts
0
November 18th, 2008 06:00
I did follow up, it turns out that he - my local SE guy - is consulting with the other developers about this. I know that is a "very thin" update, but we are investigating. I'll check in again and we'll see if they have discussed this in detail yet.
Best regards,
Dave Yates
EMC TSE3
Benevolent Host S/W & Mainframe Forum Moderator