VNX File concepts and reconfiguration

Question

Greetings.

I've been handed a system, but haven't had to do much to it in the last year or so. All the Block configuration seems so clear and easy to manage, yet the File is almost the opposite.

I might be a little long winded, so please forgive me in advance.

Here's what I have for File:

RAID5 with five 3TB disks. These are part of a Storage Pool named "File".

There are 3 LUNS in Storage Pool "File", each 1TB.

NAS-FILESRV1

NAS-FILESRV2

NAS-FILESRV3

The Storage Pool has 7831GB in unused capacity that I haven't figured out how to use.

There is also two "Storage Pool for File"

"File" is a Mapped pool.

"FILESRV1" contains 3 volumes (d14, d15, d16) and is 3TB in size.

And then there is a File System named......

"FILESRV1" which is 2TB and contains Volumes d14 and D15. I haven't yet figured out how to expand this.

Volume d16 contains 33 slices, where d14 & d15 each contain 3 slices. I'm not sure why, but might guess that this isn't good, unless this is how snapshots are stored.

And....I just love this.

"FILESRV1" is a path off of of a CIFS server named MDMH-FILESRV1.

I'm not a fan of RAID5 with large disks, so I acquired new disks and built a RAID6 Storage Pool currently named "File2" and intend on using the left over RAID5 disks with additional disks to expand my Block storage (8+2 in my capacity tier).

There is a theme here. The indvidual who configured this, named everything the same. I generally like to differentiate (like in Cisco Call Manager, mdmh_dp is our Dial Plan), and thus, I would rename File, file_sp to indicate this is a storage pool.

I found:

How to move a file system between different storage pools while keeping the exports and shares

and it seems pretty clear if I just blindly follow the steps, not quite understanding all the Command Line. But, will I need a replication file system setup for this to succeed? Can I Unisphere to do this instead (Storage, File System Migration, Migrations/File Systems/Mounts)

$ nas_replicate -create File_rep -source -fs FILESRV1 -destination -pool File2 -interconnect id=20001 -max_time_out_of_sync 10

Although I refer to Interconnect ID 20001 (verified to be the loopback interconnect), it results in:

Error 14764146700: Storage pool File2 is not found for remote system id 0.

I'd like to move everything from Storage Pool "File" to Storage Pool "File2" as seamless to users as possible. We are a 24x7 shop (a tiny little 25 bed hospital), but a brief file share outage in the evening is generally unseen by users.

I have a storage pool. Do I create a Storage Pool for File within this somehow? I don't see that...Create offers me a from Storage Pool, but not which Storage Pool. And I don't have a Template Pool that contains any available space...

Severity:	Info
Brief Description:	This storage system does not have a Template Pool that contains any available space.
Full Description:	The selected storage system does not have any potential storage available for use.
Recommended Action:	Choose a different storage system or select the Create from Meta Volume option to create an empty pool.
Message ID:	26844725269

Trying to create a Meta Pool

Severity:	Info
Brief Description:	No volumes available, only empty pool can be created.
Full Description:	All the volumes on the system might be in use currently. You can create an empty storage pool now and later extend the pool by adding volumes to it.
Recommended Action:	To make real use of a storage pool, one or more volume(s) are required. It is recommended to create the storage pool when the volumes are available.
Message ID:	26844725255

So it looks like I've got to create volumes, yet, under create volume, I've got "No volumes to create a new volume".

I'm unclear on the relationships between Storage Pool, Storage Pool for File, File System, LUN, Volume. I know just enough about linux to hurt myself but have been playing with it off and on for a few years and am comfortable slicing up a disk to make file systems, even with different formats. Windows is just super easy. For block storage, to divide a pool up into LUNs is pretty logical. A pool is made up of a bunch of physical Volumes is also pretty logical. VNX references and file leaves me somewhat confused.

We do have support for 4 more years, but every call to EMC results in a return call at 3am or a "Professional Services" reference when I ask them how to do something, so I'm feeling pretty sour about calling support.

I really appreciate any bits of knowledge that might fall my way. I appreciate references to documentation, web sites, or whatever.

Thank you so much for reading through this.

David

dynamox · Answer

David,

let's start gathering some facts about system configuration, remove storage system serial numbers frome these output (starts with APM) . Login as nasadmin to the control station and run the following commands:

nas_pool -info -all

nas_disk -l

nas_pool -size

nas_fs -l

server_df ALL

It might be easier to paste all this info in text file and attach it to your reply.

dsain1 · Answer

Thanks for the reply dynamox.   I've attached the requested.

dynamox · Answer

David, so this is what i see: Pool names: File  - empty FILESRV1 - ~3TB in size, 397G free test - empty File system: FILESRV1 - ~2TB in size, ~1TB used. Remember that on VNX you have 'block' pool(s), out of those pool(s) you carve out LUNs that you add to 'filestorage' storage group.  At that point datamovers can rescan and discover those LUNs and add them to their own pool (File, FILESRV1 and test in your case). It sounds like you have additional capacity in you block pool that you could use to create more 1TB LUNs, add them to filestorage storage group and give yourself more capacity to extend 'FILESRV1' pool.  Is the goal to completly migrate off R5 raid groups ?

dsain1 · Answer

dynamox, Yes, my goal is to 'retire' the RAID5 Storage Pool named 'File' after I figure out how to migrate the File System 'FILESRV1' to a new RAID6 Storage Pool (File2). Once I move to the new RAID6 Storage Pool, I have 3 unused disks to combine with the existing RAID5 disks to add to my Storage Pool named 'Block' which is near capacity. I have 3 Storage Pools.  One is mixed RAID named 'Block', one RAID5 named 'File' and my new RAID6 named 'File2'.  Are all of these 'Block' pools?  Block meaning what (the way I understand is a block file system is presented as raw disk and formatted differently than file systems)?  Or, is 'block' a way of referenceing every disk in the system? Storage Pool 'Block' contains all of the various sized LUNS that we hand off to VMware hosts and contains 3 RAID groups.  It's 22884GB with less than 10% free. Storage Pool 'File' contains three 1024GB LUNS and a lot of empty space. Storage Pool 'File2' has no LUNS created/assigned at this time. Is there a best practice suggesting the 1024GB size is ideal?  Is there anything wrong with allocating all the available space in the Storage Pool to the LUN? I played around a little and created 5 1024GB LUNs, assigned to Storage Pool 'File2' and assigned it to Storage Group '~filestorage' (which shows host=Celerra_VNX-Controller-A).  Some progress.  I'm starting to understand how it all nests together... Now that those LUNs are part of ~filestorage, I'm not sure where to go... Update: I found that the new LUNs cannot be part of a Storage Group, so I located the ~filesystem Storage Group and removed the new LUNs.  I'm now migrating to the new LUNs (I see the new ones have disappeared once I started the Migrate). I still want to extend the Storage Pool for File named FILESRV1, however I still get No volumes available to extended the selected storage pool. I have 2 LUNs that are part of a Storage Group and one that is Not.  Once I get that worked out, I should be pretty close... Update: However I have no Volumes available to create a new Volume when trying to create a Volume to extend the Storage Pool for File.  Seems like a Catch .22.  I found one guy talking about creating RAID groups instead of Storage Pools.  I'm not quite sure what the difference is.  Under RAID groups, we have 3, each of which is configured as a single Hot Spare disk, one of each, SATA Flash, SAS and NL SAS. One question I do have.  The File System FILESRV1 has 2TB.  I can't expand it by 512GB.  The Storage Pool FILESRV1 has 3TB.  I see under Extend with Storage Pool the only option of [FILESRV1 (388.4GB)].  I'm assuming space is lost due to perhaps partitioning, formatting, snapshots.  It seems like a pretty significant portion of disk (~611.6GB). Thank you! David

dynamox · Answer

David,are you in the US ? It will be easier if i jump on a Teamviewer session with you and we go over what you have and what you can do with it.&#xa0; You can buy me a beer at EMC World .&#xa0;

dsain1 · Answer

Good morning dynamox. Yes, I am in the US. And I wish I was going to EMC world, however our budget is really tight and we are a small shop.

I managed to get data migrating. Had to create LUNS, but not add them to a storage group. Once this is done, I'm all good. Mostly.

I'll still need to expand the Storage Pool for File, but at this point, I think I'm grasping most of the disk concepts here. Just finished reading about slices.

I'm still up in the air about Volumes. Can't create a volume because no volumes exist to create the volumes. But I've got lots of free space and 3 empty 1024GB LUNs. I've got some more reading to do. Today's a busy day, so maybe tomorrow.

David

Rainer_EMC · Answer

I agree – forget about volumes and slices – the manuals just mention MVM for old times sake Just use a file pool and let AVM do the work of slicing and stripe/concat

dynamox · Answer

i am not sure why you are working with volumes and slices, while that's an option it would be so much easier to work with pools ( and i mean pools within VNX datamovers, not block pools)

dsain1 · Answer

Long winded.&#xa0; Sorry...@dynamox @Rainer_EMC I appreciate you input above.The volume error messages are cryptic at best, and completely useless to me.&#xa0; In excluding them, I was able to find an answer.I bumbled around for a while on the internet.&#xa0; Finally I just searched for 'VNX Storage Pools for File' and came across an article (linked at the bottom) that mentions ensuring the HLU is over 15.&#xa0; In digging around I found the 3 LUNs connected had Host LUN IDs of 16, 17 and 18.&#xa0; From that I inferred that the HLU must be the Host LUN ID.&#xa0; The new LUNs I had created had lower numbers.From within the wizard to create the LUN, there is no way of specifying a HLU.&#xa0; If one then right clicks on the new LUN and selects Add to Storage Group, the simple wizard there does not give an option to create/modify/change the HLU.&#xa0; I knew that under Hosts, Storage Groups I could remove LUNs from a group, so I went there to see if I could add.&#xa0; In the Select LUNs option when right clicking on ~filestorage, I could see that I can double click the LUN to select it, whereupon it appears in the table in the lower window and the Host LUN ID field is blank.&#xa0; Clicking on it presents a drop down (a bug here prevents the arrow from displaying), where one can select the desired number.&#xa0; If you DO NOT select a Host LUN ID, one will be picked for you, starting at the lowest number, whereupon you will probably be screwed.In reading Managing Volumes and File Systems with VNX AVM from EMC, I see Chapter 3 discusses Configuring disk volumes and refers to a Gateway NAS system that connect to a VNX for Block storage system.&#xa0; I'm guessing the Data Movers are a gateway also, although the documented list specifically lists VNX VG2 and VG8.&#xa0; On page 66 of this document, in steps 1 and 3 of Provide storage from a VNX or legacy CLARiiON system to a gateway system, it indicates that Host LUN identifer must be greater than or equal to 16 for user LUNs. However the steps for Unisphere neglect to mention the selection of the HLU.&#xa0; If I were using the CLI, it's indicated as a switch.I note that the GUI is not designed for neophytes.&#xa0; If you don't know where to find the hidden HLU field, yer screwed.&#xa0; The training that was sold to us was consumed by an engineer that left us 2 weeks after completing it a year ago, so I've been unable to get further education (hopefully our future director is willing to spend the money).&#xa0; School of hard knocks for me.So these are the instructions I wrote up for future users of our system: Log into Unisphere. Choose the System List tab.&#xa0; Choose VNX-Controller-A Hover over the Storage tab and click LUNs. Choose Create at the bottom of the LUNs page.&#xa0; Set to Pool Set Storage Pool for new LUN to FILE (our SP name). Set volume size to 1TB or 1024GB.&#xa0; All LUN sizes must match in ~filestorage. Click Apply.&#xa0; Click Yes in the pop-up box.&#xa0; Click OK in the additional pop-up box From here you can Add to Storage Group, but if you add this to the ~filestorage Storage Group, Unisphere will automaticallyassign a Host LUN ID.&#xa0; IF THIS ID US UNDER 16, THE DATA MOVER WILL NOT SEE THE LUN AND YOU CANNOT ADD IT. Hover over the Hosts icon at the top and choose Storage Groups.&#xa0; Right Click on ~filestorage and choose Select LUNs.&#xa0; Browse to the LUN on the appropriate Storage Processor (SPA or SPB) and double click the name to select it. This adds the LUN to the Selected LUNs window at the bottom.&#xa0; Although it's not displayed, there is a selection box under Host LUN ID.&#xa0; Click the Host LUN ID field relating to the target LUN and choose an unused ID number (HLU).&#xa0; If this HLU is less than 16, it won't be recognize on a rescan. Click OK, OK and Yes to complete. Now go back to Storage tab and hover - choose Storage Pools for File. On the right side, locate File Storage window and choose Rescan Storage Systems. Acknowledge the dialogs. Wait for the dialog box to give you an OK and wait a few minutes for the rescan to complete.&#xa0; You can check if you like by hovering over the System tab and choosing Background Tasks for File.&#xa0; When the rescan is complete, right click on the desired Storage Pool for Block pool, and choose Extend.&#xa0; You will note a new Volume is available . Choose OK. You'll see the Storage Pool for File is now larger by the capacity of the LUN you created.allocating storage pool LUNs to file storage

Rainer_EMC · Answer

Hi,

yes you are right – how to set the HLU isn’t obvious.

However in all newer OS revs for quite some time now even if you create a LUN with a lower HLU and put them into the filestorage group the data mover will silently renumber that LUN to the next free value above 15 when it initially detects it via server_devconfig / rescan storage.

Rainer

dsain1 · Answer

Thanks for the reply, Rainer. I did rescan at some point. Even let things sit for a weekend and still no luck. I did have one (d17) that I could not add, but could see and with the help of someone at support, removed it. Not sure where in the process I rescanned the storage.

Just to poke at it, I created another LUN and added it and rescanned again. I see that it changed from 6 to 20. I wonder what I did wrong, but I'm glad I worked past it and have a slightly deeper understanding of how things work.

David

VNX

VNX File concepts and reconfiguration

Was this post helpful?