How to get file list with Atmos?

Hi Suli,

You currently cannot call listDirectory on the root node ("/") of Atmos onLine.  Try updating your code to use a virtual root directory, e.g. "/root/'.

For your 2nd question, I realized that the upload and download helpers haven't been updated to accept paths.  I have attached updated versions that will work properly.  To upload your object to a specific path, use the createObjectOnPath method.  For example:

uploadHelper.createObjectOnPath( new ObjectPath( "/root/file.txt" ), new File( "file.txt" ), null, null );

There is no need to create the directory.  It will be automatically created when the first file is uploaded into the path.

Note that Atmos supports two methods for identifying objects: object identifiers and namespace paths.  These are encapsulated in the wrappers as ObjectId and ObjectPath.  If you want to store your objects in a filesystem-like way with directories and files, you can use the createObjectOnPath method with an ObjectPath.  However, if you want to take a more database-like approach you can call createObject() and it will return you an identifier.  You can store this identifier in your own database for future reference or assign "listable metadata" to the object and locate it later with the listObjects method.

--Jason

Jason,

This is similar to Nirvanix SDK.

Thanks,

Suli

Jason,

Again, thanks for you answer. Now I have a couple more questions.

For the listDirectory() methods, specify "/root/" as root. Is that a temporary or it will be the way?

I used your new uploadHelper() to specify a path. Just want to confirm with you that the path specify is a path to the file in EMC onLine, right? What's the relationship between the file name and ID. Can I identify a file using file name instead of the ID?

Also, you provided a new downloadHelper file as well, but I don't see any method to allow my to specify a file path for download in EMC onLine? Should it be one?

Last (for now), can information like creation date/time, last modify date/time, size be associated with the file in EMC onLine? I believe those are very important information.

Thanks,

Suli

Oops, never mind the downloadHelper() question. I overlooked the "Identifier" object.

I guess I answered the question of the difference between Id vs. file name.

I guess either one can be used. right? Thanks.

For the listDirectory() methods, specify "/root/" as root. Is that a temporary or it will be the way?

Yes, for now it has been decided that "/" is not listable.  This is to discurage pollution of the root node for customers with many sub-tennants, causing performance degredation.  You can use any path you want; "/root/" is just an example.

Yes, for most operations you can use either ObjectPath or ObjectID.  The only difference is during create.

As for your question about create/modified, if you call getSystemAttributes you will get mtime (modified time), ctime (create time), and size.  These are the values as far as Atmos is concerned (when you created or modified the object in atmos).  If you want to persist the original create/modify time of the file you uploaded, you can create your own user metadata objects and attach them to the object during create or by using setUserMetadata().

Jason,

You answered all my questions. Thanks.

Suli

(P.s. yes, getSystemMetadata() works fine for me)

If you create a new '/root/' to avoid the limitation of not being able to list objects on '/' can you use that new root as an effective flat namespace? What if your data does not actually have a hierarchy and you want a flat namespace that has large scale and is listable?

Thanks,

Rob

Theoretically, yes.  I'm sure you'll see some performance impact after you get 10s of 1000s of items in one folder (just like a regular filesystem).

So I can have a flat scalable namespace without queries (ListObjects) or I have to add a real hierarchy to my namespace so I can get query support?

Is Atmos really targeting file system based applications rather than generic object storage applications?

Thanks,

Rob

The intent is that the /namespace interface will be used for filesystem-like applications and the /object interface will be used for generic object storage.  Currently if you want to query the generic object storage you will have to use the listObjects method or implement your own SQL index for searching.  Eventually, an XPath search API will be added for more comprehensive searching of the object metadata.

We are using the /object interface. How do you use ListObjects with the /object interface without adding tags that appear to limit scalability at the tag level to a million objects (10's of 1000's)?