SEAD's Clowder component has a service api that allows you to read/write/annotate/tag/delete datasets and collections (among other things). These services are used within SEAD's bulk upload/sync command line tool and can be called from Curl or your own code to allow your application to read/write directly to/from Clowder spaces.
...
- Please make sure you have JAVA installed on your computer. If you don't, you can download it here: https://java.com/en/download/
- Download the SEAD Uploader jar file : sead2.13.jar.
- On your computer, open the root directory that contains files and folders you would like to upload to your Project Space in SEAD.
- Put the sead2.13.jar file in this directory.
- Holding the SHIFT button, right click on your mouse to open a menu. Select the "Open Command Window Here" option.
- Once you have the Command Window open, invoke the SEADUploader by typing the following in the Command Window:
java -cp sead2.13.jar org.sead.uploader.clowder.SEADUploader <-listonly> <-limit=<X>> <-skip=<n>> <-verify> <-ex<Y>> ex=<Y>> <-key=<apiKey>> <-forcenew> -server=<serverUrl> -id=<id> <directories /files list...>
where:
-listonly: write information about what would/would not be transferred without doing any upload
...
-skip=<n>: skip the first <n> files found on disk before starting to check whether files exist on the server and uploading those that are not yet in SEAD (any required collections will be automatically created)
-ex<Y>ex=<Y>: exclude any file that matches the provided regular expression pattern, e.g. -ex^ex=^\..* (exlude files that start with a period) -ex*.txt (exclude all files ending in .txt). Multiple repeats of this flag can be used to exclude based on multiple patterns.
...
-key=<apiKey>: use an API Key created you create (on your Profile page) to avoid having to enter a username/password.
-id=<id>: if you know a dataset exists, specifying it's id here will improve performance as the uploader won't have to scan through all datasets to find it.
-server=<serverUrl>serverUrl: the base URL of the SEAD/Clowder server you're interacting with, e.g. -server=https://sead2.ncsa.illinois.edu
directories /files lists list - a list of one or more directory or file names the Uploader should work on. The Uploader will recurse (depth first) through the files and subdirectories contained within any listed directory.
...
2.0 examples: use of the Uploader creates one Dataset with Folders and Files inside
java -cp sead2.13.jar org.sead.uploader.clowder.SEADUploader -listonly listonly -server=https://sead2.ncsa.illinois.edu mydir
Using SEAD's 2.0 instance, check ./mydir and list the Dataset and all Folders and Files that would be created without the -listonly flag
java -cp sead2.13.jar org.sead.uploader.clowder.SEADUploader SEADUploader -server=https://sead2.ncsa.illinois.edu mydir
Using SEAD's 2.0 instance, create a 'mydir' Dataset in your account on sead2.ncsa.illinois.edu and create Folders and Files for all contained items. Each File will be annotated with metadata indicating the original path (user metadata: "instanceOf (http://purl.org/vocab/frbr/core#embodimentOf) with the value /mydir for the mydir directory in this example and /mydir/<relative path> for all Files). Note that if this command is run without the -merge flag, it will create a new Dataset and upload all folders and files again, even if a Dataset already exists from a prior run (which is not usually what you'd want). -merge makes the Uploader check for an existing Dataset and only uploads missing content, or folders/files added to your local directory since the last upload.
java -cp sead2.13.jar org.sead.uploader.clowder.SEADUploader -limit100 limit100 -server=https://sead2.ncsa.illinois.edu mydir
Using SEAD's 2.0 instance, locate/create a 'mydir' Dataset in your account on sead2.ncsa.illinois.edu and create Folders and Files for the first 100 contained items. Running the command again will add the next 100 items (becasue of the -merge flag it will find/skip the ones already uploaded).
java -cp sead2.13.jar org.sead.uploader.clowder.SEADUploader -verify verify -server=https://sead2.ncsa.illinois.edu mydir
Using SEAD's 2.0 instance, verify that all files previously uploaded to SEAD 2 are exactly the same as those on your disk. Since SEAD 2 currently calculates hash values with a delay, the -verify flag should be added on a second run of the Uploader.
...
If you experience problems with using the SEADUploader, please contact SEAD at SEADdatanet@umich.edu and we will be happy to walk you through the steps.