Table of Contents
About this Page
This page contains some initial ideas about how to organize data within Clowder for this project.
General Clowder Information
There are three basic sections within Clowder:
- Spaces
- Collections
- Datasets
Access Control Details:
- Being a Clowder User does not automatically provide Access to all Spaces
- Access is established at the Space level
- Access must be set for each Space individually
- All Datasets and Collections inherit permissions from the Space to which they belong
Organization Ideas
Naming Conventions
Some ideas for how to name items that are placed in Clowder:
- When able, include the name of the Sensor in the title of Spaces, Collections, and Datasets
- Including the name of the Sensor may make it is easier to search for items
Using the Sections Within Clowder
It would be good to have a general organization methodology moving forward.
This section will contain various ideas for organization.
One idea for utilizing and organizing these sections:
- Spaces
- Named after each Sensor or a general category (e.g.: "How To" Documentation, Images, etc.)
- Contains both Collections of Datasets and individual Datasets
- Can have an image appear in the listing on Clowder
- Collections of Datasets
- Naming
- Include the Sensor name
- Sub-collections can be used to logically organize datasets, for example data ranges
- Contains Datasets
- Naming
- Datasets
- Naming
- When possible, the name would include the name of the Sensor
- Some Datasets can contain raw data and other derived data
- Can contain Folders that contain files
- Should be placed in Collections
- Metadata
- Datasets can have Metadata
- Files in Datasets can have Metadata
- Can have Tags
- Tags are searchable
- Can view a list of all Tags in the application
- Naming
This is an example illustrating this idea:
- This is based upon the Flux Tower data (information about this Dataset is available in this Wiki space at Flux Tower Data)
- Space
- There is only one of these for this Space
- Space Name: Flux Tower Site
- Space Description: Flux Tower located at
location-description
- Space Image: Suggestion - provide an image of the site or the instrument or a logo (if available)
- Collection
- There can be several of these for this Space
- Data Measurements Example
- Collection Name: Flux Tower Data Files
date-range
- The date-range could be years or months depending upon the size of the dataset
- Collection Description: Collection of Data Files for
date-range
- Collection Content: This will contain all relevant Datasets for this collection
- Example A:
date-range
is a year- If
date-range
is 2010 - Then Datasets would be all the Datasets that include 2010 data
- If
- Example B:
date-range
is January 2010 to March 2010- If
date-range
is January 2010 to March 2010 - Then Datasets would be all the Datasets that include 2010 data from January to March
- If
- Example C:
date-range
is the Year 2010- If
date-range
is 2010 - Then there would be one Dataset for all 2010 data
- If
- Example A:
- Collection Name: Flux Tower Data Files
- Documentation Example
- Collection Name: Flux Tower Documentation
- Collection Description: Documentation for the Flux Tower Sensor
- Collection Content: This will contain any relevant documentation that describes the Sensor (types of measurements gathered, etc.)
- Datasets would be present in this collection for enhanced organization
- Files can be of any preferred format
- Datasets
- There can be several of these for each Collection in the Space
- To simplify managing data, either of these can be implemented:
- Individual Datasets can be grouped by month
- Use Folders in yearly Datasets to group files by month
- Without Folders Example
- Dataset Name: Flux Tower Raw Data Sept 2016, Flux Tower Data 5TE Soil Probe 30 cm Sept 2016, etc.
- Flux Tower
information
Sept 2016 - The
information
would just describe the general content of the Dataset (type of measurement, type of data, etc.)
- Flux Tower
- Dataset Description: Flux Tower Raw Data collected during the month of February 2016
- A Dataset with Flux Tower
information
for February 2016
- A Dataset with Flux Tower
- Dataset Metadata: This will contain any relevant Metadata
- Dataset Tags: This will contain any relevant Tags
- Dataset Content: This will contain any relevant files
- Dataset Name: Flux Tower Raw Data Sept 2016, Flux Tower Data 5TE Soil Probe 30 cm Sept 2016, etc.
- With Folders Example
- Dataset Name: Flux Tower Raw Data 2016
- Flux Tower
information
for 2016 - The
information
would just describe the general content of the Dataset (type of measurement, type of data, etc.)
- Flux Tower
- Dataset Description: Flux Tower Raw Data collected during the year of 2016
- A Dataset with Flux Tower
information
for 2016
- A Dataset with Flux Tower
- Dataset Metadata: This will contain any relevant Metadata
- Dataset Tags: This will contain any relevant Tags
- Dataset Content: This will contain Folders for each month of data
- Folder 1: January 2016
- This will contain all January files
- Folder 2: February 2016
- This will contain all February files
- Folder 3: March 2016
- This will contain all March files
- Keep creating Folders for the year as needed
- Folder 1: January 2016
- Dataset Name: Flux Tower Raw Data 2016
Organization Tasks
General Tasks to be completed:
- Decide upon the organization methodology
- Removing any extraneous or duplicated data
- Remove "imlczo" Space and move Datasets contained within to appropriate Spaces
- Where possible, moving and renaming items to fit the decided upon organization methodology
- Provide a "How To" wiki page about the decided upon organization methodology
- Make some example changes in Clowder that reflect the ideas presented herein