Page History

...

• FILE - Originally extractors primarily operated on one file. Trigger when file is added to Clowder.
  
  • *.file.#

  • *.file.image.#

    field	description
    id	file UUID
    intermediateId	file UUID (deprecated)
    datasetId	id of dataset file was added to
    filename	file name
    secretKey	Clowder API key
    host	Clowder host URL

• DATASET - Later, support for dataset extractors was added. Trigger when file is added to a dataset.
  • *.dataset.file.added

  • *.dataset.file.removed

    field	description
    id	file UUID
    intermediateId	file UUID (deprecated)
    datasetId	id of dataset file was added to
    secretKey	Clowder API key
    host	Clowder host URL

  These trigger when a file is added to a dataset. 
  • Because the message contents are otherwise identical, PyClowder currently uses the presence of 'filename' field in message to determine whether to handle as a file or dataset extraction.
  • Max just updated PyClowder2 pull request to include routing_key in the parameters for extractors, so we can check that instead of checking 'filename' field.
    
• METADATA - Later, support for metadata-triggered extractors was added.
  • *.metadata.added

  • *.metadata.removed

    field	description
    id	file or dataset UUID
    metadata	md that was added/removed

    This sends correct messages to RabbitMQ from Clowder but we need to evaluate this one as well because PyClowder has some rough edges in determining how to handle these messages, as they may not require a file OR a dataset to be downloaded or considered if the extractor can work with the metadata alone.

• COLLECTION -  Eventually, we may want to trigger extractors that process arbitrary collections of datasets.
  • Not quite sure how we're gonna do this yet.
    
    

...