This documentation may be out of date. For the most recent documentation for the archival extractors, see the READMEs in GitHub: https://github.com/clowder-framework/extractors-archival
The archival process in Clowder is an optional addon to the already optional RabbitMQ extractor framework.
...
Two archival extractor implementations exist that currently depend on which ByteStorageDriver you are using:
- ncsa.archival.disk: Moves the file from one specially-designated folder on disk to another (requires write access to Clowder's data directory)
- ncsa.archival.s3: Changes the Storage Class of an object stored in S3 (requires write access to Clowder's bucket in S3)
...
Both the file size and the inactivity period can be configured according to your preferences.
Configuration Path | Default | Description |
---|---|---|
archiveAutoInterval | 0 | If == 0, disable automatic archiving. If > 0, check every |
archiveAutoDelay | 120 | Number of seconds to wait before starting the first iteration of the automatic archival loop. |
archiveAutoAfterInactiveCount | 90 | Number of units a file can go un-downloaded before it is considered "inactive". |
archiveAutoAfterInactiveUnits | days | The units for the inactivity timeout above (e.g. "90 days" old) |
archiveAutoAboveMinimumStorageSize | 1000000 | The minimum number of bytes for a file to be considered as a candidate for automatic archival. |
ncsa.archival.disk
This image has been pre-built as clowder/extractors-archival-disk
.
...