...
Within the Polyglot framework, each tool is known as a converter to emphasize its data format conversion capabilities. A converter is a wrapper script, written in any scripting language, such as bash, R, Python. A converter resides within a software server, which also hosts third-party software or libraries which provide the actual format conversion. The software server orchestrates the operation of the converter tool which runs the third-party software, handling input and output files.
Similarly, a tool within Clowder framework is known as an extractor within Clowder framework to emphasize on the extraction of metadata from data contentfiles. An extractor is a program that extracts metadata from within the file content, analyzes a file. It analyses the file’s contents and tags it according to some specific classification or criteria, etc. It resides in a distributed environment, such as a cloud server, as an extraction service and . It listens to a message queue for extraction requests from DTS API. The extraction process is triggered based on specific file types (MIME Types) and the metadata extracted are then made available through the DTS API.
In the BD framework, extractor and converter are the extensible units, and can be contributed by researchers across different research domains. To enable contributions from the research community, a Tools Catalogue was designed and implemented where a user can register and share their BD tool extractor or converter with other users.
How can I contribute to BD data transformation services?
...