...
Within the Polyglot framework, each tool is known as a converter to emphasize its data format conversion capabilities. A converter is a wrapper script, written in any scripting language, such as bash, R, Python. A converter resides within a software server, which also hosts third-party software or libraries which provide the actual format conversion. The software server orchestrates the operation of the converter tool which runs the third-party software, handling input and output files.
Similarly, a tool within Clowder framework is known as an extractor within Clowder framework to emphasize on the extraction of metadata from data contentfiles. An extractor is a program that extracts metadata from within the file content, analyzes a file. It analyses the file’s contents and tags it according to some specific classification or criteria, etc. It resides in a distributed environment, such as a cloud server, as an extraction service and . It listens to a message queue for extraction requests from DTS API. The extraction process is triggered based on specific file types (MIME Types) and the metadata extracted are then made available through the DTS API.
In the BD framework, extractor and converter are the extensible units, and can be contributed by researchers across different research domains. To enable contributions from the research community, a Tools Catalogue was designed and implemented where a user can register and share their BD tool extractor or converter with other users.
How can I contribute to BD data transformation services?
...
Why should I contribute?
Researchers often build/ develop new tools for their study/ research, in order to extract useful information from unstructured /or semi-structured data. A lot of efforts effort goes into that developing new tools and such efforts are often unacknowledged. The Instead, the value of their the research is usually measured in terms of publications and the analysis done on the data obtained from those unstructured data. Also, some of these extraction/conversion tasks . In addition, similar tool development efforts are repeated by multiple researchers within the same domain of science. Towards acknowledging such effortstool development effort, we we built the BD Tools Catalogue, where members of different research community could communities can contribute and share their tools within BD framework. So, In the BD Tools Catalogue you get proper credit for your effort in creating the new tool.
...
Can I contribute if my tool uses a third-party proprietary software?
Yes, you can submit your tool which , even if it uses third-party proprietary software. However, before it is made available as Data Transformation Services, we will require need to review the license agreement. Our current focus has been is on using opensource software for building bd open source software to build Brown Dog tools.