The Brown Dog DTS is a highly extensible/distributed service providing a uniform means of managing and accessing transformation capabilities within the web. Utilized tools can come in the form of command line applications, GUI driven applications, libraries, and/or other services. Here we go over the process of preparing a new transformation tool, either an extractor or a converter, for usage with the DTS.
Using the BD Development Base
BD-base runs the necessary dockerized Brown Dog Data Transformation Services (Clowder, Polyglot, Fence, RabbitMQ, MongoDB, Redis, an example extractor, an example converter, and the BD CLI) and combines them into one integrated program.
- After downloading BD-tmux, users can simply run bash script under command-line to start BD-tmux.
| Code Block |
|---|
bd.sh |
The BD-tmux script will split your terminal into panes and start each of the services needed for the Brown Dog Data Transformation Services. It provides a useful and convenient way to view the logs of running services in panes.
Users can switch between panes using Tmux commands. In bottom pane, users can run BDCLI commands to interact with Brown Dog Data Transformation Services (Username: fence Password: testing).
There is an example to perform a conversion from jpg to bmp.
- To do extraction, users need to login local Clowder Web Service (http://user_machine_ipaddress:9000 with username: admin@test.com and password testing0909) to accept ''Terms of Service'', then run bd commands as below example:
Extractors
Here we described the process for taking a working piece of code and deploying it as a Brown Dog extractor. It is assumed that the method can be invoked from a single call. In this example, we are using the python extractor wrapper and will invoke a python function. In a very similar fashion, a method developed in a language other than python can be invoked using subprocess.
...
| Code Block | ||
|---|---|---|
| ||
docker-compose up -d
docker build -t <your_extractor_name> .
docker run --rm -i -t --link <your_extractor_name_with_only_alphabets>_rabbitmq_1:rabbitmq <your_extractor_name> |
...
| Code Block | ||
|---|---|---|
| ||
RUN chown polyglot /home/polyglot/polyglot/scripts/sh/MyTool_convert.sh && \
chmod +x /home/polyglot/polyglot/scripts/sh/MyTool_convert.sh |
3. Test the Converter
Build the Dockerfile and start the converter:
| Code Block | ||
|---|---|---|
| ||
docker-compose stop
docker build –t mytool .
docker-compose up |
Usage of BD-tmux
...
. |
...
- After downloading BD-tmux, users can simply run bash script under command-line to start BD-tmux.
| Code Block |
|---|
bd.sh |
The BD-tmux script will split your terminal into panes and start each of the services needed for the Brown Dog Data Transformation Services. It provides a useful and convenient way to view the logs of running services in panes.
...
There is an example to perform a conversion from jpg to bmp.
...