Ergo does not currently provide direct support for executing arbitrary code from the Analysis Framework; however, it is possible to execute arbitrary code (e.g. executables/scripts written in other languages) with the current version. This page will layout some thoughts we've had for handling executing arbitrary code. There are plans for the next version of Ergo to refactor the analysis framework to bring in a scientific workflow engine that can provide better support for executing command line tools written in languages other than Java.

 

Requirements

The machine that will execute the arbitrary code must either have the software and supporting libraries already installed on the machine or they must be bundled with the plug-in.

Custom Iterator

To handle executing arbitrary code, the built in simple iterator will likely be insufficient since the analysis task will be passed a Java data access object (e.g. a FeatureDataset containing access to shapefile data) that will need to be converted into something the executable/script expects (e.g. write the shapefile data to disk). A custom iterator would allow you to setup an analysis task with all required inputs for the analysis and then let the analysis task setup/run the executable/script. You can learn more about creating a custom iterator in the Ergo Developer Tutorial.

Analysis Task

Java provides a ProcessBuilder class that can be used to create operating system processes. The process builder provides methods to specify the command to execute, program arguments, environment variables, and redirect standard error and standard output so it can be written to a log. If you need to execute arbitrary code and aren't familiar with ProcessBuilder, follow the Javadoc link to learn more about the API. In this scenario, a custom iterator would call an analysis task with all inputs specified by the user and the task would be responsible for:

  1. Convert the input data to a format expected by the executable (if required)
  2. Creating a ProcessBuilder with the parameters required for execution (command, arguments, environment variables, etc)
  3. Call ProcessBuilder.start() to start the job. The task will be required to monitor this process until it's complete.
  4. Store results from the process through Ergo's data mechanism for storing job results. This may require converting the results from the format generated by the executable/script to what Ergo expects.

The developer tutorial provides a step by step example to illustrate creating a custom iterator and analysis task. This should be enough to provide the boilerplate of what would be required for executing arbitrary code. For an example analysis that uses a custom iterator/task, see the edu.illinois.ncsa.ergo.eq.nbsr plug-in.


 

  • No labels