This tutorial introduces how Kurator-Akka workflows are specified in YAML files.
What is YAML?
YAML is plain text format for representing data organized as lists of values or sets of key-value pairs (mappings or dictionaries). The values in these lists and in the key-value pairs can themselves be lists or mappings. YAML is a superset of JSON (every JSON document is a valid YAML document) that uses white space (rather than braces and quotes) to organize data and is thus easy to read.
Kurator-Akka uses YAML to represent workflow definitions so that workflows can be specified and executed without using any software development tools (other than a text editor), and so that other programs can generate workflow specifications simply by producing a YAML file. Kurator-Akka uses yaml-spring-loader (written by Scott McPhillps for the RestFlow system) to parse YAML files and to create using Spring the workflow components the files describe.
Note that
See http://yaml.org/ for more information about YAML and a list of libraries in C/C++, Ruby, Python, Java, Perl, C#, and other languages for composing and parsing YAML.
Example workflow
We will use the hello.yaml workflow from the Kurator-Akka distribution to show how workflows are specified in YAML. You can extract this YAML file from the kurator-akka jar file using the unzip command (the -j option prevents any directories from being created during inflation):
$ unzip -j kurator-akka-0.2-executable.jar org/kurator/akka/samples/hello.yamlArchive: kurator-akka-0.2-executable.jar inflating: hello.yaml$
The contents of hello.yaml are as follows:
imports:
- classpath:/org/kurator/akka/actors.yaml
components:
- id: GreetingSource
type: ConstantSourceActor
properties:
parameters:
value: Hello World!
- id: GreetingPrinter
type: PrinterActor
properties:
listensTo:
- !ref GreetingSource
- id: HelloWorldWorkflow
type: Workflow
properties:
actors:
- !ref GreetingSource
- !ref GreetingPrinter
parameters:
greeting:
actor: !ref GreetingSource
parameter: value
Structure of a YAML workflow definition file
Inspect the contents of hello.yaml above. Kurator-Akka expects each YAML workflow definition file to have a mapping (set of key-value pairs) as the top-level data structure. A colon in YAML indicates that the preceding string is a key, and that the following value or block of text is the value assigned to that key. The valid keys of this top-level mapping are imports, types, and components, with the result that Kurator-Akka workflow definition files have up to three top-level sections: an imports section, a types section, and a components section. The hello.yaml file does not have a types section.
The imports section (the block of text following the imports line) must be a list of other YAML files to be included in the current workflow definition. List items are preceded by a dash. The value given to the components is the list of workflow components comprising the workflow. We will focus on the workflow components for the remainder of this tutorial page.
Workflow components
The components section of hello.yaml contains declarations for two actors and for the workflow as whole. Actors are the active, data processing components of workflow. The workflow as a whole is considered a component as well, with the workflow component declaration containing references to the actor components that comprises the workflow.
Each component in hello.yaml is described by a mapping with three keys: id, type, and properties. The id of a component is an arbitrary text string that is used to uniquely identify it, and usually indicates the function of the component in the workflow. The id also enables components to refer to each other.
The type of a component refers to a declaration in the type section of the current YAML file, or in a YAML file directly or indirectly included in the imports section. Type declarations ultimately all refer to Java classes, and are covered in later tutorial.
Finally, components of have properties that represent the configuration of the component in the current workflow.
Examing the components in hello.yaml
The first component in hello.yaml is the actor with id GreetingSource:
- id: GreetingSource
type: ConstantSourceActor
properties:
parameters:
value: Hello World!
This declaration states that the component named GreetingSource is an instance of the ConstantSourceActor. ConstantSourceActor emits a value (or sequence of values) at the beginning of a workflow run. The value emitted by a ConstantSourceActor is specified by the value parameter. Parameters represent a subset of component properties that are available to the component at run-time. Properties not specified as parameters are used by the KuratorAkka framework to compose the workflow as a whole and are not visible to running components. Here, GreetingSource is configured to emit the string, 'Hello World!' when the workflow starts running.
The second component in hello.yaml is the actor with id GreetingPrinter:
- id: GreetingPrinter
type: PrinterActor
properties:
listensTo:
- !ref GreetingSource
GreetingPrinter is an instance of PrinterActor. A PrinterActor writes any data it receives to an output stream (which defaults to stdout of the process running the workflow). The listensTo property of GreetingPrinter is assigned a list containing a single element with the value !ref GreetingSource. Actors in a workflow generally receive the data they process from other actors in the same workflow, and the listensTo property on an actor lists the actors that this actor receives data from. The !ref keyword indicates that the string following it is not a literal value but a reference to (identifier of) a component defined elsewhere in the workflow. Here, the GreetingActor is configured to receive any data emitted by the GreetingSource actor, and thus will receive the string 'Hello World!' emitted by GreetingSource when the workflow is run.
The final component of in hello.yaml is identifies as HelloWorldWorkflow:
- id: HelloWorldWorkflow type: Workflow properties: actors: - !ref GreetingSource - !ref GreetingPrinter parameters: greeting: actor: !ref GreetingSource parameter: value
HelloWorldWorkflow is an instance of Workflow, and is used to specify which actors defined in the yaml file (or in imported yaml files) comprise a single workflow. A Workflow also gives values for properties that affect the workflow as a whole. The actors property is assigned a list of references to the actors in the workflow, while the parameters property is used to expose parameters on individual actors in the workflow as configurable parameters of the workflow.
Here, HelloWorldWorkflow is configured to comprise two actors, GreetingSource and GreetingPrinter. The value parameter of the GreetingSource actor is exposed as the greeting parameter of the workflow.