Obtain Source Code

The source code is available from the NCSA opensource Git repository:

  git clone https://opensource.ncsa.illinois.edu/fisheye/git/dfdl.git

we are currently working (collectively) on the master branch.

Requirements & Building from Source

Daffodil requires Java 1.6 or higher. All scripts provided for running and testing are written in bash and tested in GNU Linux. Those scripts are not required though, so Daffodil should be able to run in other platforms as long as Java is installed

SBT is required for building, and depedency management.  See the README and BUILD files in the root of your local clone of the git repository.  In short, you need to download and put the correct SBT Launcher in your path.  Daffodil is currently using version 0.11.3 of SBT.  The correct launcher can be downloaded from http://xsbt.org/.

Daffodil is written in Scala 2.9, which also must be installed in order to run Daffodil.

Note: You will likely need to set available JVM memory for ant larger. For example, on a 64-bit Linux system, I have this in my .bashrc:

export JAVA_OPTS="-XX:MaxPermSize=512m -Xmx6000m -Xms40m -Xss2m"

This might seem generous (6Gig!), and the more critical thing is the MaxPermSize, which defaults to much too small.

Compiling in eclipse

There are IDE's and eclipse plug-ins that can be used to build daffodil. See:

  http://www.scala-lang.org/node/91

Some time-dependent information: The As of November 2011, the Eclipse Indigo IDE (also known as version 3.7) and the current Scala plug-in for it (release 29, 2.0.0-beta, with this Eclipse update site: http://download.scala-ide.org/releases-29/2.0.0-beta), work for daffodil. They are robust enough to be useful, though there are some bugs still, and one must occasionally exit and re-enter Eclipse.

Tricks/Tips: You must increase the memory available, and specifically, the stack-size that Eclipse's JVM will use. This is done by editing eclipse.ini in the eclipse install directory. Below is an example of an eclipse.ini that works.

The first line "-clean" slows down starting Eclipse, but seems to be helpful given the current state of the Scala plug-in. Your mileage  may vary. The key changes are the last 3 lines. These set the minimum, maximum, and stack size to more reasonable values than the defaults. You need quite a bit of memory to use this IDE with Scala for Daffodil.

It is also very helpful when dealing with XML files to load the XML and XML-schema tools that are available in Eclipse.

-clean
-startup
plugins/org.eclipse.equinox.launcher_1.2.0.v20110502.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.1.100.v20110505
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile
-vmargs
-Xms1000m
-Xmx1250m
-Xss2m

Eclipse XML Setup

Getting eclipse to validate DFDL Schemas using its XML validation capabilities requires some configuration. Without this you will get hundreds (at least) of XML and XML Schema validation errors from the daffodil code base.

There are two files of interest to this

The eclipse-xml-catalog is imported into Eclipse via File->Import->XML->XML Catalog.

The eclipse-settings zip file contains a .settings directory to be restored in your Eclipse workspace's ".metadata/.plugins/org.eclipse.core.runtime/.settings" directory.

These settings and catalog will change the configuration of eclipse substantially, and may not be suitable for projects other than Daffodil, so save your existing eclipse install directory and workspace's .metadata/.plugins/org.eclipse.core.runtime/.settings directory, if you are trying to share it across more than just Daffodil. You'll likely need to split out into two distinct workspaces and two distinct copies of eclipse.

With the settings and XML Catalog above, you can edit DFDL Schemas, and get support from Eclipse.

For example,

Contrarywise....

Eclipse Unicode Setup

On Linux, UTF-8 and good unicode fonts are the default.

On Windows, a US install normally doesn't default to a Unicode friendly environment. One of the features of the Scala programming language is that it is completely unicode aware.

But, to make a USA windows default install of Windows itself, and Eclipse unicode friendly, you have to take a few steps.

These are described in the solution to this issue: https://opensource.ncsa.illinois.edu/jira/browse/DFDL-81. See the comments at the bottom for the settings. Basically, you have to install the Microsoft Japanese Language Pack.

Intellij Setup

Intellij comes ready-equipped to support Daffodil with only a few changed required to add the plugins for Scala and SBT support.

In order to make sure all the required libraries, including Scala, are downloaded you should first run sbt from the command line.  This will call Ivy to retrieve any missing libraries so that when you run Intellij you can point to the right libraries for the project build.

Next, in the Intellij UI, open File->Settings and in the left pane scroll down to the section labeled Plugins (this is below the middle divider).  In plugins select "Browse repositories,,," and from this list select "Scala" and "SBT" (use the search feature to help find them).  Then click the Download and install icon (the disk with the arrow toolbar button or via context menu).  When the two plugins are loaded, close Intellij and restart.

Once restarted, reopen settings and open the SBT option under the project settings (you can ignore the Scala settings as these defaults are sufficient).  Under "SBT launcher JAR file" enter the path to where you downloaded the sbt-launch,jar above and under the VM parameters enter the value specified by the JAVA_OPTS export.

Next you need to configure daffodil to use the version of the Scala compiler downloaded by Ivy.  This will be a local setting and not the global FCS setting so each module has to be updated individually.  Right-click on daffocil-core and select "Open Module Settings".  Then for each module, open up it's children, select "Scala" from the list of children and click the "Use ordinary compiler" for the Compiler Installation.  This should default to scala-2.9.1 which is the version downloaded by Ivy and configured in the Intellij project.

Finally, under the Edit Configurations menu (drop-down) make sure you have a target.  You can create a new target to build all tests by clicking the plus sign and then selecting "JUnit" as the test type.  Then in the dialog make sure you select the correct class path (i.e. daffodil-core) and path to the core and then check "Run SBT Action" and in the "..." dialog select "test:products".