You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

 

If you are having a problem with Daffodil, and think that perhaps you have found a bug, then we suggest you:

  • check JIRA (to see if the bug is already reported)
  • create a TDML file for reproducing the bug/issue
  • create a JIRA ticket and attach your TDML
    OR
  • email the issue to daffodil-users@oss.tresys.com (you have to sign up for that list to be able to send to it.)

Details are below.

A TDML file is often useful just to ask a question about how something in DFDL works, for example, to get a clarification. It allows for a level of precision that is often lacking, but also often required when discussing complex data format issues.

Check JIRA to See if your Issue is Already There

First you should give a search of our JIRA tickets to see if the problem is already recorded.

Here's the front page: https://opensource.ncsa.illinois.edu/jira/browse/DFDL

Here's a list of all tickets about bugs, new features, and improvements. In reverse chronological order (most recent first). You may want to change the issue type, or status specifications to narrow down the list, but most commonly you would just put some search keywords into the search box.

Don't go crazy with this searching however, because if you can't find it with reasonable effort (a few good guesses at search terms), then whatever is there isn't tagged sufficiently anyway.

If you do find a bug or a closely related issue that is open status, then you can add your information to it as a comment if you prefer, rather than creating a new issue. Just knowing that another person has run into the issue is helpful at assigning fix priorites.

Create a TDML File that Illustrates the Issue

The absolutely best way to report a bug is by creating a TDML test file that demonstrates the problem.

TDML stands for "Test Data Markup Language". It is a way of specifying a DFDL schema, the test data, the expected result or expected error/diagnostic messages, and it is all in a single self-contained XML file.

By convention, a TDML file uses file extension ".tdml".

Below is an annotated TDML file for a very simple example:

<?xml version="1.0" encoding="ASCII"?>
<!-- 
Example of a self-contained test described in a TDML file
 -->

 <tdml:testSuite 
  suiteName="Suspected Bugs" 
  description="Illustrates issues found 2013-04-01"
  xmlns:tdml="http://www.ibm.com/xmlns/dfdl/testData" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" 
  xmlns:xs="http://www.w3.org/2001/XMLSchema"
  xmlns:ex="http://example.com">

  <!--
    Use defineSchema to include a DFDL schema directly inside the TDML file.
    You can alternatively put the DFDL schema in a separate file if you prefer.

    Each defineSchema has a name, so that one TDML file can contain tests which reference
    different DFDL schemas. 

    To embed a schema inside the TDML you don't include the <xs:schema...> element from
    the schema file, nor do you need to wrap the top-level DFDL annotation objects with
    xs:annotation and xs:appinfo.

    In other words, inside a defineSchema you can directly put: 
    dfdl:defineFormat, dfdl:defineEscapeSchema,
    dfdl:format (for the default format), xs:element, xs:simpleType, xs:complexType, xs:group.
   -->

  <tdml:defineSchema name="s1">

    <dfdl:defineFormat name="myDefaults">
      <dfdl:format lengthKind="implicit" representation="text"
        lengthUnits="bytes" encoding="US-ASCII" initiator="" terminator=""
        separator="" ignoreCase="no" textNumberRep="standard" />
    </dfdl:defineFormat>

    <dfdl:format ref="myDefaults" />

    <!-- 
      imagine we are reporting a bug with date/time functionality, and
      this element exercises the feature of concern.
     -->
     
 <xs:element name="dateTimeText" type="xs:dateTime" 
      dfdl:calendarPattern="MM.dd.yyyy 'at' HH:mm:ss ZZZZ" dfdl:calendarPatternKind="explicit"
	  dfdl:lengthKind="explicit" dfdl:length="{ 35 }" />

    <!-- That's it for the schema for this small example -->
  </tdml:defineSchema>

<!-- 
   Here is a test case that exercises the above schema.

   A single TDML file can contain many test cases like the one below. This
   example has only one.

   You must give the name of the model (aka the schema), that can be the name of a
   schema defined immediately in this file like above, or a file name.

   You must also give the name of the root element that the test will use.
 -->

<tdml:parserTestCase name="dateTimeText" root="dateTimeText"
    model="s1" description="date time issue"> <!-- description is optional --> 

<!--
   The data for your test is given by the tdml:document element.

   Notice specifically the use of the CDATA bracketing of the data. This
   insures that no unintended whitespace gets inserted around your data.
  -->
    <tdml:document><![CDATA[04.02.2013 at 14:00:56 GMT-05:00]]></tdml:document>

  <!--
   The infoset element gives the expected infoset, expressed as an XML fragment.
  -->

  <tdml:infoset>
  <!--
     Always need this extra tdml:dfdlInfoset element as well
    -->
       <tdml:dfdlInfoset>

    <!--
      Here is our actual expected result, where the date and time
      is now in XML's cannonical representation for these.
      -->
       <dateTimeText>2013-04-02T14:00:56-05:00</dateTimeText>
      </tdml:dfdlInfoset>
    </tdml:infoset>

<!-- end of the test case -->
 </tdml:parserTestCase>
<!-- end of the whole TDML file --> 
</tdml:testSuite>

Suppose you save the above out as a file "myDateTimeBug.tdml". You can then run it using the Daffodil command line tool:

daffodil test myDateTimeBug.tdml

When specifying the test data, there are other ways to do this than using just text.

You can specify the test data in hexadecimal, in individual bits, or you can direct Daffodil to find the data in an external file.

These are illustrated here. You just change the way the tdml:document element is specified to include tdml:documentPart children elements:

    <tdml:document>

      <!--
          A document part with type="text" is text. Use CDATA to avoid whitespace changes.

          So in the example below, the line ending after '250;' and after '967;' are intentional
          parts of the data.
        -->

      <tdml:documentPart type="text"><![CDATA[quantity:250;
hardnessRating:967;
]]></tdml:documentPart>

      <!-- 
          In 'text' both XML character entities, and DFDL's own character entities are interpreted.

          So here is a NUL terminated string that contains a date with some Japanese Kanji characters.
          The Japanese characters are expressed using XML numeric character entities. The NUL termination
          is expressed using a DFDL character entity.

          In this example one has no choice but to use a DFDL character entity. The NUL character (which has character
          code zero), is not allowed in XML documents, not even using an XML character entity. So you 
          have to write '%NUL;' or '%#x00;' to express it using DFDL character entities.
        -->

      <tdml:documentPart type="text"><![CDATA[1987&#x5E74;10&#x6708;&#x65e5; BCE%NUL;]]></tdml:documentPart>

      <!--
          Type 'byte' means use hexadecimal to specify the data. Freeform whitespace is allowed. 
          Actually, any character that is not a-zA-Z0-9 is ignored. So you can use "." or "-" to separate
          groups of hex digits if you like.
       -->
 
      <tdml:documentPart type="byte">
            9Abf e4c3
            A5-E9-FF-00
      </tdml:documentPart>
      
       <!--
          Type 'bits' allows you to specify individual 0 and 1. Any character other than 0 or 1 is ignored.
           
          The number of bits does not have to be a multiple of 8. That is, whole bytes are not required.
         -->

       <tdml:documentPart type="bits">
            1.110 0.011 1 First 5 bit fields.
       </tdml:documentPart>

       <!--
          Type 'file' means the content is a file name where to get the data
         -->
  
       <tdml:documentPart type="file">/some/directory/testData.in.dat</tdml:documentPart>

    </tdml:document>

 

Further details on TDML will go in a more detailed guide/page about writing TDML.

If you use the external schema file or external data file capabilities, then of course you need to send those files along with your TDML.

 

 

  • No labels