dfdl

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Checkpoint

Adding tests for tunables and fixing typo in setTunable

DFDL-1143

DaffodilTunables is no longer a globally accessed class.

SSRD/ERD, PState/UState, Compiler, DataProcessor receive

an immutable instance of DaffodilTunables.

DFDL-1143

  1. … 49 more files in changeset.
Fix xs:hexBinary equality check

Before any sort of equality check on a xs:hexBinary element would need

to be converted to a string. I have now added an equality checker for

byte arrays that allows direct comparison of xs:hexBinary elements

without the need for conversion.

DFDL-1618

Fix eclipse classpath errors.

    • -25
    • +0
    /eclipse-projects/test-ibm1/.classpath
Ensure Bit/ByteOrderChange parsers always exist at root

If the dfdl:byteOrder or dfdl:bitOrder properties were not provided,

Daffodil did not complain, and it would not insert a Bit/ByteOrderChange

parser at root. This meant that Daffodil would use a default value,

which in some cases caused incorrect behavior. It also meant that the

check for valid bit and byte orders (e.g. not bigEndian &

leastSignificantBitFirst) never occurred since that only happend in

ByteOrderChange parsers, meaning it could allow invalid bit/byte order

combinations.

This changes Daffodil so the root element will always insert Bit and

ByteOrderChange parsers, ensuring that the validity of the properties

are correct and ensuring correct parsing/unparsing. This means that

byteOrder is now always required, even in some cases where it might not

technically be needed.

This also modifies the initial UState/PState creation to not set

bit/byte order, but to instead rely on the Bit/ByteOrderChange parsers

to always be inserted first thing.

DFDL-1835

Calculate the approximate length of delimited and pattern lengthKinds for alignment

By assuming the length of delimited and pattern lengthKinds was 1,

Daffodil was unable to optimize out some alignment unparsers, which

could lead to deadlocks.

This changes that so the length of delimited or pattern lengthKinds is a

multiple of the length of the encoding. This allows Daffodil to optimize

out some alignment unparsers and prevent deadlocks.

DFDL-1836

Adding test for bit order without byte order specified

DFDL-1468

Fix Xerces resource resolution for includes/imports

After we found a resource (like from an include/import), we reported the

location to Xerces incorrectly. For example, say we resolved the

systemID at

org1/xsd/payload.xsd

to the absolute URI at

file:/foo/bar/baz/org1/xsd/payload.xsd

When we created the new Input to return where we found payload.xsd to

Xerces, we set the systemID to the original relative systemID

(org/xsd/payload.xsd), and set the baseURI to the absolute URI

(file:/foo/bar/baz/org1/xsd/payload.xsd). To Xerces, this meant that the

resource was found at the first path relative to the second path. But

the first path relative to the second path is

file:/foo/bar/baz/org1/xsd/org1/xsd/payload.xsd

So parts of the path end up doubled, and Xerces cannot actually find the

resource. We thought this was a bug in Xerces and added heuristics to

remove these doubled paths, but upon further inspection, it was just a

bug in how we created Input's that Xerces used.

This patch fixes this so that the Input sets the systemId to the

resolved absolute URI, and does not set a baseURI. Xerces will not try

to do any relative stuff and the absolute resource will work. This also

means we can remove the heuristics that tried to remove doubled paths

(but which did not actually work in non-trivial cases).

Also, fix how arrayIndexStack and occursBoundsStack modifications are

undone when errors occurr with Arrays. Rather than the Array pushing and

a child rep parser popping on error, the array should do both the push

and pop. The logic was wrong before and things were not popped correctly

in some cases.

DFDL-1832, DFDL-1183

    • -0
    • +23
    /test-stdLayout/src/test/scala/org1/TestOrg1.scala
    • -0
    • +24
    /test-stdLayout/src/test/scala/org2/TestOrg2.scala
Added new test-stdLayout project for testing include/import.

This new project (within daffodil) is for testing the standard project

layout. In particular, there are include/import bugs associated with

it.

There is a test in scala-debug which fails with a path containing

.../org2/xsd/org2/xsd....

Which is clearly wrong given that the directory is

src/main/resources/org2/xsd

Tests cover cases where a schema file in

src/main/resources/org1/xsd tries to include one from

src/main/resources/org2/xsd.

And where a schema file in

src/test/resources/org2/xsd tries to include one from

src/main/resources/org2/xsd.

And where a TDML embedded schema tries to include one from

src/main/resources/org2/xsd.

DFDL-1832

    • -2
    • +2
    /eclipse-projects/test-stdLayout/.project
    • -0
    • +29
    /eclipse-projects/test-stdLayout/README.txt
    • -0
    • +20
    /test-stdLayout/src/main/resources/org2/xsd/formats.dfdl.xsd
    • -0
    • +38
    /test-stdLayout/src/main/resources/org2/xsd/payload.dfdl.xsd
    • -0
    • +30
    /test-stdLayout/src/main/resources/org2/xsd/types.dfdl.xsd
    • -0
    • +1
    /test-stdLayout/src/test/resources/org1/test-outer-data1.txt
    • -0
    • +35
    /test-stdLayout/src/test/resources/org1/testStdLayout.tdml
    • -0
    • +1
    /test-stdLayout/src/test/resources/org2/test-data1.txt
  1. … 6 more files in changeset.
Ensure the infoset exists before trying to display it in the debugger

If you run 'info infoset' before stepping at least once, there will be

no infoset to visit. This checks to make sure that an infoset exists

before trying to print it in the debugger.

DFDL-1834

Modify test to verify that dfdl:inputValueCalc and dfdl:outputValueCalc cannot be on the same element

DFDL-1027

SDE when arrays or optional elements contain dfdl:outputValueCalc property

DFDL-1546

Add license information for ICU

The ICU package does contain license information, but the link it points

to is dead. Manually override it to a link in the svn repo that works.

DFDL-1825

Fixed test embedded-with-include so it actually includes.

Had cut/paste error and the test case was requesting the wrong schema.

Throw an Not Yet Implmemented SDE for encodingErrorPolicy="error"

It isn't supported and can result in weird behavior and mark pool leaks.

Best to just not allow it.

DFDL-1828

Improve nil checking for infoset outputters

The InfosetOutputters use _.isNilled to determine if an infoset element

is nil. However, _.isNilled can throw an exception if the nilled flag

has not been set (e.g. while debugging). This could lead to thrown

exceptions during the 'info infoset' debugger command. This patch adds a

new isNilled helper function to the InfosetOutputter trait that performs

the correct logic to check if the nilled flag is set and determine if an

element is nilled, and updates all the InfosetOutputters to use this

function.

DFDL-1829

Updating copyright date in LICENSE

Updating input for automated CLI test

Updated line counter to match current file structure.

Also includes NiFi and Spark, all new DFDL schemas, etc.

    • -11
    • +18
    /daffodil-lib/src/test/scala/LineCounter.scala
Modify isError API logic to support validation errors

- isError now returns true if either there are processing errors OR

validation errors

- New methods, isProcessingError and isValidationError, are added to

give the ability to differentiate why isError is true

- Remove the canProceed() method and deprecate it in the java/scala API.

This is just a wrapper around !isError

- Rename status to processorStatus to clearly differentiate between the

new validationStatus

- Remove unused isValidationSuccess method

- Remove status from the processor constructors--this is a holdover from

when PState was immutable

DFDL-1821

  1. … 13 more files in changeset.
Added some tests for fixed length strings

Testing situations with fixed length strings with a default value and

fixed length string with a length of 0

Added simple test to verify division expressions

This has already been fixed in commit

3153e271dab64b029bbdde627bba14c42f370357 but no tests were added to

cover the example given in the bug report, dealing with number types

getting mixed up, ending in a divide by zero.

DFDL-1688

Adding tests for outputValueCalc in hidden groups.

DFDL-1546

Move TestOutputValueCalcAndAlignment from scala-new to scala

Verifies DFDL-1518

Add CLI delimiterStack test

DFDL-1283

Comment out unused imports in scala-new

Closing Ticket: Added test to add coverage for case when value is false. Moved existing test and new test to scala from scala-new.

DFDL-437

Ignore all *.sbt files in the root directory except for build.sbt

This is useful for automated build systems to add custom *.sbt files to

add customizations to the build. For example, adding custom sbt

configurations to publish to a local repository. Note that this is

necessary since our sbt configurations makes a release a SNAPSHOT if

there are any untracked files (e.g. custom sbt file) or files with

changes. If we don't ignore this *.sbt files, then it will also build a

SNAPSHOT. This all helps Daffodil build cleanly on a jenkins automated

build and publishing to artifactory.

DFDL-1666

Fix test to have correct value and move out of debug

F7FF came frome the specification, but it is wrong. It should be F7DC.

Fix the tests and moved it out of debug.

DFDL-1455