dfdl

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Added test_bitOrderOVC1 which illustrates DFDL-1843

This is a Runtime SDE due to an interaction of bitOrder with

outputValueCalc, discovered originally in the Link16 schema, but

isolated here outside of that context.

DFDL-1843

Checkpoint

Adding tests for tunables and fixing typo in setTunable

DFDL-1143

DaffodilTunables is no longer a globally accessed class.

SSRD/ERD, PState/UState, Compiler, DataProcessor receive

an immutable instance of DaffodilTunables.

DFDL-1143

  1. … 49 more files in changeset.
DaffodilTunables is no longer a globally accessed class.

SSRD/ERD, PState/UState, Compiler, DataProcessor receive

an immutable instance of DaffodilTunables.

DFDL-1143

  1. … 49 more files in changeset.
Review fixes!

DFDL-1143

Review fixes!

DFDL-1143

Review fixes! DaffodilTunables now a case class to leverage copy constructor with named values.

DFDL-1143

Review fixes! Correct so that the suppression/warnID code works.

DFDL-1143

Review fixes! DaffodilTunables now immutable.

DFDL-1143

Fix xs:hexBinary equality check

Before any sort of equality check on a xs:hexBinary element would need

to be converted to a string. I have now added an equality checker for

byte arrays that allows direct comparison of xs:hexBinary elements

without the need for conversion.

DFDL-1618

Fix xs:hexBinary equality check

Before any sort of equality check on a xs:hexBinary element would need

to be converted to a string. I have now added an equality checker for

byte arrays that allows direct comparison of xs:hexBinary elements

without the need for conversion.

DFDL-1618

Fix eclipse classpath errors.

    • -25
    • +0
    /eclipse-projects/test-ibm1/.classpath
Review fixes!

DFDL-1143

Review fixes!

DFDL-1143

  1. … 11 more files in changeset.
Updates DataProcessor to obtain its tunables from the SSRD/ERD. These are the initial compiler-set values.

Whenever DataProcessor updates tunables, it updates

the values in the SSRD/ERD.

When parse is called, the initial states are creating

using the values from the SSRD/ERD at the time of

the call.

Adds clone method to DaffodilTunableParameters.

Adds a test verifying the cloning appears to work.

DFDL-1143

Fixup! Review fixes!

DFDL-1143

Fixup! Review fixes.

DFDL-1143

Completes changes to remove DaffodilTunables as a Global object.

DFDL-1143

  1. … 33 more files in changeset.
Ensure Bit/ByteOrderChange parsers always exist at root

If the dfdl:byteOrder or dfdl:bitOrder properties were not provided,

Daffodil did not complain, and it would not insert a Bit/ByteOrderChange

parser at root. This meant that Daffodil would use a default value,

which in some cases caused incorrect behavior. It also meant that the

check for valid bit and byte orders (e.g. not bigEndian &

leastSignificantBitFirst) never occurred since that only happend in

ByteOrderChange parsers, meaning it could allow invalid bit/byte order

combinations.

This changes Daffodil so the root element will always insert Bit and

ByteOrderChange parsers, ensuring that the validity of the properties

are correct and ensuring correct parsing/unparsing. This means that

byteOrder is now always required, even in some cases where it might not

technically be needed.

This also modifies the initial UState/PState creation to not set

bit/byte order, but to instead rely on the Bit/ByteOrderChange parsers

to always be inserted first thing.

DFDL-1835

Ensure Bit/ByteOrderChange parsers always exist at root

If the dfdl:byteOrder or dfdl:bitOrder properties were not provided,

Daffodil did not complain, and it would not insert a Bit/ByteOrderChange

parser at root. This meant that Daffodil would use a default value,

which in some cases caused incorrect behavior. It also meant that the

check for valid bit and byte orders (e.g. not bigEndian &

leastSignificantBitFirst) never occurred since that only happend in

ByteOrderChange parsers, meaning it could allow invalid bit/byte order

combinations.

This changes Daffodil so the root element will always insert Bit and

ByteOrderChange parsers, ensuring that the validity of the properties

are correct and ensuring correct parsing/unparsing. This means that

byteOrder is now always required, even in some cases where it might not

technically be needed.

This also modifies the initial UState/PState creation to not set

bit/byte order, but to instead rely on the Bit/ByteOrderChange parsers

to always be inserted first thing.

DFDL-1835

Calculate the approximate length of delimited and pattern lengthKinds for alignment

By assuming the length of delimited and pattern lengthKinds was 1,

Daffodil was unable to optimize out some aligngmnet unparsers, which

could lead to deadlocks.

This changes that so the length of delimited or pattern lengthKinds is a

multiple of the length of the encoding. This allows Daffodil to optimize

out some alignment unparsers and prevent deadlocks.

DFDL-1836

Calculate the approximate length of delimited and pattern lengthKinds for alignment

By assuming the length of delimited and pattern lengthKinds was 1,

Daffodil was unable to optimize out some alignment unparsers, which

could lead to deadlocks.

This changes that so the length of delimited or pattern lengthKinds is a

multiple of the length of the encoding. This allows Daffodil to optimize

out some alignment unparsers and prevent deadlocks.

DFDL-1836

daffodil-core, runtime, lib and io all compile.

  1. … 48 more files in changeset.
Removed unnecessary references to tunables now that it hangs off SchemaComponent.

  1. … 4 more files in changeset.
Changes before Expressions rabbit hole dealing with use of tunable in step evaluation.

  1. … 51 more files in changeset.
Adding test for bit order without byte order specified

DFDL-1468

fixup! Fix Xerces resource resolution for includes/imports

    • -0
    • +24
    /test-stdLayout/src/test/scala/org2/TestOrg2.scala
Fix Xerces resource resolution for includes/imports

After we found a resource (like from an include/import), we reported the

location to Xerces incorrectly. For example, say we resolved the

systemID at

org1/xsd/payload.xsd

to the absolute URI at

file:/foo/bar/baz/org1/xsd/payload.xsd

When we created the new Input to return where we found payload.xsd to

Xerces, we set the systemID to the original relative systemID

(org/xsd/payload.xsd), and set the baseURI to the absolute URI

(file:/foo/bar/baz/org1/xsd/payload.xsd). To Xerces, this meant that the

resource was found at the first path relative to the second path. But

the first path relative to the second path is

file:/foo/bar/baz/org1/xsd/org1/xsd/payload.xsd

So parts of the path end up doubled, and Xerces cannot actually find the

resource. We thought this was a bug in Xerces and added heuristics to

remove these doubled paths, but upon further inspection, it was just a

bug in how we created Input's that Xerces used.

This patch fixes this so that the Input sets the systemId to the

resolved absolute URI, and does not set a baseURI. Xerces will not try

to do any relative stuff and the absolute resource will work. This also

means we can remove the heuristics that tried to remove doubled paths

(but which did not actually work in non-trivial cases).

Also, fix how arrayIndexStack and occursBoundsStack modifications are

undone when errors occurr with Arrays. Rather than the Array pushing and

a child rep parser popping on error, the array should do both the push

and pop. The logic was wrong before and things were not popped correctly

in some cases.

DFDL-1832, DFDL-1183

    • -0
    • +0
    /test-stdLayout/src/test/scala/TestOrg2.scala
    • -0
    • +23
    /test-stdLayout/src/test/scala/org1/TestOrg1.scala
Fix Xerces resource resolution for includes/imports

After we found a resource (like from an include/import), we reported the

location to Xerces incorrectly. For example, say we resolved the

systemID at

org1/xsd/payload.xsd

to the absolute URI at

file:/foo/bar/baz/org1/xsd/payload.xsd

When we created the new Input to return where we found payload.xsd to

Xerces, we set the systemID to the original relative systemID

(org/xsd/payload.xsd), and set the baseURI to the absolute URI

(file:/foo/bar/baz/org1/xsd/payload.xsd). To Xerces, this meant that the

resource was found at the first path relative to the second path. But

the first path relative to the second path is

file:/foo/bar/baz/org1/xsd/org1/xsd/payload.xsd

So parts of the path end up doubled, and Xerces cannot actually find the

resource. We thought this was a bug in Xerces and added heuristics to

remove these doubled paths, but upon further inspection, it was just a

bug in how we created Input's that Xerces used.

This patch fixes this so that the Input sets the systemId to the

resolved absolute URI, and does not set a baseURI. Xerces will not try

to do any relative stuff and the absolute resource will work. This also

means we can remove the heuristics that tried to remove doubled paths

(but which did not actually work in non-trivial cases).

Also, fix how arrayIndexStack and occursBoundsStack modifications are

undone when errors occurr with Arrays. Rather than the Array pushing and

a child rep parser popping on error, the array should do both the push

and pop. The logic was wrong before and things were not popped correctly

in some cases.

DFDL-1832, DFDL-1183

    • -0
    • +23
    /test-stdLayout/src/test/scala/org1/TestOrg1.scala
    • -0
    • +24
    /test-stdLayout/src/test/scala/org2/TestOrg2.scala