dfdl

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Review fixes!

DFDL-1143

Review fixes! DaffodilTunables now a case class to leverage copy constructor with named values.

DFDL-1143

Review fixes! Correct so that the suppression/warnID code works.

DFDL-1143

Review fixes! DaffodilTunables now immutable.

DFDL-1143

Fix xs:hexBinary equality check

Before any sort of equality check on a xs:hexBinary element would need

to be converted to a string. I have now added an equality checker for

byte arrays that allows direct comparison of xs:hexBinary elements

without the need for conversion.

DFDL-1618

Fix xs:hexBinary equality check

Before any sort of equality check on a xs:hexBinary element would need

to be converted to a string. I have now added an equality checker for

byte arrays that allows direct comparison of xs:hexBinary elements

without the need for conversion.

DFDL-1618

Fix eclipse classpath errors.

    • -25
    • +0
    /eclipse-projects/test-ibm1/.classpath
Review fixes!

DFDL-1143

Review fixes!

DFDL-1143

  1. … 11 more files in changeset.
Updates DataProcessor to obtain its tunables from the SSRD/ERD. These are the initial compiler-set values.

Whenever DataProcessor updates tunables, it updates

the values in the SSRD/ERD.

When parse is called, the initial states are creating

using the values from the SSRD/ERD at the time of

the call.

Adds clone method to DaffodilTunableParameters.

Adds a test verifying the cloning appears to work.

DFDL-1143

Fixup! Review fixes!

DFDL-1143

Fixup! Review fixes.

DFDL-1143

Completes changes to remove DaffodilTunables as a Global object.

DFDL-1143

  1. … 33 more files in changeset.
Ensure Bit/ByteOrderChange parsers always exist at root

If the dfdl:byteOrder or dfdl:bitOrder properties were not provided,

Daffodil did not complain, and it would not insert a Bit/ByteOrderChange

parser at root. This meant that Daffodil would use a default value,

which in some cases caused incorrect behavior. It also meant that the

check for valid bit and byte orders (e.g. not bigEndian &

leastSignificantBitFirst) never occurred since that only happend in

ByteOrderChange parsers, meaning it could allow invalid bit/byte order

combinations.

This changes Daffodil so the root element will always insert Bit and

ByteOrderChange parsers, ensuring that the validity of the properties

are correct and ensuring correct parsing/unparsing. This means that

byteOrder is now always required, even in some cases where it might not

technically be needed.

This also modifies the initial UState/PState creation to not set

bit/byte order, but to instead rely on the Bit/ByteOrderChange parsers

to always be inserted first thing.

DFDL-1835

Ensure Bit/ByteOrderChange parsers always exist at root

If the dfdl:byteOrder or dfdl:bitOrder properties were not provided,

Daffodil did not complain, and it would not insert a Bit/ByteOrderChange

parser at root. This meant that Daffodil would use a default value,

which in some cases caused incorrect behavior. It also meant that the

check for valid bit and byte orders (e.g. not bigEndian &

leastSignificantBitFirst) never occurred since that only happend in

ByteOrderChange parsers, meaning it could allow invalid bit/byte order

combinations.

This changes Daffodil so the root element will always insert Bit and

ByteOrderChange parsers, ensuring that the validity of the properties

are correct and ensuring correct parsing/unparsing. This means that

byteOrder is now always required, even in some cases where it might not

technically be needed.

This also modifies the initial UState/PState creation to not set

bit/byte order, but to instead rely on the Bit/ByteOrderChange parsers

to always be inserted first thing.

DFDL-1835

Calculate the approximate length of delimited and pattern lengthKinds for alignment

By assuming the length of delimited and pattern lengthKinds was 1,

Daffodil was unable to optimize out some aligngmnet unparsers, which

could lead to deadlocks.

This changes that so the length of delimited or pattern lengthKinds is a

multiple of the length of the encoding. This allows Daffodil to optimize

out some alignment unparsers and prevent deadlocks.

DFDL-1836

Calculate the approximate length of delimited and pattern lengthKinds for alignment

By assuming the length of delimited and pattern lengthKinds was 1,

Daffodil was unable to optimize out some alignment unparsers, which

could lead to deadlocks.

This changes that so the length of delimited or pattern lengthKinds is a

multiple of the length of the encoding. This allows Daffodil to optimize

out some alignment unparsers and prevent deadlocks.

DFDL-1836

daffodil-core, runtime, lib and io all compile.

  1. … 48 more files in changeset.
Removed unnecessary references to tunables now that it hangs off SchemaComponent.

  1. … 4 more files in changeset.
Changes before Expressions rabbit hole dealing with use of tunable in step evaluation.

  1. … 51 more files in changeset.
Adding test for bit order without byte order specified

DFDL-1468

fixup! Fix Xerces resource resolution for includes/imports

    • -0
    • +24
    /test-stdLayout/src/test/scala/org2/TestOrg2.scala
Fix Xerces resource resolution for includes/imports

After we found a resource (like from an include/import), we reported the

location to Xerces incorrectly. For example, say we resolved the

systemID at

org1/xsd/payload.xsd

to the absolute URI at

file:/foo/bar/baz/org1/xsd/payload.xsd

When we created the new Input to return where we found payload.xsd to

Xerces, we set the systemID to the original relative systemID

(org/xsd/payload.xsd), and set the baseURI to the absolute URI

(file:/foo/bar/baz/org1/xsd/payload.xsd). To Xerces, this meant that the

resource was found at the first path relative to the second path. But

the first path relative to the second path is

file:/foo/bar/baz/org1/xsd/org1/xsd/payload.xsd

So parts of the path end up doubled, and Xerces cannot actually find the

resource. We thought this was a bug in Xerces and added heuristics to

remove these doubled paths, but upon further inspection, it was just a

bug in how we created Input's that Xerces used.

This patch fixes this so that the Input sets the systemId to the

resolved absolute URI, and does not set a baseURI. Xerces will not try

to do any relative stuff and the absolute resource will work. This also

means we can remove the heuristics that tried to remove doubled paths

(but which did not actually work in non-trivial cases).

Also, fix how arrayIndexStack and occursBoundsStack modifications are

undone when errors occurr with Arrays. Rather than the Array pushing and

a child rep parser popping on error, the array should do both the push

and pop. The logic was wrong before and things were not popped correctly

in some cases.

DFDL-1832, DFDL-1183

    • -0
    • +0
    /test-stdLayout/src/test/scala/TestOrg2.scala
    • -0
    • +23
    /test-stdLayout/src/test/scala/org1/TestOrg1.scala
Fix Xerces resource resolution for includes/imports

After we found a resource (like from an include/import), we reported the

location to Xerces incorrectly. For example, say we resolved the

systemID at

org1/xsd/payload.xsd

to the absolute URI at

file:/foo/bar/baz/org1/xsd/payload.xsd

When we created the new Input to return where we found payload.xsd to

Xerces, we set the systemID to the original relative systemID

(org/xsd/payload.xsd), and set the baseURI to the absolute URI

(file:/foo/bar/baz/org1/xsd/payload.xsd). To Xerces, this meant that the

resource was found at the first path relative to the second path. But

the first path relative to the second path is

file:/foo/bar/baz/org1/xsd/org1/xsd/payload.xsd

So parts of the path end up doubled, and Xerces cannot actually find the

resource. We thought this was a bug in Xerces and added heuristics to

remove these doubled paths, but upon further inspection, it was just a

bug in how we created Input's that Xerces used.

This patch fixes this so that the Input sets the systemId to the

resolved absolute URI, and does not set a baseURI. Xerces will not try

to do any relative stuff and the absolute resource will work. This also

means we can remove the heuristics that tried to remove doubled paths

(but which did not actually work in non-trivial cases).

Also, fix how arrayIndexStack and occursBoundsStack modifications are

undone when errors occurr with Arrays. Rather than the Array pushing and

a child rep parser popping on error, the array should do both the push

and pop. The logic was wrong before and things were not popped correctly

in some cases.

DFDL-1832, DFDL-1183

    • -0
    • +23
    /test-stdLayout/src/test/scala/org1/TestOrg1.scala
    • -0
    • +24
    /test-stdLayout/src/test/scala/org2/TestOrg2.scala
Added new test-stdLayout project for testing include/import.

This new project (within daffodil) is for testing the standard project

layout. In particular, there are include/import bugs associated with

it.

There is a test in scala-debug which fails with a path containing

.../org2/xsd/org2/xsd....

Which is clearly wrong given that the directory is

src/main/resources/org2/xsd

Tests cover cases where a schema file in

src/main/resources/org1/xsd tries to include one from

src/main/resources/org2/xsd.

And where a schema file in

src/test/resources/org2/xsd tries to include one from

src/main/resources/org2/xsd.

And where a TDML embedded schema tries to include one from

src/main/resources/org2/xsd.

DFDL-1832

    • -2
    • +2
    /eclipse-projects/test-stdLayout/.project
    • -0
    • +29
    /eclipse-projects/test-stdLayout/README.txt
    • -0
    • +20
    /test-stdLayout/src/main/resources/org2/xsd/formats.dfdl.xsd
    • -0
    • +38
    /test-stdLayout/src/main/resources/org2/xsd/payload.dfdl.xsd
    • -0
    • +30
    /test-stdLayout/src/main/resources/org2/xsd/types.dfdl.xsd
    • -0
    • +1
    /test-stdLayout/src/test/resources/org1/test-outer-data1.txt
    • -0
    • +35
    /test-stdLayout/src/test/resources/org1/testStdLayout.tdml
    • -0
    • +1
    /test-stdLayout/src/test/resources/org2/test-data1.txt
  1. … 6 more files in changeset.
Added new test-stdLayout project for testing include/import.

This new project (within daffodil) is for testing the standard project

layout. In particular, there are include/import bugs associated with

it.

There is a test in scala-debug which fails with a path containing

.../org2/xsd/org2/xsd....

Which is clearly wrong given that the directory is

src/main/resources/org2/xsd

DFDL-1832

    • -2
    • +2
    /eclipse-projects/test-stdLayout/.project
    • -0
    • +20
    /test-stdLayout/src/main/resources/org2/xsd/formats.dfdl.xsd
    • -0
    • +38
    /test-stdLayout/src/main/resources/org2/xsd/payload.xsd
    • -0
    • +30
    /test-stdLayout/src/main/resources/org2/xsd/types.dfdl.xsd
    • -0
    • +1
    /test-stdLayout/src/test/resources/org1/test-outer-data1.txt
    • -0
    • +35
    /test-stdLayout/src/test/resources/org1/testStdLayout.tdml
    • -0
    • +1
    /test-stdLayout/src/test/resources/org2/test-data1.txt
    • -0
    • +21
    /test-stdLayout/src/test/scala-debug/org1/TestOuter.scala
    • -0
    • +21
    /test-stdLayout/src/test/scala/org2/TestPayloadAndTypes.scala
Added new test-stdLayout project for testing include/import.

This new project (within daffodil) is for testing the standard project

layout. In particular, there are include/import bugs associated with

it.

There is a test in scala-debug which fails with a path containing

.../org2/xsd/org2/xsd....

Which is clearly wrong given that the directory is

src/main/resources/org2/xsd

Tests cover cases where a schema file in

src/main/resources/org1/xsd tries to include one from

src/main/resources/org2/xsd.

And where a schema file in

src/test/resources/org2/xsd tries to include one from

src/main/resources/org2/xsd.

And where a TDML embedded schema tries to include one from

src/main/resources/org2/xsd.

DFDL-1832

    • -2
    • +2
    /eclipse-projects/test-stdLayout/.project
    • -0
    • +29
    /eclipse-projects/test-stdLayout/README.txt
    • -0
    • +38
    /test-stdLayout/src/main/resources/org2/xsd/payload.dfdl.xsd
  1. … 6 more files in changeset.
Replaces global references to tunables with class instances. Removes unused imports. Adds necessary class structures to daffodil-lib for use between compile-time and run-time.

DFDL-1143

  1. … 14 more files in changeset.
Ensure the infoset exists before trying to display it in the debugger

If you run 'info infoset' before stepping at least once, there will be

no infoset to visit. This checks to make sure that an infoset exists

before trying to print it in the debugger.

DFDL-1834

Modify test to verify that dfdl:inputValueCalc and dfdl:outputValueCalc cannot be on the same element

DFDL-1027