Uploaded image for project: 'Daffodil'
  1. Daffodil
  2. DFDL-1809

Need MS-Windows Unicode support even just for "sbt test"

XMLWordPrintableJSON

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Normal Normal
    • 2.0.0
    • 2.0.0
    • Documentation
    • None
    • MS Windows

      A user reports sbt test fails.

      TestAlignedData.test_fillByte_04 fails as follows:

      [info] Test edu.illinois.ncsa.daffodil.section12.aligned_data.TestAlignedData.test_fillByte_04 started
      [error] Test edu.illinois.ncsa.daffodil.section12.aligned_data.TestAlignedData.test_fillByte_04 failed: edu.illinois.ncsa.daffodil.tdml.TDMLException: "Diagnostic message(s) were expected but not found."
      [error] Expected: Schema Definition Error
      [error] fillByte
      [error] single-byte character
      [error] encoding
      [error] UTF-8
      [error] ?
      [error] 3 bytes, took 0.078 sec
      [error]     at edu.illinois.ncsa.daffodil.tdml.VerifyTestCase$.verifyAllDiagnosticsFound(TDMLRunner.scala:1107)
      [error]     at edu.illinois.ncsa.daffodil.tdml.UnparserTestCase.runUnparserExpectErrors(TDMLRunner.scala:1030)
      [error]     at edu.illinois.ncsa.daffodil.tdml.UnparserTestCase$$anonfun$runProcessor$10.apply(TDMLRunner.scala:905)
      [error]     at edu.illinois.ncsa.daffodil.tdml.UnparserTestCase$$anonfun$runProcessor$10.apply(TDMLRunner.scala:903)
      [error]     at scala.util.Either$RightProjection.foreach(Either.scala:468)
      [error]     at edu.illinois.ncsa.daffodil.tdml.UnparserTestCase.runProcessor(TDMLRunner.scala:903)
      [error]     at edu.illinois.ncsa.daffodil.tdml.TestCase.run(TDMLRunner.scala:623)
      [error]     at edu.illinois.ncsa.daffodil.tdml.DFDLTestSuite.runOneTestWithDataVolumes(TDMLRunner.scala:339)
      [error]     at edu.illinois.ncsa.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:327)
      [error]     at edu.illinois.ncsa.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:122)
      [error]     at edu.illinois.ncsa.daffodil.section12.aligned_data.TestAlignedData.test_fillByte_04(TestAlignedData.scala:206)

      This happens because the test is a negative test, looking for the SDE associated with using a multi-byte character as the fill-byte. The test has dfdl:fillByte="年" (that's a japanese kanji, in case it looks like something odd.).

      On MS-Windows, if you don't have Unicode support installed, then that schema file won't read correctly. That kanji will be substituted by a "?", and that allows the test to run without issuing the SDE that fill byte must be a single byte character. Hence, the test fails.

      Conclusion: We must update wiki instructions to insist that MS Windows users install the Japanese Language Pack. Currently it's just a final detail on the 2nd of two Eclipse-setup pages. But this is needed even if you are just running sbt.

              jadams Joshua Adams
              mbeckerle.dfdl Mike Beckerle
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: