I've been creating tests for XPath numeric functions (abs, floor, ceiling, etc) and came across a strange issue when testing round-half-to-even:
I set up a schema:
<xs:element name="round-hte">
|
<xs:complexType>
|
<xs:sequence dfdl:separator=",">
|
<xs:element name="num" type="xs:float" dfdl:lengthKind="delimited"/>
|
<xs:element name="precision" type="xs:int" dfdl:lengthKind="delimited"/>
|
<xs:element name="roundval" type="xs:float" dfdl:inputValueCalc="{ fn:round-half-to-even(../ex:num,../ex:precision) }"/>
|
</xs:sequence>
|
</xs:complexType>
|
</xs:element>
|
I have a test with the following as input data:
3.0000075,6
|
so the first element in my infoset should just be the input data, but I get this:
<num>3.0000074</num>
|
I have no idea why the 5 is getting changed to a 4, but it happens in another test I created as well.
See test_xPathFunc_round_hte_07 and test_xPathFunc_round_hte_03. It also appears that floats have a decimal precision limit of 7 digits (see test_xPathFunc_round_hte_08) in:
M daffodil-test/src/test/resources/edu/illinois/ncsa/daffodil/section23/dfdl_functions/Functions.tdml
M daffodil-test/src/test/scala-debug/edu/illinois/ncsa/daffodil/section23/dfdl_expressions/TestDFDLExpressionsDebug.scala
M daffodil-test/src/test/scala/edu/illinois/ncsa/daffodil/section23/dfdl_expressions/TestDFDLExpressions.scala