Daffodil is the first DFDL implementation to provide the advanced features of
- dfdl:inputValueCalc (IVC for short)
- dfdl:outputValueCalc (OVC for short)
- dfdl:hiddenGroupRef
In addition Daffodil implements almost the complete DFDL expression language, including all the xpath functions, and the DFDL-namespaced functions such as dfdl:valueLength and dfdl:contentLength (though with some restrictions - as of this writing - on when length units of 'characters' can be specified.)
These features are essential for highly complex binary formats like:
- PCAP
- Asterix
- MIL-STD-2045 and related MIL-STD and STANAGs
This page is a location to centralize notes about implementation of advanced DFDL features, and to highlight places where the DFDL specification may need to be clarified, augmented, or corrected.
SDE for element in hidden group that has no default, nor dfdl:outputValueCalc
Within a hidden group, if an element has no fixed/default value, and no OVC, then because it is hidden and so does not appear in the infoset, there's no way for it to get a value at all when unparsing.
Daffodil issues an SDE in this situation.
TBD: this SDE may be too strong: If the element has IVC, then it doesn't need a value for unparsing unless an expression that is used at unparse time (i.e., not in an assert or discriminator) references the element.
IVC and OVC on the same element
In many situations we have found that we need both IVC and OVC on the same element. This occurs when the element is in a hidden group, and is essentially being used as a variable.
The dfdl:newVariableInstance doesn't eliminate this problem, as there are situations where one needs an "array variable", that is a variable associated with each index of an array, but referenced from expressions used to compute things outside of the scope of that array element.
TBD: is OVC truly needed here, or is this a result of the SDE above for elements that have no way to get a value at unparse time - but the element turns out to be unused at unparse time.
Array Variables and dfdl:newVariableInstance
In many cases dfdl:newVariableInstance is not sufficient because what are needed are array variables.
.e., when parsing element i+1 you have to refer to a "variable" defined as part of element i.
You can't do that with dfdl:newVariableInstance because of the scope it has. If you introduce a dfdl:newVariableInstance for an array element, it's scope will be that element only. If you introduce it outside the array element there will be only one instance for the whole array.
But you can do an "array variable" with a hidden element. (Doesn't have to be hidden, but that's the sensible way to use it.)
The headache being the relative path to it., and then the fact that the most natural thing to do is to put both IVC and OVC on it (though the OVC may not be needed in many cases).