Uploaded image for project: 'Daffodil'
  1. Daffodil
  2. DFDL-1458

pull-style unparser



    • Type: Wish
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: never
    • Component/s: API, Back End, Unparsing
    • Labels:


      Using java.io.PipedOutputStream, java.io.PipedInputStream, and InvertControl, we should be able to implement a pull-style unparser where normal reading from some inputStream in effect drives the unparsing to an outputStream that feeds into the input stream.

      val inputStream: InputStream = DataProcessor.pullUnparser(infosetCursor)

      Reading bytes from the inputStream will force pulling of buffers of data from an internal output stream and co-routines with the unparser that is populating the outputStream.

      A throw of an UnparseError must occur on the main co-routine if the pullUnparser encounters such. This will occur when data being pulled contains the representation of the element whose unparsing creates the error, not before.

      Ex: if there's 100 unparsed bytes in the buffer, but what's next should be the representation of an outputValueCalc element whose expression caused an UnparseError. Well the 100 bytes should be delivered all the way to the inputStream and from it to the application reading from it. Only on the next read call on the inputStream should that cause the UnparseError to be issued.

      However, the error thrown may aggregate multiple UnparseErrors & RuntimeSchemaDefinitionErrors, as not only the first one whose element creates the error should be reported, but potentially there are others from unparsing of later elements (into buffers) that are relevant to understanding what went wrong while unparsing.

      A buffered output stream "block size" can avoid excessive context switching by delaying the resume of the consuming co-routine (main, which is reading from the inputStream) until a block of data has been produced by the unparser. (Note that the minimum block size would be 1 byte, even though unparsing may occur at the level of individual bits.)

        Gliffy Diagrams




              Unassigned Unassigned
              mbeckerle.dfdl Mike Beckerle
              0 Vote for this issue
              1 Start watching this issue