Uploaded image for project: 'Daffodil'
  1. Daffodil
  2. DFDL-1359

CLI Unparse Performance - Memory Issues when running large file sets

XMLWordPrintableJSON

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • 2.0.0
    • 1.1.0
    • CLI
    • None

      When attempting to measure the performance of unparsing thousands of messages, the following error occurs:

      Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread
      at java.lang.Thread.start0(Native Method)
      at java.lang.Thread.start(Thread.java:713)
      at scala.xml.pull.XMLEventReader.<init>(XMLEventReader.scala:45)
      at edu.illinois.ncsa.daffodil.Main$$anonfun$79.apply(Main.scala:792)
      at edu.illinois.ncsa.daffodil.Main$$anonfun$79.apply(Main.scala:785)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
      at scala.collection.immutable.Range.foreach(Range.scala:141)
      at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
      at scala.collection.AbstractTraversable.map(Traversable.scala:105)
      at edu.illinois.ncsa.daffodil.Main$.run(Main.scala:785)
      at edu.illinois.ncsa.daffodil.Main$.main(Main.scala:1104)
      at edu.illinois.ncsa.daffodil.Main.main(Main.scala)

      when attempting to kill the process with Ctrl-C, the following is output:

      OpenJDK 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGINT to handler- the VM may need to be forcibly terminated

              efinnegan Elizabeth Finnegan
              jchab Jessie Chab
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: