Uploaded image for project: 'Daffodil'
  1. Daffodil
  2. DFDL-1403

CLI Performance: Exceptions when parsing binary files with multiple threads

XMLWordPrintableJSON

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Normal Normal
    • 2.0.0
    • 2.0.0
    • Back End
    • None

      The pcap performance tests are experiencing errors. The errors vary depending on how many files are being parsed/threads are being used.

      2 threads and 10 files:

      [~/daffodil] (1.2.0)$ ~/daffodil/daffodil-cli/target/universal/stage/bin/daffodil performance  -s ~/daffodil/daffodil-examples/src/test/resources/edu/illinois/ncsa/daffodil/pcap/pcap.dfdl.xsd -t 2 -N 10 ~/daffodil/daffodil-examples/src/test/resources/edu/illinois/ncsa/daffodil/pcap/icmp.cap
       
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
      !!   An unexpected exception occurred. This is a bug!   !!
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
       
       Please report this bug and help us fix it:
       
        https://opensource.ncsa.illinois.edu/confluence/display/DFDL/How+to+Report+a+Bug
       
       Please include the following exception, the command you
       ran, and any input, schema, or tdml files used that led
       to this bug.
       
       
      edu.illinois.ncsa.daffodil.exceptions.Abort: Invariant broken. Should not be allowing propagation of java.lang.IndexOutOfBoundsException
      edu.illinois.ncsa.daffodil.exceptions.Assert$.abort(Assert.scala:105)
      edu.illinois.ncsa.daffodil.exceptions.Assert$.invariantFailed(Assert.scala:146)
      edu.illinois.ncsa.daffodil.processors.WithParseErrorThrowing$class.withParseErrorThrowing(ParseErrors.scala:204)
      edu.illinois.ncsa.daffodil.processors.PrimParser.withParseErrorThrowing(ProcessorBases1.scala:45)
      	at edu.illinois.ncsa.daffodil.exceptions.Assert$.abort(Assert.scala:105)
      	at edu.illinois.ncsa.daffodil.exceptions.Assert$.invariantFailed(Assert.scala:146)
      	at edu.illinois.ncsa.daffodil.processors.WithParseErrorThrowing$class.withParseErrorThrowing(ParseErrors.scala:204)
      	at edu.illinois.ncsa.daffodil.processors.PrimParser.withParseErrorThrowing(ProcessorBases1.scala:45)
      	at edu.illinois.ncsa.daffodil.processors.parsers.BinaryNumberBaseParser.parse(BinaryNumberParsers.scala:203)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.edu$illinois$ncsa$daffodil$processors$DataProcessor$$doParse(Runtime.scala:244)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor$$anonfun$parse$3.apply(Runtime.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor$$anonfun$parse$3.apply(Runtime.scala:206)
      	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
      	at edu.illinois.ncsa.daffodil.ExecutionMode$$anonfun$usingRuntimeMode$1.apply(ExecutionMode.scala:80)
      	at edu.illinois.ncsa.daffodil.ExecutionMode$$anonfun$usingRuntimeMode$1.apply(ExecutionMode.scala:80)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.parse(Runtime.scala:205)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.parse(Runtime.scala:189)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85$$anonfun$87.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85$$anonfun$87.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.util.Timer$.getTimeResult(Timer.scala:91)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85.apply(Main.scala:870)
      	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
      	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)

      10 threads 100 files:

      [~/daffodil] (1.2.0)$ ~/daffodil/daffodil-cli/target/universal/stage/bin/daffodil performance  -s ~/daffodil/daffodil-examples/src/test/resources/edu/illinois/ncsa/daffodil/pcap/pcap.dfdl.xsd -t 10 -N 100 ~/daffodil/daffodil-examples/src/test/resources/edu/illinois/ncsa/daffodil/pcap/icmp.cap
      Unexpected throw of java.nio.BufferOverflowException
      Unexpected throw of java.nio.BufferOverflowException
      Unexpected throw of java.nio.BufferOverflowException
      Unexpected throw of java.nio.BufferOverflowException
      Unexpected throw of java.lang.IllegalArgumentException
       
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
      !!   An unexpected exception occurred. This is a bug!   !!
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
       
       Please report this bug and help us fix it:
       
        https://opensource.ncsa.illinois.edu/confluence/display/DFDL/How+to+Report+a+Bug
       
       Please include the following exception, the command you
       ran, and any input, schema, or tdml files used that led
       to this bug.
       
       
      java.nio.BufferOverflowException
      	at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:206)
      	at edu.illinois.ncsa.daffodil.io.LongConverter$class.populateSmallBuf(ByteBufferDataInputStream.scala:1290)
      	at edu.illinois.ncsa.daffodil.io.Converter_LE_MSBFirst$.populateSmallBuf(ByteBufferDataInputStream.scala:1350)
      	at edu.illinois.ncsa.daffodil.io.Converter_LE_MSBFirst$.getSignedLong(ByteBufferDataInputStream.scala:1353)
      	at edu.illinois.ncsa.daffodil.io.ByteBufferDataInputStream.getSignedLong(ByteBufferDataInputStream.scala:616)
      	at edu.illinois.ncsa.daffodil.io.ByteBufferDataInputStream.getSignedBigInt(ByteBufferDataInputStream.scala:641)
      	at edu.illinois.ncsa.daffodil.io.ByteBufferDataInputStream.getUnsignedBigInt(ByteBufferDataInputStream.scala:698)
      	at edu.illinois.ncsa.daffodil.processors.parsers.BinaryNumberBaseParser$$anonfun$parse$1.apply$mcV$sp(BinaryNumberParsers.scala:212)
      	at edu.illinois.ncsa.daffodil.processors.WithParseErrorThrowing$class.withParseErrorThrowing(ParseErrors.scala:201)
      	at edu.illinois.ncsa.daffodil.processors.PrimParser.withParseErrorThrowing(ProcessorBases1.scala:45)
      	at edu.illinois.ncsa.daffodil.processors.parsers.BinaryNumberBaseParser.parse(BinaryNumberParsers.scala:203)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.edu$illinois$ncsa$daffodil$processors$DataProcessor$$doParse(Runtime.scala:244)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor$$anonfun$parse$3.apply(Runtime.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor$$anonfun$parse$3.apply(Runtime.scala:206)
      	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
      	at edu.illinois.ncsa.daffodil.ExecutionMode$$anonfun$usingRuntimeMode$1.apply(ExecutionMode.scala:80)
      	at edu.illinois.ncsa.daffodil.ExecutionMode$$anonfun$usingRuntimeMode$1.apply(ExecutionMode.scala:80)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.parse(Runtime.scala:205)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.parse(Runtime.scala:189)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85$$anonfun$87.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85$$anonfun$87.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.util.Timer$.getTimeResult(Timer.scala:91)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85.apply(Main.scala:870)
      	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
      	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)

      10 threads 10,000 files:

      [~/daffodil] (1.2.0)$ ~/daffodil/daffodil-cli/target/universal/stage/bin/daffodil performance  -s ~/daffodil/daffodil-examples/src/test/resources/edu/illinois/ncsa/daffodil/pcap/pcap.dfdl.xsd -t 10 -N 10000 ~/daffodil/daffodil-examples/src/test/resources/edu/illinois/ncsa/daffodil/pcap/icmp.cap
      Unexpected throw of java.nio.BufferOverflowException
      Unexpected throw of java.nio.BufferOverflowException
      Unexpected throw of java.nio.BufferOverflowException
       
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
      !!   An unexpected exception occurred. This is a bug!   !!
      !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
       
       Please report this bug and help us fix it:
       
        https://opensource.ncsa.illinois.edu/confluence/display/DFDL/How+to+Report+a+Bug
       
       Please include the following exception, the command you
       ran, and any input, schema, or tdml files used that led
       to this bug.
       
       
      edu.illinois.ncsa.daffodil.exceptions.Abort: Invariant broken. Should not be allowing propagation of java.lang.IndexOutOfBoundsException
      edu.illinois.ncsa.daffodil.exceptions.Assert$.abort(Assert.scala:105)
      edu.illinois.ncsa.daffodil.exceptions.Assert$.invariantFailed(Assert.scala:146)
      edu.illinois.ncsa.daffodil.processors.WithParseErrorThrowing$class.withParseErrorThrowing(ParseErrors.scala:204)
      edu.illinois.ncsa.daffodil.processors.PrimParser.withParseErrorThrowing(ProcessorBases1.scala:45)
      	at edu.illinois.ncsa.daffodil.exceptions.Assert$.abort(Assert.scala:105)
      	at edu.illinois.ncsa.daffodil.exceptions.Assert$.invariantFailed(Assert.scala:146)
      	at edu.illinois.ncsa.daffodil.processors.WithParseErrorThrowing$class.withParseErrorThrowing(ParseErrors.scala:204)
      	at edu.illinois.ncsa.daffodil.processors.PrimParser.withParseErrorThrowing(ProcessorBases1.scala:45)
      	at edu.illinois.ncsa.daffodil.processors.parsers.BinaryNumberBaseParser.parse(BinaryNumberParsers.scala:203)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.SequenceCombinatorParser.parse(ElementKindParsers.scala:224)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.parsers.ComplexTypeParser.parse(ElementKindParsers.scala:71)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser$$anonfun$parse$1.apply(Parser.scala:209)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at edu.illinois.ncsa.daffodil.processors.SeqCompParser.parse(Parser.scala:209)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:140)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase$$anonfun$parse$3.apply(ElementCombinator1.scala:139)
      	at scala.Option.foreach(Option.scala:236)
      	at edu.illinois.ncsa.daffodil.processors.StatementElementParserBase.parse(ElementCombinator1.scala:139)
      	at edu.illinois.ncsa.daffodil.processors.Parser.parse1(Parser.scala:136)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.edu$illinois$ncsa$daffodil$processors$DataProcessor$$doParse(Runtime.scala:244)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor$$anonfun$parse$3.apply(Runtime.scala:213)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor$$anonfun$parse$3.apply(Runtime.scala:206)
      	at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
      	at edu.illinois.ncsa.daffodil.ExecutionMode$$anonfun$usingRuntimeMode$1.apply(ExecutionMode.scala:80)
      	at edu.illinois.ncsa.daffodil.ExecutionMode$$anonfun$usingRuntimeMode$1.apply(ExecutionMode.scala:80)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.parse(Runtime.scala:205)
      	at edu.illinois.ncsa.daffodil.processors.DataProcessor.parse(Runtime.scala:189)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85$$anonfun$87.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85$$anonfun$87.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.util.Timer$.getTimeResult(Timer.scala:91)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85.apply(Main.scala:874)
      	at edu.illinois.ncsa.daffodil.Main$$anonfun$83$$anonfun$84$$anonfun$85.apply(Main.scala:870)
      	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
      	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)

      We can split these out into separate tickets if they seem to be separate issues.

              jchab Jessie Chab
              jchab Jessie Chab
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: