Uploaded image for project: 'MAEviz'
  1. MAEviz
  2. MAE-364

It would be great if a FeatureCollection could write to disk as it is being created

XMLWordPrintableJSON

    • Icon: Improvement Improvement
    • Resolution: Won't Fix
    • Icon: Minor Minor
    • None
    • 2.0
    • Data Model
    • None

      This is releated to issue MAE-363. When an analysis is creating a feature dataset as a result, a FeatureCollection is being created in memory because of the way that Geotools implemented the class. This can be inefficient for large datasets that are being created in an analysis, especially since there is no good reason for the entire thing to be in memory. It would be a great addition if we could extend the FeatureCollection idea and pass it a file to write to immediately in the local cache. As a feature is created and added to the collection, it would write to disk immediately so at most, only a few features are in memory at a time (multiple outputs might lead to multiple features in memory, one for each feature dataset result). This would result in a much lighter memory footprint and more memory can be made available for other operations.

      Some part of the MAEviz system would have to handle requests for a file or files to write to that are then made available to the analysis so it can pass it to the new FeatureCollection object(s).

      We can probably learn something from looking at how geotools deals with database transactions since this is probably what they are already doing (one feature at a time).

              Unassigned Unassigned
              cnavarro Christopher Navarro
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Created:
                Updated:
                Resolved: