LoadEventNexus: check on memory use while loading

[Janik Zikovsky]

I'm not sure where you can try this since it is a 16GB file (1 hour measurement). What I did was try to load the first 1GB (3600/15=240) using the time filtering in the loader. I still went into swap until I "Sort"ed the data when it went down to ~3GB of memory as I expected. Please check into what created 17GB+ of temporary memory during the load. The end result is correct, but something about the process is not quite right.

Also, CompressEvents doesn't get the handle little "copy input workspace name" button. I'm suspicious that either the parameter type is wrong, or the auto-generated gui needs to be informed of event workspaces. This is just a minor usability issue.

P²

[Janik Zikovsky]

Next, it would be nice if one could create a "compressed event workspace" on loading. Then I could run more tests on my desktop. This may actually take precedence over the other items since it makes development life easier.

[Janik Zikovsky]

run is NOM_553

[Janik Zikovsky]

(In [9819]) Refs #2588: Call to release memory while loading event nexus. Also, option to compress events.

[Janik Zikovsky]

With last modification off: 53 sec to load 240 sec of NOM_553. Memory increased more evenly with a slight (500 MB) reduction at the end after clearing memory.

With last modification on: 60 sec to load 240 sec of NOM_553. You could see the memory go both up and down during.

In neither case on my machine did the memory come close to 16 GB.

[Janik Zikovsky]

(In [9821]) Fixes #2588: Call to release memory only if above a certain threshold of memory used. Should save a couple of seconds vs calling it each time.

[Vickie Lynch]

NOM_553 with FilterByTimeStop=240 took 68.4 seconds and 3.55 GB memory for workspace and 4.8GB total memory with no compression. With CompressTolerance=0.01, 69.2 sec and 3.3GB for workspace and 4.7GB total.

[Stuart Campbell]

This ticket has been transferred to github issue ​3435