Large Mcstas file very slow with Load dialog

[Martyn Gigg]

While testing #9137 it became apparent that it took too long a time to populate the Load dialog (about 20 seconds). A quick look at the stack trace in debug seems to indicate it is spending a long time walking the tree hierarchy with NeXus.

This needs to be investigated and fixed. The test file is noted in the description on #9137

[Anders Markvardsen]

Works on Windows, i.e. loading mccode_very_large.h5 brings up load dialog instantly.

I have put mccode_very_large.h5 on
Olympic\Babylon5\Scratch\Anders.

[Martyn Gigg]

I still see this problem on Linux.

[Anders Markvardsen]

Just tried again on Windows, and I was not looking at the right place. The same effect on Windows is seen. After selecting file it looks a while before load dialog windows readjust itself with options specific for this loader

[Martyn Gigg]

I just did a quick check with a debug build by breaking while the dialog is loading. It seems to be spending all of its time in Mantid::Kernel::NexusDescriptor::walkFile. For completeness the back trace is

#0  0x00007ffff0fe6add in lseek64 () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007fffe6692676 in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#2  0x00007fffe668c442 in H5FD_read () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#3  0x00007fffe667ac31 in H5F_accum_read () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#4  0x00007fffe667e4c8 in H5F_block_read () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#5  0x00007fffe662b5d9 in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#6  0x00007fffe663f18e in H5C_protect () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#7  0x00007fffe662547d in H5AC_protect () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#8  0x00007fffe6626fe4 in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#9  0x00007fffe662712a in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#10 0x00007fffe662712a in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#11 0x00007fffe662712a in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#12 0x00007fffe662712a in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#13 0x00007fffe6628fb5 in H5B_get_info () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#14 0x00007fffe6647c8f in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#15 0x00007fffe6651fa9 in H5D__chunk_bh_info () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#16 0x00007fffe6661dbd in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#17 0x00007fffe66f553c in H5O_get_info () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#18 0x00007fffe66ae713 in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#19 0x00007fffe66bbf15 in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#20 0x00007fffe66bc7c6 in H5G_traverse () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#21 0x00007fffe66af83c in H5G_loc_info () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#22 0x00007fffe66f1fe8 in H5Oget_info_by_name () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#23 0x00007fffedeb6618 in ?? () from /usr/lib/libNeXus.so.0
#24 0x00007fffe66ac62b in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#25 0x00007fffe66b2385 in H5G__node_iterate () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#26 0x00007fffe66280ae in ?? () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#27 0x00007fffe66288aa in H5B_iterate () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#28 0x00007fffe66b7fdf in H5G__stab_iterate () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#29 0x00007fffe66b48dd in H5G__obj_iterate () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#30 0x00007fffe66ad72c in H5G_iterate () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#31 0x00007fffe66e658e in H5Literate () from /usr/lib/x86_64-linux-gnu/libhdf5.so.7
#32 0x00007fffedeb7f57 in ?? () from /usr/lib/libNeXus.so.0
#33 0x00007ffff09eef45 in NeXus::File::getNextEntry() () from /usr/lib/libNeXusCPP.so.0
#34 0x00007ffff09f2d8a in NeXus::File::getEntries(std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > >&) () from /usr/lib/libNeXusCPP.so.0
#35 0x00007ffff09f31f7 in NeXus::File::getEntries() () from /usr/lib/libNeXusCPP.so.0
#36 0x00007ffff79b203f in Mantid::Kernel::NexusDescriptor::walkFile (this=this@entry=0x7fffffffca90, file=..., rootPath=..., 
    className=..., pmap=..., level=level@entry=3)
    at /mnt/data1/GitHub/mantid-master/Code/Mantid/Framework/Kernel/src/NexusDescriptor.cpp:240
#37 0x00007ffff79b21f3 in Mantid::Kernel::NexusDescriptor::walkFile (this=this@entry=0x7fffffffca90, file=..., rootPath=..., 
    className=..., pmap=..., level=level@entry=2)
    at /mnt/data1/GitHub/mantid-master/Code/Mantid/Framework/Kernel/src/NexusDescriptor.cpp:255
#38 0x00007ffff79b21f3 in Mantid::Kernel::NexusDescriptor::walkFile (this=this@entry=0x7fffffffca90, file=..., rootPath=..., 

[Anders Markvardsen]

From discussion with Martyn, on a more detailed investigation where the additional time spend is to create the NexusDescriptor for the confidence() method.

From looking at structure mccode_very_large.h5 this was not clear why this should be the case.

I have added mccode_100000.h5 and mccode_1million.h5 to Olympic\Babylon5\Scratch\Anders also.

Note #11141 added as part of this investigation

[Stuart Campbell]

This ticket has been transferred to github issue ​10054