Ticket #8509 (closed: fixed)
Fix header skip in LoadAscii
Reported by: | Keith Brown | Owned by: | Keith Brown |
---|---|---|---|
Priority: | blocker | Milestone: | Release 3.1 |
Component: | Framework | Keywords: | |
Cc: | Blocked By: | ||
Blocking: | Tester: | Martyn Gigg |
Description
In #7732 I updated the LoadAscii Algorithm in order for it to read the new multi-spectra format.
However the header skipping function doesn't seem to work as well and it's causing LoadLotsOfFiles to fail on Rhel6.
The file it dislikes is DIRECTM1_15785_12m_31Oct12_v12.dat as the header contains some data-like information which seems to be confusing it.
Failed test report: https://builds.sns.gov/job/ornl_clean_rhel6_mpi_systemtested/248/testReport/SystemTests/LoadLotsOfFiles/LoadLotsOfFiles/
Change History
comment:2 Changed 7 years ago by Keith Brown
Refs #8509 Fixed skipping calculation and fixed other various errors
The calculation of how much was skipped was wrong as i was skipping over lines that were neither data or invalid wihtout taking note of them, a new conter was added - validRows - in order to keep track of how many valid lines there have been consecutively (data or not). This is then used instead of rowsToMatch when calculating numToSkip.
Corrected spelling of "Column" from "Collumn" in many places.
Fixed the bad line check to allow + and - as it had not been updated when they were classed as allowed leading characters
Refactored some of the unit test in order to allow the tests that don't use SaveAscii directly to access a correct absolute path.
Changeset: 760a6b6e0af4508c114208f596cbb420ce36eec5
comment:4 Changed 7 years ago by Keith Brown
Refs #8509 Fixing OSX complier error
OSX didn't like a std::ofstream being given a const std::string, so i've added .c_str() to all calls to getTestFileName() within an ofstream constructor
Changeset: 939ae3bb31b09ca21daa2a22221f976e49d3b6c0
comment:5 Changed 7 years ago by Keith Brown
- Status changed from inprogress to verify
- Resolution set to fixed
This should be fixed now as it's not failing any unit tests locally and it'll happily load the file it originally complained about.
To Tester:
Make sure all Tests pass and make sure it'll load DIRECTM1_15785_12m_31Oct12_v12.dat in particular.
comment:6 Changed 7 years ago by Martyn Gigg
- Status changed from verify to verifying
- Tester set to Martyn Gigg
comment:7 Changed 7 years ago by Martyn Gigg
- Status changed from verifying to closed
Merge remote-tracking branch 'origin/bugfix/8509_Fix_LoadAscii_Header_Skip'
Full changeset: 76a0da3e1213aed809153e5d906f557251712d99
Refs #8509 processheader() now checks more thoroughly
processheader() used to get called after checking the base number of collumns, which was what was cusing the error.
processheader() is now called before setting the base number of collumns and will set the base number of columns as it checks, but only if the user doesn't specifiy a number of columns to skip. processheader() will check for a number of valid lines with the same number of columns (between 2 and 4 inclusive), skipping over and commented or blank lines, and resetting the matching counter when it hits an invalid line of any sort.
Changeset: 72de8df305d1fe8984256f1033e9e1dafad61dc1