Ticket #8509 (closed: fixed)

Opened 7 years ago

Last modified 5 years ago

Fix header skip in LoadAscii

Reported by: Keith Brown Owned by: Keith Brown
Priority: blocker Milestone: Release 3.1
Component: Framework Keywords:
Cc: Blocked By:
Blocking: Tester: Martyn Gigg

Description

In #7732 I updated the LoadAscii Algorithm in order for it to read the new multi-spectra format.

However the header skipping function doesn't seem to work as well and it's causing LoadLotsOfFiles to fail on Rhel6.

The file it dislikes is DIRECTM1_15785_12m_31Oct12_v12.dat as the header contains some data-like information which seems to be confusing it.

Failed test report: https://builds.sns.gov/job/ornl_clean_rhel6_mpi_systemtested/248/testReport/SystemTests/LoadLotsOfFiles/LoadLotsOfFiles/

Change History

comment:1 Changed 7 years ago by Keith Brown

  • Status changed from new to inprogress

Refs #8509 processheader() now checks more thoroughly

processheader() used to get called after checking the base number of collumns, which was what was cusing the error.

processheader() is now called before setting the base number of collumns and will set the base number of columns as it checks, but only if the user doesn't specifiy a number of columns to skip. processheader() will check for a number of valid lines with the same number of columns (between 2 and 4 inclusive), skipping over and commented or blank lines, and resetting the matching counter when it hits an invalid line of any sort.

Changeset: 72de8df305d1fe8984256f1033e9e1dafad61dc1

comment:2 Changed 7 years ago by Keith Brown

Refs #8509 Fixed skipping calculation and fixed other various errors

The calculation of how much was skipped was wrong as i was skipping over lines that were neither data or invalid wihtout taking note of them, a new conter was added - validRows - in order to keep track of how many valid lines there have been consecutively (data or not). This is then used instead of rowsToMatch when calculating numToSkip.

Corrected spelling of "Column" from "Collumn" in many places.

Fixed the bad line check to allow + and - as it had not been updated when they were classed as allowed leading characters

Refactored some of the unit test in order to allow the tests that don't use SaveAscii directly to access a correct absolute path.

Changeset: 760a6b6e0af4508c114208f596cbb420ce36eec5

comment:3 Changed 7 years ago by Keith Brown

  • Owner set to Keith Brown

comment:4 Changed 7 years ago by Keith Brown

Refs #8509 Fixing OSX complier error

OSX didn't like a std::ofstream being given a const std::string, so i've added .c_str() to all calls to getTestFileName() within an ofstream constructor

Changeset: 939ae3bb31b09ca21daa2a22221f976e49d3b6c0

comment:5 Changed 7 years ago by Keith Brown

  • Status changed from inprogress to verify
  • Resolution set to fixed

This should be fixed now as it's not failing any unit tests locally and it'll happily load the file it originally complained about.

To Tester:

Make sure all Tests pass and make sure it'll load DIRECTM1_15785_12m_31Oct12_v12.dat in particular.

comment:6 Changed 7 years ago by Martyn Gigg

  • Status changed from verify to verifying
  • Tester set to Martyn Gigg

comment:7 Changed 7 years ago by Martyn Gigg

  • Status changed from verifying to closed

Merge remote-tracking branch 'origin/bugfix/8509_Fix_LoadAscii_Header_Skip'

Full changeset: 76a0da3e1213aed809153e5d906f557251712d99

comment:8 Changed 5 years ago by Stuart Campbell

This ticket has been transferred to github issue 9353

Note: See TracTickets for help on using tickets.