Windows build does not work without precompiled headers enabled

[Alex Buts]

If one tries to build Mantid under windows without precompiled headers, the build fails as number of includes (e.g. std::string) is missed

[Martyn Gigg]

Agreed that we should fix this. We now have a windows debug build job that would be a good candidate to keep checking that this keeps working. Ill update configuration at ​http://builds.mantidproject.org/view/Develop%20Builds/job/develop_clean_win7_debug/

[Alex Buts]

refs #10059 This seems fix it.

Changeset: 502492018e45a5ddbe173f7731de1b7cfcd6a24c

[Alex Buts]

refs #10059 removed unnecessary include, which is not in the header

Changeset: d573b1170a672930381b8243db7c6648f9aa3368

[Alex Buts]

The fix appear to be trivial as only Timer had a string which should be in .h instead of .cpp file

Unfortunately, I've started it from the #9961 (merged into master) so better make this ticket dependent on #9961

An industrious tester should try to build Win without precompiled headers and see it failing. Then the ticket would fix the problem.

I am building Mantid on Win using rather elaborate script provided below. Invoking it should be actually easy

just run

>>br.py build release 

from the repository root.

The script have dependencies on Mantid&Paraview and VC locations (and my VC location is non-standard), which can be easy modified from the script header and I have no idea what would happens if some of these are wrong. (may be it would run fine, except VC)

Otherwise you can build Mantid without headers playing with cmake or just rely on you judgement believing the changes are ok.

Test script:

"""
Script to simplify building Mantid on Windows and testing tickets

Intended to run antomatically and build number of Mantid instances on Windows without user interaction.
Currently works as cron script which executes the tasks, described in job_description.xml file.
Automatically pulls remote branches, merges testing tickets to temporary branches and makes Mantid instances for project analysis.

Logs results into log file. 

"""
#!/usr/bin/python
import os
import sys
import subprocess
import shutil
import datetime
import inspect
import copy
import numpy as np
from xml.dom import minidom


class br():
    def __init__(self):
        # the program which performs build (make) 
        self._make=['c:/Windows/Microsoft.NET/Framework64/v4.0.30319/msbuild.exe'];
        # the parameters for this program
        self._make_par=['/nologo','/m:12','/nr:false']
        # the parameters for making configuration
        self._cmake   = ['cmake']
        # cmake parameters:
        self._cmake_par = ['-G','Visual Studio 11 Win64','-DCONSOLE=ON','-DMAKE_VATES=ON','-DENABLE_CPACK=ON','-DQT_ASSISTANT_FETCH_IMAGES=OFF','-DUSE_PRECOMPILED_HEADERS=OFF','-DParaView_DIR=c:/programming/Paraview_3_98Dev/bin']

        # default Mantid git repository location:
        self._MANTID_Loc='c:/Mantid/'
        # the file with user properties, located in mantid repository root and used as basic generic properties file (not to set 
        # commont searh/data directories, paraview path etc. for each build)
        self._prop_file ='Mantid.user.properties'
        #self._MANTID_Loc='d:/Data/Mantid_GIT_dev/'
        #self._MANTID_Loc='d:/Data/Mantid_GIT/'

        # build location with respect to the Mantid Git repository location
        self._MANT_Build_relLoc='Code/builds/'
        # Mantid path template specifying all additional references to libraries to build Mantid
        self._MANTID_Path_base='c:/programming/Paraview_3_98Dev/bin/Release;{MANTID}Code/Third_Party/lib/win64;{MANTID}/Code/Third_Party/lib/win64/Python27;{PATH}'
        # set PATH=C:\Builds\ParaView-3.98.1-source\build\bin\Release;%WORKSPACE%\Code\Third_Party\lib\win64;%WORKSPACE%\Code\Third_Party\lib\win64\Python27;%PATH%
        # Mantid projects necessary for short build (minimal projects to start Mantid):
        self._MANTID_short={'Framework':'Framework.vcxproj','MantidPlot':'MantidPlot.vcxproj','MantidQT/Python':'mantidqtpython.vcxproj'}

        # cmd: the command processor for the environment to build the project and the batch files to set up necessary environment
        self._cmd = 'cmd.exe /s /c "{0} && echo "{1}" && set"'
        # the bat file used to set up environment for visual studio and C++ or nothing if the environment to build has been set up globally on the current machine
        self._set_up_env = 'c:/programming/VS2012/VC/vcvarsall.bat';

        # default file name for the log file
        path = os.getcwd();
        self._log_fname = os.path.join(path,'br_job_log_file.log');
  
    @property
    def log_fname(self):
        return self._log_fname
    @log_fname.setter
    def log_fname(self,new_fname):

        if os.path.isabs(new_fname):
            self._log_fname = new_fname;
        else:
            path = os.getcwd();
            self._log_fname = os.path.join(path,new_fname);
    @staticmethod
    def buildProjName(proj_name):
        """
        Build name of Mantid sub-project from short (sub) project name.

        The project name is the name of visual studio project runnable by nmake/cmake 
        """

        names = proj_name.split('/');
        name = "".join(names);
        return name+'.vcxproj'

    def find_target_build_path(self,branch_id,merge_id,repo_root,build_on_base=False):
        """
        Build target build path from defaults or add branch ID to it
        """
        if not(repo_root[len(repo_root)-1] == '/' or repo_root[len(repo_root)-1] == '\\' ):
            repo_root +='/'
        if build_on_base :
            build_path = repo_root+self._MANT_Build_relLoc+'br_master';
        else:
            if merge_id == branch_id:
                build_path = repo_root+self._MANT_Build_relLoc+'br_'+str(branch_id);
            else:
                build_path = repo_root+self._MANT_Build_relLoc+'br_'+str(merge_id)+'_'+str(branch_id);

        return build_path;


    def run_build(self,target,buildType=None,env=None):
        """
        method runs build for the project and if the short build fails, tries to rebuild the project/
        runs nmake
        Assumes that the branch to build is current and cmake has already build the project directory and the project file 
    
        @param target     -- the name of the target project (sln file, build by cmake)
        @param build_type -- type of the project build, which the target project understands (Debug, Release, ReleaseWithDebugInfo). If None, uses Release
        @param env        -- the enviromental variables necessary to run msbuild/mvc -- if none, the OS has to be set up to find MSBuild & mvc from command line


        """

        if buildType is None:
            buildType  = 'Release'

        # form the build command combining make progam and its parameters
        normalBuild=self._make+self._make_par+['/p:Configuration='+buildType,target]

        # run build command
        err=subprocess.call(normalBuild,env=env)
        if err != 0:
            # form the rebuild command combining make progam and its parameters
            Rebuild = self._make+self._make_par+['/p:Configuration='+buildType,'/t:Rebuild',target]
            err=subprocess.call(Rebuild,env=env)

        return err


    def build_single_proj(self,*args):
        """ build single project from command line:

        >>br build [*Release|Debug|DWRI|All,Main] [*Full|Fast] [*Old|Clean]

        assumes that the branch to build is checked out 
        where DWRI means DebugWithReleaseInfo, All -- all three builds and Main == [Debug & Release]
        Full/Fast -- build full project or minimal projects sufficient to build Mantid
        Old/Clean -- try to build the project over existing branch or clean project first.

        """
        accepted_args ={"Type":"*Release|Debug|RDI|All|Main","Kind":"*Full|Fast","Freshness":"*Old|Clean",'RepoPath':"$Path"};
       
        provided=self.parse_args(accepted_args,*args);
        for key,val in provided.iteritems():
            print " argument provided : ",key,' val=',val;

        env =  self.get_environment_from_batch_command(self._set_up_env)

        cur_path = os.getcwd()+'/';  
        if len(provided['RepoPath']) > 0:
            repo_path = provided['RepoPath'];
            os.chdir(repo_path);


        branchID = self.find_cur_branch_id();
        if len(branchID)==0:
            repo_path =  self._MANTID_Loc;
            os.chdir(repo_path)           
            branchID = self.find_cur_branch_id();
            if len(branchID)==0:
                raise EnvironmentError("Can not swich to default GIT repository location")

        else:
            repo_path = os.getcwd()+'/';

        # where to build the project
        build_path= self.find_target_build_path(branchID,branchID,repo_path,False);
        short = False;
        if provided['Kind'][0]=='fast':
            short  = True;
        build_clean = False
        if provided['Freshness'][0]=='clean':
            build_clean  = True;

        # Interpret key-words
        build_types = provided['Type'];
        if len(build_types)== 1:
            if build_types[0]=='All':
                build_types = ['Release','Debug','RelWithDebInfo'];
            if build_types[0]=='rdi' :
                build_types = ['RelWithDebInfo']
            if build_types[0] == 'Main':
                build_types = ['Debug','Release'];

        # decide if it is necessary to remove a single (sub) project (Debug, Release etc) or it is better to to wipe out the whole target build branch
        if build_clean :
            if len(build_types) > 1:
                build_single_clean = False;
                build_clean        = True;
            else:
                build_clean        = False;
                build_single_clean = True;
        else:
            build_single_clean = False;

        # Run builds
        for build_type in build_types:
            if build_type=='rdi':
                build_type = 'RelWithDebInfo';
            self.build_project(env,repo_path,build_path,True,short,build_clean,build_type,build_single_clean)
            build_clean = False;

        os.chdir(cur_path);



    def build_project(self,env,repo_path,build_path,first_build=True,short=False,build_all_clean=False,buildType=None,build_single_clean=False):
        """   Build current project using cmake and msbuild

        env   -- environmental variables necessary to build the project (run C++ and tools)
        build_path -- the folder to build the project 
        short -- if minimal rebuild, namely minimal number of projects to start Mantid gui is needed 
        build_clean -- delete the target project build directory if its already exist
        buildType   -- The type of the build, make understands (Debug, Release etc...)

        assumes that the brunch to build is current branch
        """


        # modify path to understand Mantid project and Mantid libraries. 
        loc_env = dict()
        loc_env['PATH'] = env['Path'];
        loc_env['MANTID']= repo_path;
        env['Path']=self._MANTID_Path_base.format(**loc_env);  
        # the path to the particular build flavour (Debug|Release|DebugWithReleaseInfo etc.)
        build_flavour_path = os.path.join(build_path,'bin',buildType)

        current_dir  = os.getcwd();
        build_exists = os.path.exists(build_path);

        if build_all_clean and build_exists:
            print "Clean build: removing existing project from: {0}".format(build_path);
            shutil.rmtree(build_path,True)
            print "           : finished removing existing project ---------------------";
            build_exists=os.path.exists(build_path)
        if build_single_clean and build_exists:
            build_flavour_exists = os.path.exists(build_flavour_path);
            if build_flavour_exists:
                print "Clean build: removing existing project flavour from: {0}".format(build_flavour_path);
                shutil.rmtree(build_flavour_path,True)
                print "           : finished removing existing project flavour ------------";

        if not build_exists:
            os.mkdir(build_path);

        os.chdir(build_path )

        if first_build:
            # form cmake command:
            # code path:
            code_path  = repo_path+'Code/Mantid'
            cmake = self._cmake+self._cmake_par+[code_path]
            # run cmake
            err=subprocess.call(cmake,env=env)
            if err != 0:
                os.chdir(current_dir);
                raise RuntimeError("Can not execute cmake")

        # run msbuild
        if short :
            # process minimal set of projects, necessary to start and run Mantid
            for dir,proj_name in self._MANTID_short.iteritems():
                os.chdir(build_path+'/'+dir)
                err=self.run_build(proj_name,buildType,env);
                if err>0:
                    errMessage="Can not execute msbuild for :"+proj+'.vcxproj';
                    break
        else:
            errMessage="Can not execute msbuild for : Mantid.sln"
            err=self.run_build('Mantid.sln',buildType,env);


        # return back to the source directory
        os.chdir(current_dir);
        if err != 0:
            raise RuntimeError(errMessage)
        else: # copy Mantid.parameters to target build directory if prop file exist and target properties file does not
            prop_file = os.path.join(repo_path,self._prop_file);
            if os.path.exists(prop_file):
                targ_file = os.path.join(build_flavour_path,self._prop_file)
                if not os.path.exists(targ_file):
                    shutil.copyfile(prop_file,targ_file);


    def get_environment_from_batch_command(self,env_cmd):
        """
        Take a command (either a single command or list of arguments)
        and return the environment created after running that command.
        Note that if the command must be a batch file or .cmd file, or the
        changes to the environment will not be captured.
  
        """
        if not isinstance(env_cmd, (list, tuple)):
            env_cmd = [env_cmd]
        # construct the command that will alter the environment
        env_cmd = subprocess.list2cmdline(env_cmd)
        # create a tag so we can tell in the output when the proc is done
        tag = 'Done running command'
        # construct a cmd.exe command to do accomplish this
        cmd = self._cmd.format(env_cmd,tag)

        # launch the process
        proc = subprocess.Popen(cmd, stdout=subprocess.PIPE)
        # parse the output sent to stdout
        reject = True
        result={};
    # define a way to handle each KEY=VALUE line
        handle_line = lambda l: l.rstrip().split('=',1)
        for line in proc.stdout:
            # consume whatever output occurs until the tag is reached
            if not reject:
                kv = handle_line(line);
                result[kv[0]]=kv[1];
                continue
            if tag in line:           
                reject = False

        # let the process finish
        proc.communicate()
        return result
    def build_target_br_name(self,br_ID_to_merge_to,source_br_ID):
        """
        defines convention for target branch name and calculates this name from the source branches names;
        """
        rez = '__'+str(br_ID_to_merge_to);
        if br_ID_to_merge_to != source_br_ID:
           rez = rez+'_'+str(source_br_ID);

           return rez;


    def find_cur_branch_id(self):
        """
        returns current branch ID (the string between / and number including number
        or empty string if not on a branch 
        """
        err = subprocess.call(['git','status'])
        if err != 0: # not on Git
            return ""

        # get the names of the current branch of the the repository
        result = subprocess.check_output(['git','rev-parse','--abbrev-ref','HEAD']) # 
        result=result.rstrip();
        rez = result.split('/');
        if len(rez)>1:
            result = rez[-1];
        rez = result.split('_')
        if len(rez) == 1:
            return rez[0]

        result = "";
        for parts in rez:
            try:
                int(parts)
                result = result+parts;
                break;
            except ValueError:
                result =result+part+'_';
        return result

    def find_full_banch_name(self,branch_id,exclude_service=True):
        """
        find full branch name on GIT from the branch id 
        branch id is a piece of string or number, which identifies the branch.
        (e.g. the ticket number)

        if exclude_service is True, ignores branches which start with __
        
        assumes that the current folder is a git folder
        """

        # get the names of all branches in the repository
        result = subprocess.check_output(['git','branch','-a']) # 

        branch_id = str(branch_id)
        branches = result.split('\n')
        local_branch_name="";
        remote_branch_name="";
        for branch_name in branches:
            if '*' in branch_name:
                current_branch = branch_name.strip('* ')
                branch_name    = current_branch


            branch_name = branch_name.strip();

            # skip service branches which can be target only and never the source
            if branch_name[:2] == '__' and exclude_service:
                continue

            if branch_id in branch_name:
                if 'remote' in branch_name:
                    full_name = branch_name.split('origin/');
                    remote_branch_name = full_name[1]
                else:
                    local_branch_name = branch_name


        if len(local_branch_name) == 0 and len(remote_branch_name) == 0: 
            return "","ERR: can not find the branch with ID {0} ".format(branch_id),current_branch


        return local_branch_name,remote_branch_name,current_branch;

    @staticmethod
    def check_branch_synchronized(branch_name, other_branch_name = None):
        # check if local branch indeed synchronized with the remote branch
        if other_branch_name is None:
            other_branch_name = 'origin/'+branch_name
        try:
            result = subprocess.check_output(['git','diff',branch_name,other_branch_name]) # 
        except:
            return False

        if len(result) == 0:
            return True
        else:
            return False

    @staticmethod
    def check_and_remove_existing(hold_on_branch):
        """
        """
       # get the names of local branches in the repository
        err=""
        result = subprocess.check_output(['git','branch']) # 
        if hold_on_branch in result:
            err = subprocess.call(['git','branch','-D',hold_on_branch]) #
            if err!=0: # may be we are on this branch?
                err = subprocess.call(['git','checkout','master']) #
                err = subprocess.call(['git','branch','-D',hold_on_branch]) #
        return err

    @staticmethod
    def convert_boolean(attribute,field_name):
        """
          get boolean attribute from field name and convert its symbolic contents to boolean
        """
        field_contents=attribute.getAttribute(field_name)
        if len(field_contents) == 0 or field_contents != 'True':
                field_contents =False
        else:
                field_contents =True

        return field_contents

        
    def get_job_attributes(self,job):
        """
        processes xml string to obtain job attributes from it
        """
        # The ID of the branch (ticket number )
        branch_ID = job.getAttribute("branch_ID");

        # 
        repo_root  = job.getAttribute("mantid_root")
        if len(repo_root) == 0:
                repo_root = self._MANTID_Loc
        if not(repo_root[len(repo_root)-1] == '/' or repo_root[len(repo_root)-1] == '\\' ):
            repo_root +='/'


        merge_to = job.getAttribute("merge_to")
        if len(merge_to) == 0:
                merge_to = branch_ID

        build_on_base = self.convert_boolean(job,"build_on_base")
        skip_this     = self.convert_boolean(job,"skip_this")
        clean_build   = self.convert_boolean(job,"clean_build")


        return branch_ID,repo_root,merge_to,build_on_base,skip_this,clean_build

    def prepare_working_branch(self,archive_path,merge_to_ID,branch_id,dry_run=False):
        """
        method checks if the branch identified by ID is present, 
        merges it with specified branch ID or checks existing branch out and 
        returns the GIT name of the branch to build.
        The name is constructed from the branch id(s) if the branch do not exist. 

        returns tupe (err,branch_name). If everything is ok err is empty string
        
        current branch in GIT repository becomes the branch with name branch_name
        """

        current_dir=os.getcwd();
        os.chdir(archive_path);

        # update repository info
        err = subprocess.call(['git','fetch','-p'])
        if err != 0 :
           return "ERR: can not fetch current repository status";


        wk_local_branch,wk_remote_branch,current_branch = self.find_full_banch_name(branch_id);
        if 'ERR:' in wk_local_branch:
            return "TargetBranch: {0} in archive: {1}".format(wk_remote_branch,archive_path),"";
        # we want to build current branch which does not have remote. just stay with current local branch
        if wk_local_branch == current_branch and len(wk_remote_branch)==0:
            return "",current_branch

        # only one branch has been given and it is the branch we need to build
        if branch_id == merge_to_ID:
            merge_remote_branch = wk_remote_branch
            merge_local_branch  = wk_local_branch
            err = subprocess.call(['git','checkout',merge_local_branch])
            if err != 0 :
                  return "ERR: checkout target branch: "+merge_local_branch,"";

            err = subprocess.call(['git','pull'])
            if err != 0 :
                  return "ERR: pull target branch: "+merge_local_branch,"";

            return "",merge_local_branch;
        else:
            merge_local_branch,merge_remote_branch,current_branch = self.find_full_banch_name(merge_to_ID);
            if 'ERR:' in merge_remote_branch :
                return "Base branch: {0} in archive: {1}".format(merge_remote_branch,archive_path),"";
                
        #-------------------------------------------------------------------------------------------

        # the branch which will be created from two source branches (e.g for testing). 
        # It should not be real branch present on Git or branch which contain any reasonable 
        # information on it
        holdon_br_name = self.build_target_br_name(merge_to_ID,branch_id);
        if holdon_br_name == current_branch: # merge both branches into it
            err = self.merge_2current(wk_local_branch,wk_remote_branch);
            if len(err)==0:
                  err = self.merge_2current(merge_local_branch,merge_remote_branch);
        else:                              # create this branch, delete it if it existed
            holdon_local_branch,holdon_remote_branch,current_branch = self.find_full_banch_name(holdon_br_name,False);
            if len(holdon_local_branch) == 0 : # no such branch, has to be created
                err=self.merge_one2another_and_create_new(wk_local_branch,wk_remote_branch,merge_local_branch,merge_remote_branch,holdon_br_name);
            else :                             # branch found, 
              if len(holdon_remote_branch)>0: # has remote counterpart;
                return "Remote Hold-on branch {0} in archive {1} exists. Do not know what to do about it".format(holdon_br_name,archive_path),"";
              else: # local hold-on branch exist and it is not the current branch
                    # delete existing local hold-on branch according to the git workflow
                    err = subprocess.call(['git','branch','-D',holdon_local_branch])
                    if err != 0 :
                            return "ERR: deleting base branch: {0}".format(holdon_local_branch),"";
    
                    err=self.merge_one2another_and_create_new(wk_local_branch,wk_remote_branch,merge_local_branch,merge_remote_branch,holdon_br_name);

        os.chdir(current_dir);
        return err,holdon_br_name

    @staticmethod  
    def merge_2current(br1_local,br1_remote):
        """
        merge branches, specified as the arguments into current GIT branch.
        """
        if len(br1_local)>0:
            err= subprocess.call(['git','merge',br1_local])
            if err != 0 :
                err= subprocess.call(['git','merge','--abort'])
                return "ERR: merging "+br1_local+" to current branch"
            else:
                err ="";
        else:
           if len(br1_remote)>0:
                err= subprocess.call(['git','merge','origin/'+br1_remote])
                if err != 0 :
                    err= subprocess.call(['git','merge','--abort'])
                    return "ERR: merging "+br1_remote+" to current branch"
                else:
                    err="";
           else:
                err="ERR: both local and remote branches are empty"
        #
        if br1_local != br1_remote :
                err= subprocess.call(['git','merge','origin/'+br1_remote])
                if err != 0 :
                    err= subprocess.call(['git','merge','--abort'])
                    return "ERR: merging "+br1_remote+" to current branch"
                else:
                    err="";


        return err;

    def merge_one2another_and_create_new(self,wk_local_branch,wk_remote_branch,merge_local_branch,merge_remote_branch,holdon_br_name):
        """
        """
        if len(wk_local_branch) > 0 and len(wk_remote_branch)>0: # local branch exist
            err = subprocess.call(['git','checkout',wk_local_branch])
            if err != 0 :
               return "ERR: checkout working branch: "+wk_local_branch;
            err = subprocess.call(['git','pull',wk_local_branch])
            if err != 0 :
               return "ERR: pulling working branch: "+wk_local_branch;
        #else:  # local working branch does not exist, will work with remote


        err = subprocess.call(['git','checkout',merge_local_branch])
        if err != 0 :
            return "ERR: checkout merge branch: "+merge_local_branch;
        if len(merge_remote_branch)>0:
            err = subprocess.call(['git','pull'])
            if err != 0 :
               return "ERR: pulling merge branch: "+merge_remote_branch;

        # create new branch from the merge branch to work with
        err = subprocess.call(['git','checkout','-b', holdon_br_name])
        if err != 0 :
            return "ERR: creating new branch  "+holdon_br_name; 

        # merge with target local branch
        err = self.merge_2current(wk_local_branch,wk_remote_branch);
        if len(err)>0:
            return err+' to '+merge_local_branch;

        return "";


    def run_batch_job(self,*argi):
        """ runs batch job described by the job description file provided as argument

        Usage: 
        >>br batch [job_description_file]
        >>br test [job_description_file]

        job_description_file is xml file with the information understood by cron_job. It can be full file path or file name 
        for file available in the search path

        if no job description file provided, default job description file job_description.xml is used.       
        
        >>br test [job_description_file] 
        or 
        >>br batch test [job_description_file] 
        
        option provided, the script does not build the projects described in the xml file but just mergres/pulls git and creates appropriate branches

        This option is used to test if merges/working with git goes smoothly for all jobs described in the file before running 
        the builds themselves (which usually takes long time to complete)
        The results of this test are placed into "job_description_file".log file
        """
        default_job_descr = 'job_description.xml'
        accepted_args ={"Type":"*batch|test","File":"$File"};

        provided=self.parse_args(accepted_args,*argi);

        batch_par = provided['File'];
        if len(batch_par) < 1:
            job_descr = default_job_descr 
        else:
            job_descr= batch_par;

        if not os.path.exists(job_descr):
            raise RuntimeError("Can not find job description file: {0}".format(job_descr))

        if provided['Type'][0]=='batch':
            self.cron_job(job_descr); 
        else:
            self.cron_job(job_descr,True); # Dry run



    def append_log(self,string_to_add):
        """
        """
        #f=open(self.log_fname,'a')
        #f.write(string_to_add);
        #f.close();
        with open(self.log_fname,'a') as f:
            f.write(string_to_add);


    def cron_job(self,job_description_file,dry_run=False):
        """ method processes job description file and runs all Mantid build job described there
        """

        # found job description file and prepare log file with the same name

        if os.path.isabs(job_description_file):
            path,file = os.path.split(job_description_file);
        else:
            path = os.getcwd();
            file = job_description_file;

        fname,fext = os.path.splitext(file);
        self._log_fname = os.path.join(path,fname + '.log');

        log_head=datetime.datetime.now().strftime(":%B,%d,%Y::%I:%M%p::")
        mess = ("----------------------------------------------------------------------\n"
                "{0} Starting cron jobs for parameters: {1}\n").format(log_head,job_description_file)
        self.append_log(mess);


        # parse job description file
        try:
            domObj=minidom.parse(job_description_file)
        except Exception as error:
            err_type = type(error)
            mess = "{0} ERR: problem with job description file: {1} in working folder {2} \n err type: {3} message: {4}\n".format(log_head,job_description_file,os.getcwd(),err_type,error.message);
            self.append_log(mess);
            return
        jobs = domObj.getElementsByTagName("job")

        env = {}
        if len(jobs) == 0:
            log_head='+'.ljust(len(log_head))
            self.append_log("{0}, No jobs found".format(log_head))
            return
        else:
            env =  self.get_environment_from_batch_command(self._set_up_env)

        # go through all jobs in job description and run them with the parameters, obtained from the job description
        for job in jobs :
            # retrieve job attribute or its default values if the attributes are missing
            branch_ID,repo_root,merge_to,build_on_base,skip_this,clean_build =self.get_job_attributes(job)
            if skip_this:
                 log_head=datetime.datetime.now().strftime(":%B,%d,%Y::%I:%M%p::")
                 self.append_log("{0} Skipping job with ID {1} \n".format(log_head,branch_ID))
                 continue


            Target_path = self.find_target_build_path(branch_ID,merge_to,repo_root,build_on_base)

            # prepare branches to build e.g pull/create/merge it from GIT and change code to it. Return its name for logging purposes
            err,local_branch_toBuild = self.prepare_working_branch(repo_root,merge_to,branch_ID);
            if len(err) != 0:
                    log_head='+'.ljust(len(log_head))
                    self.append_log("{0}, Can not prepare source branch: {1}, scipping this job\n".format(log_head,err))
                    continue


            # Go through all kind of builds selected for the branch
            builds = job.getElementsByTagName('build')
            # set first build to True to allow cmake to run
            fast_build = False
            first_build  = True
            key_log = "Skipping"
            if dry_run :
                key_log = "Dry Run "
            for buildType in builds:
                build_type  = buildType.getAttribute('type')
                fast_build  = self.convert_boolean(buildType,"minmal_build")
                skip_this   = self.convert_boolean(buildType,"skip_this")
                log_head=datetime.datetime.now().strftime(":%B,%d,%Y::%I:%M%p::")
                if skip_this or dry_run:
                    self.append_log("{0} {4} build {3} run for Mantid, branch {1} located in: {2} \n".format(log_head,local_branch_toBuild,repo_root,build_type,key_log))
                    continue

                self.append_log("{0} Starting {3} run for Mantid, branch {1} located in: {2} \n".format(log_head,local_branch_toBuild,repo_root,build_type))

                # Try to build actual project
                try:
                     log_head='+'.ljust(len(log_head))
                     self.append_log("{0} Build parameters: First build: {1}; Fast Build: {2}, Clean Build: {3}\n".format(log_head,str(first_build),str(fast_build),str(clean_build)))
                     self.build_project(env,repo_root,Target_path,first_build,fast_build,clean_build,build_type)
                     # second build do not need cmake to run
                     first_build  = False
                     log_head=datetime.datetime.now().strftime(":%B,%d,%Y::%I:%M%p::")
                     self.append_log("{0} Finished {2} run for Mantid, branch {1} \n".format(log_head,branch_ID,build_type))
                except RuntimeError, err:
                     #log_head=datetime.datetime.now().strftime(":%B,%d,%Y::%I:%M%p::")
                     log_head='+'.ljust(len(log_head))
                     self.append_log("{0} ERR: {1} while building target project {2}\n".format(log_head,str(err),build_type))
                     

                # second build type should not be clean as it will wipe out the first build regardless of the first run was successful or not
                clean_build = False


        log_head=datetime.datetime.now().strftime(":%B,%d,%Y::%I:%M%p::")
        self.append_log(("{0} finished Mantid cron job {1}\n"+
                         "--------------------------------\n").format(log_head,job_description_file))     
    @staticmethod
    def parse_args(accept_args,*args):
        """
        Generic parser for arbitrary input arguments selected from the range provided :
        """
        def process_argi(defaultVal,sample,foundAt,possibilities,*argi):

           #if len(argi) == 1:        
           #     if type(argi) == type(()):
           #         argi = argi[0]

           if len(argi) == 0:
               result = defaultVal;
               if len(possibilities)>1:
                    return [result]
               else:
                    return result
        
           result = [];
           for ic in range(0,len(argi)): 
               if foundAt[ic] :
                  continue
               argument = argi[ic].lower();
               if len(possibilities) == 1 :
                   if possibilities[0][0] == '$':
                       # it is a file name. Mind the case
                       result = argi[ic];
                       foundAt[ic] = True;
                       return result;
               if type(argument)==str and argument in sample:

                   for case in possibilities:
                       if argument in case:
                           result.append(case.strip('*'));
                           foundAt[ic] = True;
                
           if len(result) == 0:
               result = defaultVal;
               if len(possibilities)>1:
                   result = [defaultVal];
           return result;

        # init;
        rez_keys = accept_args.keys();
        result ={key : [] for key in rez_keys};
    
        found_arg = np.zeros(len(args),dtype=bool)
        # loop over all possible arguments
        for key,pos_values in accept_args.iteritems():
            pos_values    = pos_values.lower();
            possibilities = pos_values.split('|');
            default = "";
            for poss in possibilities:
                if '*' in poss:  # found default value
                    default = poss.strip('*');
                      
                    break
            result[key]= process_argi(default,pos_values,found_arg,possibilities,*args);

        if len(args) > 0 and not(np.all(found_arg)):
            for i in xrange(len(args)):
                if not found_arg[i]:
                    print 'Invalid agument: ',args[i]
            raise KeyError("Unknown input argument")

        return result;

    def help(self,options=None,**kwarg ):
        """ Prints help for the module
        """
        print "Script to build Mantid branches "
        print " Usage: \n>>br option [sub-options] "

        if options is None:
            print " Type: \n>>br help for list of options "
            return;

        print " Where known options are: "
        for key in options:
            print key,"\t :\t",
            print options[key].__doc__
        print "Type: \n>>br help option for more information about this option (not yet implemented)"
        #print "br debug|release|DebWithRelInfo [fast]  -- build debug/release/debug with release version of the build"
        #print "Where ""fast"" if present means minumal rebuild. The build occurs for the branch and repository you are currently in."
        #print "If working directory is not the Git repository, br changes to default git repository, defined in the script"

def test_parse_arg():
    builder = br();
    accepted_args ={"Type":"*Release|Debug|DWRI|All|Main","Kind":"*Full|Fast","Freshness":"*Old|Clean"};

    def myAssertEq(x,y):
         if(x != y) :
             print 'Assert fail: x={0}, y={1} '.format(x,y)

    #arg=builder.parse_args(accepted_args,[]);
    #myAssertEq(arg['Type'][0],'Release')
    #myAssertEq(arg['Kind'][0],'Full');
    #myAssertEq(arg['Freshness'][0],'Old')   
    
    argis = [];
    arg=builder.parse_args(accepted_args,*argis);
    myAssertEq(arg['Type'][0],'Release')
    myAssertEq(arg['Kind'][0],'Full');
    myAssertEq(arg['Freshness'][0],'Old')   

    argis = ['Deb','Cle'];
    arg=builder.parse_args(accepted_args,*argis);
    myAssertEq(arg['Type'][0],'Debug')
    myAssertEq(arg['Kind'][0],'Full');
    myAssertEq(arg['Freshness'][0],'Clean')   

    argis = ['Deb','DWRI','Cle'];
    arg=builder.parse_args(accepted_args,*argis);
    myAssertEq(arg['Type'][0],'Debug')
    myAssertEq(arg['Type'][1],'DWRI')
    myAssertEq(arg['Kind'][0],'Full');
    myAssertEq(arg['Freshness'][0],'Clean')   

    accepted_args ={"Type":"*batch|test","file":"$File"};
    argis =['test','SomeFile'];
    parsed=builder.parse_args(accepted_args,*argis);
    myAssertEq(parsed['Type'][0],'test')
    myAssertEq(parsed['file'],'SomeFile')

    argis=['OtherFile']
    parsed=builder.parse_args(accepted_args,*argis);
    myAssertEq(parsed['file'],'OtherFile')
    myAssertEq(parsed['Type'][0],'batch')

    argis=['OtherFile','test']
    parsed=builder.parse_args(accepted_args,*argis);
    myAssertEq(parsed['file'],'OtherFile')
    myAssertEq(parsed['Type'][0],'test')

    argis=['OtherFile','rubbish1','rubbish2']
    parsed=builder.parse_args(accepted_args,*argis);
    myAssertEq(parsed['file'],'OtherFile')
    myAssertEq(parsed['Type'][0],'batch')

    exit(0);

if __name__ == '__main__':
    builder = br();
    nargi = len(sys.argv);

    #test_parse_arg();
    if nargi <2:
        builder.help()
        sys.exit(-1);


    #builder.parse_args();
    params = sys.argv[2:nargi];
    known_options={};
    known_options['build']=lambda : builder.build_single_proj(*params)
    known_options['batch']=lambda : builder.run_batch_job(*params)
    test_arg = ['test',params[0]];
    known_options['test'] =lambda : builder.run_batch_job(*test_arg)
    known_options['help'] =lambda : builder.help(known_options)
     
    known_options['build'].__doc__  = (inspect.getdoc(builder.build_single_proj)).split('\n',1)[0];
    known_options['batch'].__doc__ = (inspect.getdoc(builder.run_batch_job)).split('\n',1)[0];
    known_options['test'].__doc__ = "Test batch job described by xml file provided. Equivalent to:  >>br batch test $filename"
    known_options['help'].__doc__  = (inspect.getdoc(builder.help)).split('\n',1)[0];


    
    option = sys.argv[1].lower();
    # for testing this repository
    #os.chdir(r'd:\Data\Mantid_GIT_test')
    #os.chdir('C:/mantid');
    
    if option in known_options:
         known_options[option]();
    else:
        print "br Unknown parameter: ",option
        builder.help(**known_options)

    #builder.cron_job(job_name)
    
#   run_key = args[1](0:2)
#    if 
# Testing
#    current_dir=os.getcwd();
#
#    bl,br,cb,bbp=builder.find_full_banch_name("4060")
##    is_sync=builder.check_branch_synchronized(bn);
#    os.chdir(current_dir);

#builder.check_merges('job_description.xml')
#tp = builder.prepare_working_parh_and_branch(builder._MANTID_Loc,'master',7492);

#env = builder.get_environment_from_batch_command(builder._set_up_env)
 #builder.build_project(env,builder._MANT_Build_relLoc,True,True);

[Martyn Gigg]

The Windows clean debug build, ​http://builds.mantidproject.org/view/Develop%20Builds/job/develop_clean_win7_debug/, has precompiled headers turned off and that seems fine.

[Martyn Gigg]

Merge remote-tracking branch 'origin/bugfix/10059_WinPrecompiledHeaders'

Full changeset: ​d154648ff0cddd40f7a0b84db8a602ee56f72962

[Stuart Campbell]

This ticket has been transferred to github issue ​10901