Add ability to run documentation tests individually

[Martyn Gigg]

Tidy up of runsphinx template. Refs #9639

Changeset: 76b2d481e00c1dd1f326ed685437ede08b1f3c85

[Martyn Gigg]

Allow single file selection for sphinx run

Refs #9639

Changeset: 2e96b850a272a7b53cfd49ef8d21dffcbf3554f0

[Martyn Gigg]

Enable sphinx to be run without MantidPlot.

Refs #9639

Changeset: 935bda129dddd5bec7975daba9ae3d2f1c7645be

[Martyn Gigg]

Separate the Sphinx doctree directories by bulder type.

Each builder type now has a separate environment so that the order of running the targets doesn't make a difference. Shared doctree directories meant that running doctest first just in vanilla python wouldn't generate the screenshots and running the html target wouldn't reread the unchanged source files to generate the images. It forced doctest to run in MantidPlot and do the screenshot generation.

Refs #9639

Changeset: b116b485470b7124c60e9b24f6aa5030b0a9952e

[Martyn Gigg]

Run docs-test target with vanilla python.

Refs #9639

Changeset: 4114116e9dac0da5d892657f69afea7ff98f1fac

[Martyn Gigg]

Add individual rst files as ctest targets.

They can be disabled by turning off the SEPARATE_DOC_TESTS cmake option. Refs #9639

Changeset: 044850a6f1219a8094bf962d3b1e0dae1e9e5bf0

[Martyn Gigg]

Turn off the separate documentation tests on the build servers

They will instead just run the single global target through MantidPlot that will be much faster than running them indivually. Refs #9639

Changeset: a5544f6b78ecd6e91b01e833aa87ca7007b4b75b

[Martyn Gigg]

Update ctest doc tests for multi-config generators.

Refs #9639

Changeset: a0c17e60f8448482a67ee297bd709b48dc5166fd

[Martyn Gigg]

Use a separate doctree directory for each file in single file mode.

Refs #9639

Changeset: abaf5941244f652e5e9f990922a555ce1502fb20

[Martyn Gigg]

Fix problem with overriding SEPARATE_DOC_TESTS variable.

Refs #9639

Changeset: bb42f6f64b13aa237e65e4ac5ddd52bb9950c669

[Martyn Gigg]

Fix bug with not executing doctests in MantidPlot properly.

Refs #9639

Changeset: 34bd7665d1c429490137a20204d0c6a47088fb45

[Martyn Gigg]

Branch: ​feature/9639_doc_tests_inside_ctest

Tester: After merging and building the code there will be a new set of tests inside ctest with the prefix AlgorithmsDocTest. Try running something like

ctest -R AlgorithmsDocTest_Rebin

or on Windows

ctest -C Debug AlgorithmsDocTest_Rebin

and you should see a subset of the tests with Rebin in the name start executing.

A known issue, which is to do with how Sphinx processes things, is that the first execution will be markedly slower than subsequent executions. This is because sphinx is still read all source files even though it doesn't do anything with the majority of them.

[Martyn Gigg]

Pass command line arguments to script in MantidPlot

In the case where MantidPlot acts in place of a standard script interpreter then set any extra arguments after the filename as sys arguments in the scripting environment, e.g. sys.argv in Python. Refs #9639

Changeset: 84cf682b6f7b612a830555d9ee57eee567d96c77

[Martyn Gigg]

Allow runsphinx_*.py scripts to be called by anyone

The driver script for running the sphinx process can now be called by a user without any special requirements. The main use case will be running a doctest individually.

It should be noted that even restricting the run to a single file with the new -R option will actually still cause Sphinx to read every source file at least once. This is baked in to Sphinx and cannot be avoided. However, the config pickling does mean that all source files are only read once and then after this only changed files are reread. Refs #9639 Refs #

Changeset: 2be01abe0e519936839930e9973f464c9b940104

[Martyn Gigg]

Remove configuration for doctests within ctest

This is never going to work with the parallel nature of ctest so we'll stick with the script-based approach. Refs #9639

Changeset: 50e31758259e9504b1c1b41a6edb9b238bc42c23

[Martyn Gigg]

Remove separate doctests flag from build scripts

It is no longer applicable. Refs #9639

Changeset: dd9b711e7ed7804088d63a72a206d915e837d259

[Martyn Gigg]

Fix uninitialized variable warning.

Refs #9639

Changeset: 9d7f8c3637b27f4f1251e5502e3812c7a845da61

[Martyn Gigg]

Catch case where no argument are passed.

Missed this when moving script around. Refs #9639

Changeset: 1b69298cdc3abf8e460c02f1a1eeeae40e252990

[Martyn Gigg]

Branch: ​https://github.com/mantidproject/mantid/compare/feature/9639_doc_tests_inside_ctestfeature/9639_doc_tests_inside_ctest

Tester: This started out being an implementation that hooked into ctest. However, after further testing it found that the parallel nature of ctest doesn't play well with Sphinx's environment pickling. The solution is now a simply driver script generated by cmake in build/docs called runsphinx_doctest.py

The is used by the main docs-test, which runs it without any arguments to run all of the tests. The script accepts a -R argument to give a regex to limit which tests are run. It is advisable to run the tests with MantidPlot as that will mimic, as close as possible, how they are run on the build servers.

As an example, to run all test names with 'Rebin' in filename you run type

bin/MantidPlot -xq docs/runsphinx_doctest.py -R Rebin

on Linux,

bin\[Release|Debug]\MantidPlot -xq docs\runsphinx_doctest.py -R Rebin

on Windows and

bin/Contents/MacOS/MantidPlot -xq docs\runsphinx_doctest.py -R Rebin

on OS X.

Note: On the first run, even with a regex, Sphinx walks the docs source tree and reads all rst files. It then only executes the doctests from the subset limited by the regex. On subsequent runs it will only parse changed files.

[Owen Arnold]

Merge remote-tracking branch 'origin/feature/9639_doc_tests_inside_ctest'

Full changeset: ​1ef3adbd841142b9f0b30877e4a928709ec1465b

[Stuart Campbell]

This ticket has been transferred to github issue ​10482