Development

In addition to extending the toolbox, development of additional core features is also possible. The easiest way to further develop SlyPI is to use the drivers in the test directory.

The general approach taken to date is to write a utility code that interfaces between the command line and the actual SlyPI classes and plugins. To test these utility codes, we have written a variety of drivers which set up the parameters and call the utilites from Python. We use the example data to test the codes, and pointers to the data is coded directly in the test scripts.

The tests are usually run from the root directory.

Slycat Interaction

The script test-slypi.py is used to test the upload sripts. In order to use this script you must have a running Slycat server and may have to modify the code to connect. The current setup uses a docker container.

python tests/integration/test-slypi.py

The results of this script are uploads of various models to the Slycat server. The VideoSwarm upload script is not tested because it requires file links that generally do not work with a docker container.

Ensemble Utilities

The remaining tests are for the ensemble utilities. The parsing funtionality is tested using test-ensemble.py.

python tests/unit/test-ensemble.py

Other tests for the ensemble are test-ensemble-convert.py, test-ensemble-reduce.py and test-ensemble-table.py. Some of the tests can be slow, so there are options to execute only certain tests.

There is a certain amount of inter-dependence between the tests. For example, test-ensemble-reduce.py and test-ensemble-convert.py must precede test-ensemble-table.py to provide files for creating tables. After running the first two tests, use

python tests/integration/test-ensemble-table.py

To run the parallel tests, you need to have ipyparallel running. The necessary procedure will vary depending on your environment. If you are running the tests locally, you can use

ipcluster start --n=4

from the slypi directory.

test-ensemble-convert.py

python tests/integration/test-ensemble-convert.py --help

usage: test-ensemble-convert.py [-h] [--delete-output-dir] [--test-UI]
                                [--test-conversions] [--test-end-state]
                                [--test-parallel] [--test-all]

Generate various test .csv files from phase-field ensemble data.

options:
  -h, --help           show this help message and exit
  --delete-output-dir  Delete output directory before starting tests.
  --test-UI            Test command line UI for convert.py.
  --test-conversions   Test conversions.
  --test-end-state     Do end-state conversions (e.g. images and movies).
  --test-parallel      Run parallel tests using ipyparallel (must have
                       ipengine running).
  --test-all           Run every test.

test-ensemble-reduce.py

python tests/integration/test-ensemble-reduce.py --help

usage: test-ensemble-reduce.py [-h] [--delete-output-dir] [--test-UI]
                               [--test-save-load] [--test-end-state]
                               [--test-time-aligned] [--test-all-time]
                               [--test-umap] [--test-parallel]
                               [--test-restart] [--test-rd-npy] [--test-all]

Generate various test .csv files from phase-field ensemble data.

options:
  -h, --help           show this help message and exit
  --delete-output-dir  Delete output directory before starting tests.
  --test-UI            Test command line UI for reduce.py.
  --test-save-load     Test save and load capability in reduce.py.
  --test-end-state     Do end-state dimension reductions.
  --test-time-aligned  Do time-aligned dimension reductions.
  --test-all-time      Do dimension reductions with all time steps
                       simultaneously.
  --test-umap          Perform umap reduction on test set.
  --test-parallel      Run parallel tests.
  --test-restart       Run restart testing.
  --test-rd-npy        Test dimension reduction loaded from rd.npy file.
  --test-all           Run every test.