How to use mini-CSEP distribution
This page provides detailed instructions on how to use a so-called mini-CSEP distribution of the CSEP Testing Center software.
CSEP V16.10.0 software depends on the following external packages:
- Python Version 3.4.2 http://www.python.org
- matplotlib Version 1.4.3 http://matplotlib.org
- numpy Version 1.9.2 http://numpy.scipy.org
- scipy Version 0.15.1 http://www.scipy.org
- pandas Version 0.17.1 http://pandas.pydata.org
- basemap Version 1.0.7 http://matplotlib.org/basemap
- java Version 1.6.0_10 http://www.java.com
- R Version 2.11.1 (2010-05-31) http://www.r-project.org
- shapely Version 1.5.2 https://pypi.python.org/pypi/Shapely
- requests Version 2.6.2 http://docs.python-requests.org/en/latest
We prefer to use packages that are part of FC22 distribution unless a newer version of the package is required by the CSEP functionality. In such a case, newer version of the package is installed manually within the virtual image which is accessible by the CSEP community for their personal use.
Getting VirtualBox Image
Generic CSEP code base is available as VirtualBox image as of CSEP Version 15.7.0 software release. VirtualBox package is available for all platforms from https://www.virtualbox.org/wiki/Downloads. The miniCSEP virtual image can be downloaded from http://cseptesting.org/downloads/vms/generic/CSEP-generic-latest.ova. Please verify md5sum value once download of the image is complete against http://cseptesting.org/downloads/vms/generic/CSEP-generic-latest.ova.md5sum. This is an exported appliance, which can be uploaded into VirtualBox environment by going to File -> Import Appliance of VirtualBox menu option.
Before starting the image, please update configuration of your VirtualBox installation by going to VirtualBox? VM -> Preferences -> Network -> Host-Only Networks. Then + ADD an adapter, the default name should be called vboxnet(). This configuration change will allow for the network connection between host and the image (in case you would like to do a file transfer between the two). IP address of the CSEP virtual image is set to 192.168.56.101.
The login combination for the image is csep/testing.
Please note that CSEP is installed under /usr/local/csep directory within the image (as defined by the $CENTERCODE environment variable), and properly configured .tcshrc resource file is already set up for the csep user account.
To verify that CSEP code base is working properly, please invoke a set of acceptance tests for the CSEP installation (with an option of using tee command to capture runtime output from acceptance tests in log files for future reference if needed):
python3 /usr/local/csep/src/generic/test/GenericDistributionTest.py -v |& tee distlog.txt python3 /usr/local/csep/src/generic/test/CSEPTest.py -v |& tee testlog.txt
Please note that acceptance tests place all runtime data into tests specific subdirectories under /home/csep/buildCSEP/results directory (as defined by the $CSEP_RESULTS_DIR environment variable).
Getting the Source Code
- Current distribution archive is available at http://cseptesting.org/downloads/CodeDistribution/miniCSEP
- Please validate downloaded file by comparing it's md5sum value against the one posted in csep.tar.gz.meta file available at the same location.
[csep@csep-op ~]$ md5sum /download/location/csepmini-16.10.0.tar.gz >> /download/location/md5.txt [csep@csep-op ~]$ diff /download/location/md5.txt /download/location/csepmini-16.10.0.tar.gz.meta
- Please extract files from miniCSEP distribution archive to the $CSEP directory
[csep@csep-op ~]$ tar zxvf /download/location/csepmini-16.10.0.tar.gz -C $CSEP
CSEP Environment Variables
Please refer to the CSEP environment variables for the description of all required variables.
If using mini-CSEP distribution on the CSEP development server at SCEC, it's recommended to copy environment variables of csep user account from /home/csep/.tcshrc into your ~/.tcshrc resource file. Please make sure that CENTERCODE environment variable points to the CSEP distribution you intend to use (/usr/local/csep or other directory you have placed the CSEP source codes to).
Building the miniCSEP from Source
ATTENTION: Please ignore this section if using /usr/local/csep installation on the CSEP development server at SCEC (csep-devel.usc.edu) or miniCSEP virtual image.
Before invoking the CSEP code, please make sure that you build the binaries, and pass acceptance tests defined for the mini-CSEP distribution.
By default, runtime directories generated by acceptance tests are placed under $CENTERCODE/src/generic/test directory (results-* sub-directory for each test). To change the default location, please update your environment to set up a top-level runtime directory for the tests:setenv CSEP_RESULTS_DIR /home/csep/path/to/CSEPTestsRuntimeDirs
Please use Makefile rules to build, install and test the distribution:
[csep@csep-op csep]$ cd $CENTERCODE [csep@csep-op csep]$ make [csep@csep-op csep]$ make install [csep@csep-op csep]$ make distcheck |& tee testlog.txt
Using the miniCSEP
A set of tutorials is available for various miniCSEP functionality:
- How to invoke RELM N (Number) and L (Likelihood) evaluation tests for two 5-year forecasts:
- How to invoke statistical T (the classical paired t-test) and W (the Wilcoxon signed-rank test) evaluation tests for two 5-year forecasts:
- How to invoke statistical T (the classical paired t-test) and W (the Wilcoxon signed-rank test) evaluation tests for three California 1-day forecasts over a testing period of 10 days that includes Napa 2014 event:
Geographical Region Selection
Default geographical region within CSEP testing framework is set to California. The selection of geographical region controls (with provided examples for California):
- Test area file (source:trunk/data/areas/RELMTestArea.mat or $CENTERCODE/data/areas/RELMTestArea.mat) which is used for preparation of observation catalog for evaluation tests.
- Collection area file (source:trunk/data/areas/RELMCollectionArea.mat or $CENTERCODE/data/areas/RELMCollectionArea.mat) which is used for preparation of input catalog to be passed to forecasts models at runtime to generate forecasts
Other testing regions, that are part of the distribution, are SWPacific, NWPacific, GlobalPDE and Global. Geographical region of interest and corresponding catalog filtering settings (aka post-processing) can be specified through following command-line options to the Python's processing module:
[csep@csep-op csep]$ python ProcessingModule --geographicalRegion=Global --postProcessFactory=$CENTERCODE/configuration/testing-regions/Global/cronjobs/PostProcessFactory.init.xml
GeographicalRegions Python module is used to create:
- A plot of the spatial distribution of the forecast (i.e., map-view of the forecast)
Forecast Python module is used to create:
- A plot of the magnitude distribution of the forecast (i.e., magnitude versus normalized probability)
CSEPXMLGeneric Python module is used for forecasts conversions:
- ASCII to XML
- XML to ASCII
EvaluationTest Python module represents any evaluation test that is defined within CSEP testing framework and used to invoke any combination of:
- Number (N) test
- Log-likelihood (L) test
- Ratio log-likelihood (R) test
- Space (S) test
- Magnitude (M) test.