Skip to content

Commit

Permalink
docs: minor wording and typographic edits in tutorials E,F,G,H
Browse files Browse the repository at this point in the history
+ provided & linked references
  • Loading branch information
mherrmann3 committed Oct 19, 2024
1 parent 358c33f commit 43f370b
Show file tree
Hide file tree
Showing 4 changed files with 48 additions and 39 deletions.
10 changes: 8 additions & 2 deletions docs/tutorials/case_e.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
E - A Time-Independent Experiment
=================================

This example shows how to run a realistic testing experiment (based on https://doi.org/10.4401/ag-4844) while summarizing the concepts from the previous tutorials.
This example shows how to run a realistic testing experiment (based on :ref:`Schorlemmer et al. 2010<References>`) while summarizing the concepts from the previous tutorials.

.. currentmodule:: floatcsep

Expand All @@ -25,7 +25,7 @@ This example shows how to run a realistic testing experiment (based on https://d
Experiment Components
---------------------

The source code can be found in the ``tutorials/case_e`` folder or in `GitHub <https://github.com/cseptesting/floatcsep/blob/main/tutorials/case_e>`_. The input structure of the experiment is:
The source code can be found in the ``tutorials/case_e`` folder or in `the GitHub repository <https://github.com/cseptesting/floatcsep/blob/main/tutorials/case_e>`_. The input structure of the experiment is:

::

Expand Down Expand Up @@ -137,3 +137,9 @@ Plot command
colormap: magma
and re-run with the ``plot`` command. A forecast figure will re-appear in ``results/{window}/forecasts`` with a different colormap. Additional forecast and catalog plotting options can be found in the :func:`csep.utils.plots.plot_spatial_dataset` and :func:`csep.utils.plots.plot_catalog` ``pycsep`` functions.


References
----------

* Schorlemmer, D., Christophersen, A., Rovida, A., Mele, F., Stucchi, M. and Marzocchi, W. (2010). Setting up an earthquake forecast experiment in Italy. Annals of Geophysics, 53(3), 1–9. doi: `10.4401/ag-4844 <https://doi.org/10.4401/ag-4844>`_
19 changes: 12 additions & 7 deletions docs/tutorials/case_f.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Experiment Components
---------------------


The source files can be found in the ``tutorials/case_e`` folder or in `GitHub <https://github.com/cseptesting/floatcsep/blob/main/tutorials/case_e>`_. The experiment structure is as follows:
The source files can be found in the ``tutorials/case_e`` folder or in `the GitHub repository <https://github.com/cseptesting/floatcsep/blob/main/tutorials/case_e>`_. The experiment structure is as follows:

::

Expand All @@ -49,7 +49,7 @@ The source files can be found in the ``tutorials/case_e`` folder or in `GitHub
Model
-----

The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (`doi:10.1785/0220200231 <https://doi.org/10.1785/0220200231>`_), with which the experiment has no interface. This means that we use **only the forecast files** and no source code. We leave the handling of a model source code for subsequent tutorials.
The time-dependency of a model is manifested here by the provision of different forecasts, i.e., statistical descriptions of seismicity, for different time-windows. In this example, the forecasts were created from an external model https://github.com/lmizrahi/etas (:ref:`Mizrahi et al. 2021<References>`_), with which the experiment has no interface. This means that we use **only the forecast files** and no source code. We leave the handling of a model source code for subsequent tutorials.



Expand All @@ -73,7 +73,7 @@ Time
Catalog
~~~~~~~

The catalog ``catalog.json`` was obtained *previously* by using ``query_geonet`` and it was filtered to the testing period. However, it can be re-queried by changing its definition to:
The catalog ``catalog.json`` was obtained *prior* to the experiment by using ``query_geonet`` and it was filtered to the testing period. However, it can be re-queried by changing its definition to:

.. code-block:: yaml
Expand All @@ -93,29 +93,34 @@ Models
For consistency with time-dependent models that will create forecasts from a source code, the ``path`` should point to the folder of the model, which itself should contain a sub-folder named ``{path}/forecasts`` where the files are located.

.. important::
Note that for catalog-based forecasts, the model should explicit the number of simulations. This is meant for forecast files that contain synthetic catalogs with zero-event simulations, and therefore do not contain the total number of synthetic catalogs used.
Note that for catalog-based forecast models, the number of catalog simulations (``n_sims``) must be specified – because a forecast may contain synthetic catalogs with zero-event simulations and therefore does not imply the total number of simulated synthetic catalogs.

Tests
~~~~~

With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used.
Having a time-dependent and catalog-based forecast model, catalog-based evaluations found in :obj:`csep.core.catalog_evaluations` can now be used.


.. literalinclude:: ../../tutorials/case_f/tests.yml
:language: yaml

.. note::
It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath
It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath.


Running the experiment
----------------------

The experiment can be run by simply navigating to the ``tutorials/case_h`` folder in the terminal and typing.
The experiment can be run by simply navigating to the ``tutorials/case_h`` folder in the terminal and typing:

.. code-block:: console
$ floatcsep run config.yml
This will automatically set all the calculation paths (testing catalogs, evaluation results, figures) and will create a summarized report in ``results/report.md``.


References
----------

* Mizrahi, L., Nandan, S., & Wiemer, S. (2021). The effect of declustering on the size distribution of mainshocks. _Seismological Research Letters, 92_(4), 2333–2342. doi: `10.1785/0220200231 <https://doi.org/10.1785/0220200231>`_
38 changes: 18 additions & 20 deletions docs/tutorials/case_g.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Here, we set up a time-dependent model from its **source code** for an experimen
Experiment Components
---------------------

The example folder contains also, along with the already known components (configurations, catalog), a sub-folder for the **source code** of the model ``pymock``. The components of the experiment (and model) are:
The example folder contains also, along with the already known components (configurations, catalog), a sub-folder for the **source code** of the model `pymock <https://git.gfz-potsdam.de/csep/it_experiment/models/pymock>`_. The components of the experiment (and model) are:

::

Expand Down Expand Up @@ -63,7 +63,7 @@ The example folder contains also, along with the already known components (confi
Model
-----

The experiment's complexity increases from time-independent to dependent mostly because we now need a **Model** (source code) to generate forecasts that changes for every time-window. The model main components are:
Transitioning from time-independent to dependent models increases an experiment's complexity because we now need a **Model** (source code) to generate forecasts that change for every time-window. A **Model**'s main components are:


* **Input**: The input consists in input **data** and **arguments**.
Expand All @@ -76,35 +76,33 @@ The experiment's complexity increases from time-independent to dependent mostly

2. The **input arguments** controls how the model's source code works. The minimum arguments to run a model are the forecast ``start_date`` and ``end_date``, which will be modified dynamically during an experiment with multiple time-windows. The experiment system will access `{model}/input/args.txt` and change the values of ``start_date = {datetime}`` and ``end_date = {datetime}`` before the model is run. Additional arguments can be set by convenience, such as (not limited to) ``catalog`` (the input catalog name), ``n_sims`` (number of synthetic catalogs) and random ``seed`` for reproducibility.

* **Output**: The model's output are the synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv` by the source code after each rone. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` folows the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT
* **Output**: The model's output are synthetic catalogs, which should be allocated in `{model}/forecasts/{filename}.csv` by the source code after each run. The format is identically to ``csep_ascii``, but unlike in an input catalog, the ``catalog_id`` column should be modified for each synthetic catalog starting from 0. The file name follows the convention `{model_name}_{start}_{end}.csv`, where ``start`` and ``end`` follows the `%Y-%m-%dT%H:%M:%S.%f` - ISO861 FORMAT.

* **Model build**: Inside the model source code, there are multiple options to build it. A standard python ``setup.cfg`` is given, which can be built inside a python ``venv`` or ``conda`` managers. This is created and built automatically by ``floatCSEP``, as long as the the model build instructions are correctly set up.
* **Model build**: Inside the model source code, there are multiple options to build it. A standard Python ``setup.cfg`` is given, which can be built inside a Python ``venv`` or ``conda`` managers. This is created and built automatically by ``floatCSEP``, as long as the the model build instructions are correctly set up.

* **Model run**: The model should be run with a simple command, e.g. **entrypoint**, to which only ``arguments`` could be passed if desired. The ``pymock`` model contains multiple example of entrypoints, but the modeler should use only one for clarity.

1. A `python` call with arguments
1. A ``python`` call with arguments:

.. code-block:: console
$ python run.py input/args.txt
2. Using a binary entrypoint with arguments (for instance, defined in the python build instructions: ``pymock/setup.cfg:entry_point``)
2. Using a binary entrypoint with arguments (for instance, defined in the Python build instructions: ``pymock/setup.cfg:entry_point``):

.. code-block:: console
$ pymock input/args.txt
3. A single binary entrypoint without arguments .
3. A single binary entrypoint without arguments, which means that the source code should internally read the input data and arguments (``input/catalog.csv`` and ``input/args.txt`` files, respectively):

.. code-block:: console
$ pymock
This means that the source code should internally read the input data and arguments, ``input/catalog.csv`` and ``input/args.txt`` files respectively.

.. important::

The model should be conceptualized as a **black-box**, whose only interface/interaction with the ``floatcsep`` system is to receive an input (i.e., input catalog and arguments) and generates an output (the forecasts).
A **Model** can be conceptualized as a **black-box**, whose only interface/interaction with the ``floatcsep`` system is to receive an input (i.e., input catalog and arguments) and subsequently generate an output (the forecasts).


Configuration
Expand All @@ -124,7 +122,7 @@ Time
Catalog
~~~~~~~

The catalog was obtained `previous to the experiment` using ``query_bsi``, but it was filtered from 2006 onwards, so it has enough data for the model calibration.
The catalog was obtained *prior* to the experiment using ``query_bsi``, but it was filtered from 2006 onwards, so it has enough data for the model calibration.

Models
~~~~~~
Expand All @@ -137,13 +135,13 @@ Models
:lines: 1-7

1. Now ``path`` points to the folder where the source is installed. Therefore, the input and the forecasts should be allocated ``{path}/input`` and ``{path}/forecasts``, respectively.
2. The ``func`` option is the shell command with which the model is run. As seen in the `Model`_ section, this could be either ``pymock``, ``pymock input/args.txt`` or ``python run.py input/args``. We use the simplest option ``pymock``, but you are welcome to try different entrypoints.
2. The ``func`` option is the shell command with which the model is run. As seen in the `Model` section, this could be either ``pymock``, ``pymock input/args.txt`` or ``python run.py input/args``. We use the simplest option ``pymock``, but you are welcome to try different entrypoints.

.. note::
The ``func`` command will be run from the model's directory and a model containerization (e.g., ``Dockerfile``, ``conda``).

3. The ``func_kwargs`` are extra arguments that will annotated to the ``input/args.txt`` file every time the model is run, or will be passed as extra arguments to the ``func`` call (Note that the two options are identical). This is useful to define sub-classes of models (or flavours) that uses the same source code, but a different instantiation.
4. The ``build`` option defines the style of container within which the model will be placed. Currently in **floatCSEP**, only the python module ``venv``, the package manager ``conda`` and the containerization manager ``Docker`` are currently supported.
3. The ``func_kwargs`` are extra arguments that will be added to the ``input/args.txt`` file every time the model is run, or will be passed as extra arguments to the ``func`` call (Note that the two options are identical). This is useful to define sub-classes of models (or flavours) that uses the same source code, but a different instantiation.
4. The ``build`` option defines the style of container within which the model will be placed. Currently in **floatCSEP**, only the Python module ``venv``, the package manager ``conda`` and the containerization manager ``Docker`` are currently supported.

.. important::
For these tutorials, we use ``venv`` sub-environments, but we recommend ``Docker`` to set up real experiments.
Expand All @@ -152,15 +150,15 @@ Models
Tests
~~~~~

With time-dependent models, now catalog evaluations found in :obj:`csep.core.catalog_evaluations` can be used.
Catalog-based evaluations found in :obj:`csep.core.catalog_evaluations` can be used.


.. literalinclude:: ../../tutorials/case_g/tests.yml
:caption: tutorials/case_g/tests.yml
:language: yaml

.. note::
It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath
It is possible to assign two plotting functions to a test, whose ``plot_args`` and ``plot_kwargs`` can be placed indented beneath.


Custom Post-Process
Expand All @@ -173,13 +171,13 @@ Custom Post-Process
:language: yaml
:lines: 22-23

This option provides `hook` for a python script and a function within as:
This option provides a `hook` for a Python script and a function within as:

.. code-block:: console
{python_sript}:{function_name}
The requirements are that the script to be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument
The script must be located within the same directory as the configuration file, whereas the function must receive a :class:`floatcsep.experiment.Experiment` as argument:

.. literalinclude:: ../../tutorials/case_g/custom_plot_script.py
:caption: tutorials/case_g/custom_plot_script.py
Expand All @@ -188,13 +186,13 @@ Custom Post-Process



In this way, the plot function can use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``tutorials/case_g/custom_plot_script.py`` can also be viewed directly on `GitHub <https://github.com/cseptesting/floatcsep/blob/main/tutorials/case_g/custom_plot_script.py>`_, where it is exemplified how to access the experiment data in runtime.
In this way, the plot function can use all the :class:`~floatcsep.experiment.Experiment` attributes/methods to access catalogs, forecasts and test results. The script ``tutorials/case_g/custom_plot_script.py`` can also be viewed directly in `the GitHub repository <https://github.com/cseptesting/floatcsep/blob/main/tutorials/case_g/custom_plot_script.py>`_, where it is exemplified how to access the experiment data at runtime.


Running the experiment
----------------------

The experiment can be run by simply navigating to the ``tutorials/case_g`` folder in the terminal and typing.
The experiment can be run by simply navigating to the ``tutorials/case_g`` folder in the terminal and typing:

.. code-block:: console
Expand Down
Loading

0 comments on commit 43f370b

Please sign in to comment.