.. _apps: Application Examples ==================== Standard processes used to analyse and visualise model data have been supplied as executable python script files which are controlled by passing an ASCII Run File and/or parameters at the command line. The applications have been designed to allow processing automation through python scripting. The batch processing scripts are written for the linux operating system, and can be run in the background using the linux ``nohup`` function. The spawning of multiple processes on windows is not as straightforward due to its single-user architecture. A method for running multiple processes on windows has not yet been developed. The applications provided are summarised in :numref:`appslisttab`. .. _appslisttab: .. csv-table:: : Standard processing scripts. :header-rows: 1 :widths: 55, 55, 55 :file: ./apps_list.csv All application scripts start with the interpreter shebang line ``#!/usr/bin/env python3`` so that the scripts can be run directly from the command line without invoking python. The scripts must have executable permissions set (this can be done on linux using the ``chmod`` command). Windows does not interpret the shebang line, so you need to invoke python to run the applications. All of the applipcations are setup so that you can get command line help to indicate what input parameters are required, for example:: $ ./otm_extract_by_location.py -h USAGE: ./otm_extract_by_location.py -r -f $ The Run Files that are passed to applications are text files that are interpreted by a basic text parser. There are a standard set of parser fields that can be used to generate the Run Files and that the parser uses to extract the input parameters. The parser fields are listed in :numref:`parserfieldtab`. The parser is currently case sensitive and the format used to set a field value is Field Name = Field Value where there link between Field Name and Field Value is an equals sign with a single space on either side. The inclusion of extra spaces may lead to errors. The intention is to improve the parser to be less strict on the format and case. There are also some restrictions on the order data are entered into the run files. In particular, where a list of values (`e.g.` location coordinates) are entered this list must be immediately preceeded by a field defining the number of entites. Other fields can be entered in any order because the parser simply checks the field name and converts the parameter to the required data type and saves it in the corresponding parser data field. Examples of input files will be given with the descriptions of the applications. .. _parserfieldtab: .. csv-table:: : Standard parser fields used in run files. :header-rows: 1 :widths: 20, 90 :file: ./parser_fields.csv .. _extrctbyloc: Extract timeseries of Open Telemac-Mascaret data by location ------------------------------------------------------------ .. _otm_extract_by_location.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/otm_extract_by_location.py The extraction of timeseries of data at specific locations within the model domain is required for model cal/val and for targeted analysis. The script `otm_extract_by_location.py`_ is used to extract data from a proprietary format Open Telemac-Mascaret selafin results file for a set of mesh nodes identified by node number, at specified geo-spatial locations identified by geo-spatial coordinates, or for all nodes within a user-defined polygon defined by a set of vertices. There are two input parameters to the script: #. Run Type: This defined type of extraction to run. Options are: * loc = geospatial locations * nod = nodes * pol = polygon #. Filename: The path and name to the Run File that defines the extraction. The form of the output depends on the type of extraction requested. Extraction by geo-spatial location will generated separated files for each location which are identified by a string passed in the Run File. Extraction by nodes or polygon produces a single file with the data for all nodes extracted and a list of corresponding node ids that can be used to interpret the data. This application assumes that the model data are stored using the standardised :ref:`directory structure ` described in the :any:`Overview ` section. The model data are assumed to be in the ``\Model\RESULTS`` directory, and the extracted data will be written to the ``\Model\EXTRACT`` directory. Two example Run Files are provided, shown in :numref:`extract01list` and :numref:`extract02list`. The first example in :numref:`extract01list` extracts data at a geo-spatial location defined by UTM easting and northing coordinates. All of the ``soi_hr_msl_04`` model files generated for the year 2023 are selected by the search string. Not text wildcards can be used in the search string. Only files with data from 15 March 2023 onwards are used and 30 days of data are extracted. The tag ``TideGauge`` is added to teh output file names to identify the data location. If multiple points are requested, each location must have a unique tag to separate the files written. Tags must not include spaces or characters not allowed in filenames. .. _extract01list: .. literalinclude:: ../apps/soi_port_askaig_tidegauge_extract_runfile.txt :language: none :caption: soi_port_askaig_tidegauge_extract_runfile.txt The second example given in :numref:`extract02list` demonstrates how to extract data for all mesh nodes within a polygon. In this case data are extracted from the ``soi_hr_msl_04`` model for 7 days starting from 7 Septenber 2023. The polygon vertices are entered as coordinate pairs of easting and northing. When the polygon is constructed the first and last points are automatically connected by the geometry functions. The polygon must not include any crossing lines or closed loops. .. _extract02list: .. literalinclude:: ../apps/soi_polygon_extract_runfile.txt :language: none :caption: soi_polygon_extract_runfile.txt .. _extrcttrnsct: Extract timeseries of Open Telemac-Mascaret data along a transect ----------------------------------------------------------------- .. _otm_extract_transect.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/otm_extract_transect.py The script `otm_extract_transect.py`_ is used to extract data along a linear transect with a user-defined along-transect sample spacing. Model data are interpolated from the nodes surrounding a given transect location onto the point using Barycentric weighted interpolation. Data are stored in order from the start point to the end point of the defined transect, and velocity data are provided in geo-coordinates and in the along- and across-transect directions. :numref:`transectlist` is an example Run File for extracting timeseries data along a transect. .. _transectlist: .. literalinclude:: ../apps/soi_across_channel_transect_runfile.txt :language: none :caption: soi_across_channel_transect_runfile.txt In this example, we are extracting an across-channel transect with the defined start and end points, with an along-transect spacing of 10.0m from the ``soi_hr_msl_04`` model. The individual results file being processed is defined by the parameter ``Run String``, where the suffix ``_2D.slf`` or ``_3D.slf`` is added internally to extract both the 2D and 3D field data at the transect locations. .. _extrctvect: Extract timeseries of Open Telemac-Mascaret 2D vector field data ---------------------------------------------------------------- .. _otm_model_vector_fields.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/otm_model_vector_fields.py The script `otm_model_vector_fields.py`_ is used to extract surface elevation and depth-averaged velocity gradient fields and the 2D vorticity field based on the depth-averaged velocity. These data are used for dynamical analysis and visualisation. The gradients and vorticity are calculated at the mesh node locations. An example of implementing the script is:: $ ./otm_model_vector_fields.py -d D:/Models/soi_hr_msl_04/RESULTS -f soi_hr_msl_04_20230319_2D.slf -c 1 -n 0 In this example the ``soi_hr_msl_04`` model 19 March 2023 day file is processed, the number of cores is set to 1 so that this run as a single process for the full model domain. When the number of cores = 1 then the set number must be 0. This is not currenty checked. This process is inefficient in its current form, as the full time series is processed for each node sequentially. A more effiecient approach is under development where a node tree is generated for the mesh so that each time step can be processed without haveing to indentify the connected nodes for every timestep. .. _btchextrctvect: Batch process for OTM 2D vector field extraction ------------------------------------------------ .. _otm_batch_model_vector_fields.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/otm_batch_model_vector_fields.py The script `otm_batch_model_vector_fields.py`_ is used to extract the 2D gradient and vorticity data by partitioning the process across an number of cores. This was done to minimise the computation time associated with this process. An example of implementing the script is:: $ ./otm_batch_model_vector_fields.py -d D:/Models/soi_hr_msl_04/RESULTS -f soi_hr_msl_04_20230319_2D.slf -c 40 In this case we are running same case as presented :ref:`above ` using ``otm_model_vector_fields.py``, but partitioned across 40 cores. This will effectively be about 40 times faster to run the process. The output are separate files for each partition set. These set files need to be merged to reproduce the full domain file. .. _mrgvect: Merge OTM batch extracted 2D vector field data ---------------------------------------------- .. _otm_merge_model_vector_fields.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/otm_merge_model_vector_fields.py The script `otm_merge_model_vector_fields.py`_ is used to reconstruct the set files of vector field data generated by the ``otm_batch_model_vector_fields.py`` process. An example of implementing the script is:: $ ./otm_merge_model_vector_fields.py -d D:/Models/soi_hr_msl_04/EXTRACT -s _SET This example combines the data generated by the example given :ref:`above ` using ``otm_batch_model_vector_fields.py``. .. _gencirc: Generate 2D circulation data as individual timestep frames ---------------------------------------------------------- .. _otm_gen_circ_frames.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/otm_gen_circ_frames.py An alternate parameter that can be used to display vorticity is the circulation. This is calculated as a path intergral of the flow around each mesh element. Dividing the circulation by the element area gives the areal average vorticity across the mesh element. The script `otm_gen_circ_frames.py`_ is used to generate individual files of circulation for each timestep in a given model 2D results files. These can be merged for analysis or used for visualisation purposes. An example of implementing the script is:: $ ./otm_gen_circ_frames.py -d D:/Models/soi_hr_msl_04/RESULTS -f soi_hr_msl_04_20230319_2D.slf -c 1 -n 0 This example process a single file output from the ``soi_hr_msl_04`` model on a single core. This application is designed in the same way as the :ref:`otm_model_vector_fields.py ` application to allow batch processing, the main difference is the data are partitioned in time instead of space. .. _btchgencirc: Batch generate 2D circulation data as individual timestep frames ---------------------------------------------------------------- .. _otm_batch_gen_circ_frames.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/otm_batch_gen_circ_frames.py The script `otm_batch_gen_circ_frames.py`_ applies batch processing to the generation of 2D circulation data described :ref:`above `. It partitions by timestep the process across number of cores. As the process generates individual files for each timestep there is no need to merge the resulting data. An example of implementing the script is:: $ ./otm_batch_gen_circ_frames.py -d D:/Models/soi_hr_msl_04/RESULTS -f soi_hr_msl_04_20230319_2D.slf -c 20 This example is running the same case as :ref:`above ` using ``otm_gen_circ_frames.py``, but partitioned across 20 cores, giving an apporimate 20 speed up in the process. .. _cstdist: Calculate OTM mesh node minimum distance to model coastline ----------------------------------------------------------- .. _otm_CoastalDistance.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/otm_CoastalDistance.py The script `otm_CoastalDistance.py`_ is used to calculate the shortest distance of each mesh node from a hard boundary. This is used for the mesh design process where mesh resolution can be controlled by node position in the model domain. An example of implementing the script is:: $ otm_CoastalDistance.py -f D:/Models/soi_hr_msl_04/RESULTS/soi_hr_msl_04_20230319_2D.slf The Open Telemac-Mascaret 2D file output includes the definition of the mesh in the form of node locations and and an element node list. The mesh is resconstucted from these data and the coastal distance calculated. It should be noted that an Open Telemac-Mascaret geometry file can also be used for this calculation as it includes the mesh data, `e.g.`, $ otm_CoastalDistance.py -f D:/Models/soi_hr_msl_04/GEO/soi_hr_msl_04_geo.slf .. _mrgocmw: Merge daily OCMW format data extracts ------------------------------------- .. _merge_ocmw_extracts.py: https://git.ecdf.ed.ac.uk/uoe-open-coastal-modelling/ocmw/-/blob/main/apps/merge_ocmw_extracts.py For long-period model runs, Telemac is typically run in restart mode generating multiple time sequential data files. For analysis and model cal/val extracted data need to be concatenated. The script `merge_ocmw_extracts.py`_ is used to apply this concatation of sequential data extracts. An example of implementing the script is:: $ ./merge_ocmw_extracts.py -d D:/Models/soi_hr_msl_04/EXTRACT -s TideGauge -t model -o soi_hr_msl_04_merged.mat This example is used to merge the timeseries of data extracted from the ``soi_hr_msl_04`` model using the ``otm_extract_by_location.py`` example shown :ref:`above `.