cdo: run subset operations (command line)#

sel operators

! cdo --help sellonlatbox
NAME
    sellonlatbox, selindexbox - Select a box of a field

SYNOPSIS
    sellonlatbox,lon1,lon2,lat1,lat2  infile outfile
    selindexbox,idx1,idx2,idy1,idy2  infile outfile

DESCRIPTION
    Selects a box of the rectangularly understood field.

OPERATORS
    sellonlatbox  Select a longitude/latitude box
                  Selects a regular longitude/latitude box. The user has to give the longitudes and latitudes of the 
                  edges of the box. Considered are only those grid cells with the grid center inside the lon/lat box.
                  For rotated lon/lat grids the parameter needs to be rotated coordinates.
    selindexbox   Select an index box
                  Selects an index box. The user has to give the indices of the edges of the box. The index of the 
                  left edge may be greater then that of the right edge.

PARAMETER
    lon1  FLOAT    Western longitude in degrees
    lon2  FLOAT    Eastern longitude in degrees
    lat1  FLOAT    Southern or northern latitude in degrees
    lat2  FLOAT    Northern or southern latitude in degrees
    idx1  INTEGER  Index of first longitude (1 - nlon)
    idx2  INTEGER  Index of last longitude (1 - nlon)
    idy1  INTEGER  Index of first latitude (1 - nlat)
    idy2  INTEGER  Index of last latitude (1 - nlat)
! cdo --help seldate
NAME
    seltimestep, seltime, selhour, selday, selmonth, selyear, selseason, seldate, 
    selsmon - Select timesteps

SYNOPSIS
    seltimestep,timesteps  infile outfile
    seltime,times  infile outfile
    selhour,hours  infile outfile
    selday,days  infile outfile
    selmonth,months  infile outfile
    selyear,years  infile outfile
    selseason,seasons  infile outfile
    seldate,startdate[,enddate]  infile outfile
    selsmon,month[,nts1[,nts2]]  infile outfile

DESCRIPTION
    This module selects user specified timesteps from infile and writes them to outfile.
    The timesteps selected depends on the chosen operator and the parameters. A range of integer values
    can be specified by first/last[/inc].

OPERATORS
    seltimestep  Select timesteps
                 Selects all timesteps with a timestep in a user given list or range.
    seltime      Select times
                 Selects all timesteps with a time in a user given list or range.
    selhour      Select hours
                 Selects all timesteps with a hour in a user given list or range.
    selday       Select days
                 Selects all timesteps with a day in a user given list or range.
    selmonth     Select months
                 Selects all timesteps with a month in a user given list or range.
    selyear      Select years
                 Selects all timesteps with a year in a user given list or range.
    selseason    Select seasons
                 Selects all timesteps with a month of a season in a user given list.
    seldate      Select dates
                 Selects all timesteps with a date in a user given range.
    selsmon      Select single month
                 Selects a month and optional an arbitrary number of timesteps before and after this month.

PARAMETER
    timesteps  INTEGER  Comma-separated list or first/last[/inc] range of timesteps. Negative values select timesteps from the end (NetCDF only).
    times      STRING   Comma-separated list of times (format hh:mm:ss).
    hours      INTEGER  Comma-separated list or first/last[/inc] range of hours.
    days       INTEGER  Comma-separated list or first/last[/inc] range of days.
    months     INTEGER  Comma-separated list or first/last[/inc] range of months.
    years      INTEGER  Comma-separated list or first/last[/inc] range of years.
    seasons    STRING   Comma-separated list of seasons (substring of DJFMAMJJASOND or ANN).
    startdate  STRING   Start date (format YYYY-MM-DDThh:mm:ss).
    enddate    STRING   End date (format YYYY-MM-DDThh:mm:ss) [default: startdate].
    nts1       INTEGER  Number of timesteps before the selected month [default: 0].
    nts2       INTEGER  Number of timesteps after the selected month [default: nts1].

select time range by date, and lon-lat box

# List files
! ls /pool/data/CMIP6/data/CMIP/MPI-M/MPI-ESM1-2-HR/historical/r1i1p1f1/Amon/tas/gn/v20190710/
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_185001-185412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_185501-185912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_186001-186412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_186501-186912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_187001-187412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_187501-187912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_188001-188412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_188501-188912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_189001-189412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_189501-189912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_190001-190412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_190501-190912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_191001-191412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_191501-191912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_192001-192412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_192501-192912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_193001-193412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_193501-193912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_194001-194412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_194501-194912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_195001-195412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_195501-195912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_196001-196412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_196501-196912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_197001-197412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_197501-197912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_198001-198412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_198501-198912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_199001-199412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_199501-199912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_200001-200412.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_200501-200912.nc
tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_201001-201412.nc
# Specify input and output
inputfile = "/pool/data/CMIP6/data/CMIP/MPI-M/MPI-ESM1-2-HR/historical/r1i1p1f1/Amon/tas/gn/v20190710/"
inputfile += "tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_200001-200412.nc"
outputfile = "tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_200001-200001.nc"
# Call cdo
! cdo -O seldate,20000101,20000131 -sellonlatbox,-40,70,-40,70 {inputfile} {outputfile}
cdo(1) sellonlatbox: Process started
cdo(1) sellonlatbox: Processed 4423680 values from 1 variable over 60 timesteps.
cdo    seldate: Processed 13806 values from 1 variable over 2 timesteps [0.14s 46MB].

Display file information

! ncdump -h {outputfile}
netcdf tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_200001-200001 {
dimensions:
	time = UNLIMITED ; // (1 currently)
	bnds = 2 ;
	lon = 117 ;
	lat = 118 ;
variables:
	double time(time) ;
		time:standard_name = "time" ;
		time:long_name = "time" ;
		time:bounds = "time_bnds" ;
		time:units = "days since 1850-1-1 00:00:00" ;
		time:calendar = "proleptic_gregorian" ;
		time:axis = "T" ;
	double time_bnds(time, bnds) ;
	double lon(lon) ;
		lon:standard_name = "longitude" ;
		lon:long_name = "Longitude" ;
		lon:units = "degrees_east" ;
		lon:axis = "X" ;
		lon:bounds = "lon_bnds" ;
	double lon_bnds(lon, bnds) ;
	double lat(lat) ;
		lat:standard_name = "latitude" ;
		lat:long_name = "Latitude" ;
		lat:units = "degrees_north" ;
		lat:axis = "Y" ;
		lat:bounds = "lat_bnds" ;
	double lat_bnds(lat, bnds) ;
	double height ;
		height:standard_name = "height" ;
		height:long_name = "height" ;
		height:units = "m" ;
		height:positive = "up" ;
		height:axis = "Z" ;
	float tas(time, lat, lon) ;
		tas:standard_name = "air_temperature" ;
		tas:long_name = "Near-Surface Air Temperature" ;
		tas:units = "K" ;
		tas:CDI_grid_type = "gaussian" ;
		tas:CDI_grid_num_LPE = 96 ;
		tas:coordinates = "height" ;
		tas:_FillValue = 1.e+20f ;
		tas:missing_value = 1.e+20f ;
		tas:comment = "near-surface (usually, 2 meter) air temperature" ;
		tas:cell_methods = "area: time: mean" ;
		tas:cell_measures = "area: areacella" ;
		tas:history = "2019-08-25T11:09:34Z altered by CMOR: Treated scalar dimension: \'height\'. 2019-08-25T11:09:34Z altered by CMOR: replaced missing value flag (-9e+33) and corresponding data with standard missing value (1e+20). 2019-08-25T11:09:34Z altered by CMOR: Inverted axis: lat." ;

// global attributes:
		:CDI = "Climate Data Interface version 2.0.5 (https://mpimet.mpg.de/cdi)" ;
		:source = "MPI-ESM1.2-HR (2017): \n",
			"aerosol: none, prescribed MACv2-SP\n",
			"atmos: ECHAM6.3 (spectral T127; 384 x 192 longitude/latitude; 95 levels; top level 0.01 hPa)\n",
			"atmosChem: none\n",
			"land: JSBACH3.20\n",
			"landIce: none/prescribed\n",
			"ocean: MPIOM1.63 (tripolar TP04, approximately 0.4deg; 802 x 404 longitude/latitude; 40 levels; top grid cell 0-12 m)\n",
			"ocnBgchem: HAMOCC6\n",
			"seaIce: unnamed (thermodynamic (Semtner zero-layer) dynamic (Hibler 79) sea ice model)" ;
		:institution = "Max Planck Institute for Meteorology, Hamburg 20146, Germany" ;
		:Conventions = "CF-1.7 CMIP-6.2" ;
		:activity_id = "CMIP" ;
		:branch_method = "standard" ;
		:branch_time_in_child = 0. ;
		:branch_time_in_parent = 0. ;
		:contact = "cmip6-mpi-esm@dkrz.de" ;
		:creation_date = "2019-08-25T11:09:34Z" ;
		:data_specs_version = "01.00.30" ;
		:experiment = "all-forcing simulation of the recent past" ;
		:experiment_id = "historical" ;
		:external_variables = "areacella" ;
		:forcing_index = 1 ;
		:frequency = "mon" ;
		:further_info_url = "https://furtherinfo.es-doc.org/CMIP6.MPI-M.MPI-ESM1-2-HR.historical.none.r1i1p1f1" ;
		:grid = "gn" ;
		:grid_label = "gn" ;
		:history = "Wed Aug 31 09:58:08 2022: cdo -O seldate,20000101,20000131 -sellonlatbox,-40,70,-40,70 /pool/data/CMIP6/data/CMIP/MPI-M/MPI-ESM1-2-HR/historical/r1i1p1f1/Amon/tas/gn/v20190710/tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_200001-200412.nc tas_Amon_MPI-ESM1-2-HR_historical_r1i1p1f1_gn_200001-200001.nc\n",
			"2019-08-25T11:09:34Z ; CMOR rewrote data to be consistent with CMIP6, CF-1.7 CMIP-6.2 and CF standards." ;
		:initialization_index = 1 ;
		:institution_id = "MPI-M" ;
		:mip_era = "CMIP6" ;
		:nominal_resolution = "100 km" ;
		:parent_activity_id = "CMIP" ;
		:parent_experiment_id = "piControl" ;
		:parent_mip_era = "CMIP6" ;
		:parent_source_id = "MPI-ESM1-2-HR" ;
		:parent_time_units = "days since 1850-1-1 00:00:00" ;
		:parent_variant_label = "r1i1p1f1" ;
		:physics_index = 1 ;
		:product = "model-output" ;
		:project_id = "CMIP6" ;
		:realization_index = 1 ;
		:realm = "atmos" ;
		:references = "MPI-ESM: Mauritsen, T. et al. (2019), Developments in the MPI‐M Earth System Model version 1.2 (MPI‐ESM1.2) and Its Response to Increasing CO2, J. Adv. Model. Earth Syst.,11, 998-1038, doi:10.1029/2018MS001400,\n",
			"Mueller, W.A. et al. (2018): A high‐resolution version of the Max Planck Institute Earth System Model MPI‐ESM1.2‐HR. J. Adv. Model. EarthSyst.,10,1383–1413, doi:10.1029/2017MS001217" ;
		:source_id = "MPI-ESM1-2-HR" ;
		:source_type = "AOGCM" ;
		:sub_experiment = "none" ;
		:sub_experiment_id = "none" ;
		:table_id = "Amon" ;
		:table_info = "Creation Date:(09 May 2019) MD5:e6ef8ececc8f338646ebfb3aeed36bfc" ;
		:title = "MPI-ESM1-2-HR output prepared for CMIP6" ;
		:variable_id = "tas" ;
		:variant_label = "r1i1p1f1" ;
		:license = "CMIP6 model data produced by MPI-M is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
		:cmor_version = "3.5.0" ;
		:tracking_id = "hdl:21.14100/af75dd9f-d9c2-4e0e-a294-2bb0d5b740cf" ;
		:CDO = "Climate Data Operators version 2.0.5 (https://mpimet.mpg.de/cdo)" ;
}

Quickplot using xarray and hvplot

import hvplot.xarray, xarray as xr

ds = xr.open_dataset(outputfile)
ds
<xarray.Dataset>
Dimensions:    (time: 1, bnds: 2, lon: 117, lat: 118)
Coordinates:
  * time       (time) datetime64[ns] 2000-01-16T12:00:00
  * lon        (lon) float64 -39.38 -38.44 -37.5 -36.56 ... 67.5 68.44 69.38
  * lat        (lat) float64 -39.74 -38.81 -37.87 -36.93 ... 67.79 68.73 69.66
    height     float64 2.0
Dimensions without coordinates: bnds
Data variables:
    time_bnds  (time, bnds) datetime64[ns] 2000-01-01 2000-02-01
    lon_bnds   (lon, bnds) float64 -39.84 -38.91 -38.91 ... 68.91 68.91 69.84
    lat_bnds   (lat, bnds) float64 -40.21 -39.27 -39.27 ... 69.19 69.19 70.13
    tas        (time, lat, lon) float32 289.4 289.2 288.9 ... 256.7 256.1 255.4
Attributes: (12/49)
    CDI:                    Climate Data Interface version 2.0.5 (https://mpi...
    source:                 MPI-ESM1.2-HR (2017): \naerosol: none, prescribed...
    institution:            Max Planck Institute for Meteorology, Hamburg 201...
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            CMIP
    branch_method:          standard
    ...                     ...
    variable_id:            tas
    variant_label:          r1i1p1f1
    license:                CMIP6 model data produced by MPI-M is licensed un...
    cmor_version:           3.5.0
    tracking_id:            hdl:21.14100/af75dd9f-d9c2-4e0e-a294-2bb0d5b740cf
    CDO:                    Climate Data Operators version 2.0.5 (https://mpi...
# Plot a single timestep using xarray
ds.tas.isel(time=0).plot()

# Dynamic plot of the entire timeseries using hvplot
#ds.tas.hvplot.quadmesh(width=600)
<matplotlib.collections.QuadMesh at 0x7fff4c9ad9c0>
_images/cdo-subset-cmip6_12_1.png