CLINT - Climate Intelligence
Contents
CLINT - Climate Intelligence#
Infilling missing data#
Gridded observational climate datasets such as HadCRUT5 often contain many missing values. To tackle this problem, it is common to fill in the gaps by using the state-of-the-art kriging interpolation or the principal component analysis-based infilling. Unfortunately, these techniques are limited by the statistical information contained in the original dataset and might not be able to efficiently reconstruct large regions of missing values and complex climatic patterns. Artificial Intelligence and more specifically deep-learning based technologies give the possibility to go beyond these techniques by learning the underlying physical patterns in the data.
The method presented here is a deep-learning inpainting technique based on a U-Net architecture. It is making use of partial convolutional layers and a loss function designed to produce semantically meaningful predictions. Models are trained using climate data from reanalysis and/or historical simulations and can be used to reconstruct missing information in observational datasets with minimal computational resources.
CRAI (Climate Reconstruction AI)#
CRAI is a state-of-the-art deep learning based inpainting technology to infill missing values in climate datasets.
The method#
The infilling algorithm is trained with complete climate datasets and masks defining realistic regions of missing observational data.
As shown in the figure, the neural network has a U-net architecture and makes use of partial convolutional layers instead of standard convolution layers. A partial convolutional layer is a combination of two successive operations:
a partial convolution: \(x'=\left\{\begin{array}{ll}\boldsymbol{W}^T\left(\boldsymbol{X}\odot\boldsymbol{M}\right) \frac{\text{sum}(\boldsymbol{1})}{\text{sum}(\boldsymbol{M})} +b, & \text{if sum}(\boldsymbol{M})>0 \\ 0, &\text{otherwise}\end{array}\right.\)
a mask update: \(m'=\left\{\begin{array}{ll}1, & \text{if sum}(\boldsymbol{M})>0 \\ 0, &\text{otherwise}\end{array}\right.\)
where \(\boldsymbol{W}\) are the convolution filter weights, \(\boldsymbol{X}\) the feature values, \(\boldsymbol{M}\) the binary mask values and \(b\) the bias.
Compared to other inpainting methods 1, the partial convolution technique offers several advantages:
it operates robustly on irregular holes
it can infill large holes
it produces semantically meaningful predictions
Installation#
CRAI is written in Python and based on the PyTorch framework and the NVIDIA CUDA® Deep Neural Network library (cuDNN). It is licensed under the terms of the BSD 3-Clause.
Step 1: clone the branch of the repository:
git clone -b clint --single-branch https://github.com/FREVA-CLINT/climatereconstructionAI.git
Step 2: create en environment with all the required dependencies using Anaconda
conda env create -f environment.yml
Step3: activate the environment
conda activate crai
Step4: install the Python package using pip
pip install .
Usage#
The software can be used to:
train a model (training)
infill climate datasets using a trained model (evaluation)
It can be used as:
a command line interface (CLI):
training:
crai-train
evaluation:
crai-evaluate
a Python library:
training:
from climatereconstructionai import train train()
evaluation:
from climatereconstructionai import evaluate evaluate()
For more details about the execution and the input arguments, please have a look to the documentation and demo examples hosted in the repository