Note
Click here to download the full example code
Comparison of deep learning architectures
from ai4water.datasets import busan_beach
from ai4water.utils.utils import get_version_info
from ai4water.experiments import DLRegressionExperiments
for k,v in get_version_info().items():
print(f"{k} version: {v}")
python version: 3.7.9 (default, Oct 19 2020, 15:13:17)
[GCC 7.5.0]
os version: posix
ai4water version: 1.06
lightgbm version: 3.3.5
tcn version: 3.5.0
catboost version: 1.1.1
xgboost version: 1.6.2
easy_mpl version: 0.21.2
SeqMetrics version: 1.3.4
tensorflow version: 2.7.0
keras.api._v2.keras version: 2.7.0
numpy version: 1.21.1
pandas version: 1.3.4
matplotlib version: 3.5.3
h5py version: 3.8.0
joblib version: 1.2.0
data = busan_beach()
print(data)
tide_cm wat_temp_c ... rel_hum tetx_coppml
index ...
2018-06-19 00:00:00 36.407149 19.321232 ... 95.000000 NaN
2018-06-19 00:30:00 35.562515 19.320124 ... 95.000000 NaN
2018-06-19 01:00:00 34.808016 19.319666 ... 95.000000 NaN
2018-06-19 01:30:00 30.645216 19.320406 ... 95.006667 NaN
2018-06-19 02:00:00 26.608980 19.326729 ... 95.006667 NaN
... ... ... ... ... ...
2019-09-07 22:00:00 -3.989912 20.990612 ... 88.170000 NaN
2019-09-07 22:30:00 -2.807042 21.012014 ... 88.256667 NaN
2019-09-07 23:00:00 -3.471326 20.831739 ... 87.833333 NaN
2019-09-07 23:30:00 0.707771 21.006086 ... 88.370000 NaN
2019-09-08 00:00:00 1.011731 20.896149 ... 87.700000 NaN
[1446 rows x 14 columns]
comparisons = DLRegressionExperiments(
input_features=data.columns.tolist()[0:-1],
output_features=data.columns.tolist()[-1:],
split_random=True,
val_fraction=0.0,
epochs=200,
ts_args={"lookback": 12},
verbosity=0,
save=False,
)
comparisons.fit(data=data,
include=['MLP',
'LSTM',
'CNNLSTM',
'TCN',
"TFT",
"LSTMAutoEncoder",
])
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
running MLP model
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
running LSTM model
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
running CNNLSTM model
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/SeqMetrics/_rgr.py:1148: RuntimeWarning: invalid value encountered in true_divide
zy = (self.predicted - np.mean(self.predicted)) / np.std(self.predicted, ddof=1)
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/SeqMetrics/_rgr.py:1148: RuntimeWarning: invalid value encountered in true_divide
zy = (self.predicted - np.mean(self.predicted)) / np.std(self.predicted, ddof=1)
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/numpy/lib/function_base.py:2691: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[:, None]
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/numpy/lib/function_base.py:2692: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[None, :]
running TCN model
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
running TFT model
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
running LSTMAutoEncoder model
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
comparisons.compare_errors('r2', data=data)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/numpy/lib/function_base.py:2691: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[:, None]
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/numpy/lib/function_base.py:2692: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[None, :]
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
best_models = comparisons.compare_errors(
'r2',
data=data,
cutoff_type='greater',
cutoff_val=0.01)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/numpy/lib/function_base.py:2691: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[:, None]
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/numpy/lib/function_base.py:2692: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[None, :]
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
comparisons.taylor_plot(data=data)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/numpy/lib/function_base.py:2691: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[:, None]
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/numpy/lib/function_base.py:2692: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[None, :]
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
<Figure size 900x700 with 2 Axes>
comparisons.compare_edf_plots(data=data)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
comparisons.compare_regression_plots(data=data)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
comparisons.compare_residual_plots(data=data)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
********** Removing Examples with nan in labels **********
***** Training *****
input_x shape: (152, 12, 13)
target shape: (152, 1)
********** Removing Examples with nan in labels **********
***** Validation *****
input_x shape: (0,)
target shape: (0,)
********** Removing Examples with nan in labels **********
***** Test *****
input_x shape: (66, 12, 13)
target shape: (66, 1)
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
/home/docs/checkouts/readthedocs.org/user_builds/ai4water-experiments/envs/latest/lib/python3.7/site-packages/ai4water/_main.py:1982: UserWarning:
argument test is deprecated and will be removed in future. Please
use 'predict_on_test_data' method instead.
use 'predict_on_{data}_data' method instead.""")
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
dot plot of model could not be plotted due to ('You must install pydot (`pip install pydot`) and install graphviz (see instructions at https://graphviz.gitlab.io/download/) ', 'for plot_model/model_to_dot to work.')
<Figure size 640x480 with 6 Axes>
comparisons.loss_comparison()
<AxesSubplot:>
Total running time of the script: ( 4 minutes 31.377 seconds)