CAMELS Australia

This example demonstrates how to use the aqua_fetch package to download and explore the CAMELS Australia dataset using the :py:class:aqua_fetch.RainfallRunoff class. Although we show it for CAMELS Australia, the same can be done for all other rainfall runoff datasets.

Note: This file runs online on readthedocs everytime the documentation is built. The server to download the CAMELS_AUS data is sometimes down and gives HTTPError: HTTP Error 500: Internal Server Error.

[1]:

import os
import site

if __name__ == '__main__':
    wd_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath('__file__')))))
    #wd_dir = os.path.dirname(os.path.realpath('__file__'))
    #wd_dir = os.path.dirname(os.path.dirname(os.path.realpath('__file__')))
    print(wd_dir)
    site.addsitedir(wd_dir)

from tabulight import EDA
import matplotlib.pyplot as plt
from easy_mpl import scatter, hist
from easy_mpl.utils import process_cbar
from aqua_fetch import RainfallRunoff
from aqua_fetch.utils import print_info

/home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest

[2]:

print_info()

numpy 1.26.4
pandas 2.1.4
aqua_fetch 1.0.1
python 3.12.10 (main, May  6 2025, 11:38:28) [GCC 9.4.0]
os posix
matplotlib 3.8.4
shapefile 2.3.1
xarray 2024.7.0
netCDF4 1.7.4
scipy 1.17.0
fiona 1.10.1
Script Executed on:  11 February 2026 08:18:57
tot_cpus 2
avail_cpus 2
mem_gib 7.555534362792969

[3]:

dataset = RainfallRunoff('CAMELS_AUS', version=1,
                         #overwrite=True,
                         #path='/mnt/datawaha/hyex/atr/gscad_database/raw/CAMELS_AUS_V1'
                         )

01_id_name_metadata.zip already exists at /home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/data/CAMELS/CAMELS_AUS
02_location_boundary_area.zip already exists at /home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/data/CAMELS/CAMELS_AUS
03_streamflow.zip already exists at /home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/data/CAMELS/CAMELS_AUS
04_attributes.zip already exists at /home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/data/CAMELS/CAMELS_AUS
05_hydrometeorology.zip already exists at /home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/data/CAMELS/CAMELS_AUS
CAMELS_AUS_Attributes&Indices_MasterTable.csv already exists at /home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/data/CAMELS/CAMELS_AUS
unzipping files in /home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/data/CAMELS/CAMELS_AUS
05_hydrometeorology already exists
02_location_boundary_area already exists
03_streamflow already exists
04_attributes already exists
01_id_name_metadata already exists
dynamic data already exists as /home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/data/CAMELS/CAMELS_AUS/camels_aus_D.nc. To overwrite, set `overwrite=True`

[4]:

dataset.start

[4]:

'19500101'

[5]:

dataset.end

[5]:

'20181231'

[6]:

stations = dataset.stations()
len(stations)

[6]:

222

[7]:

stations[0:10]

[7]:

['912101A',
 '912105A',
 '915011A',
 '917107A',
 '919003A',
 '919201A',
 '919309A',
 '922101B',
 '925001A',
 '926002A']

Static Features

[8]:

dataset.static_features

[8]:

['station_name',
 'drainage_division',
 'river_region',
 'notes',
 'lat',
 'long',
 'lat_centroid',
 'long_centroid',
 'map_zone',
 'area_km2',
 'state_outlet',
 'state_alt',
 'daystart',
 'daystart_P',
 'daystart_Q',
 'nested_status',
 'next_station_ds',
 'num_nested_within',
 'start_date',
 'end_date',
 'prop_missing_data',
 'q_uncert_num_curves',
 'q_uncert_n',
 'q_uncert_q10',
 'q_uncert_q10_upper',
 'q_uncert_q10_lower',
 'q_uncert_q50',
 'q_uncert_q50_upper',
 'q_uncert_q50_lower',
 'q_uncert_q90',
 'q_uncert_q90_upper',
 'q_uncert_q90_lower',
 'p_mean',
 'pet_mean',
 'aridity',
 'p_seasonality',
 'frac_snow',
 'high_prec_freq',
 'high_prec_dur',
 'high_prec_timing',
 'low_prec_freq',
 'low_prec_dur',
 'low_prec_timing',
 'q_mean',
 'runoff_ratio',
 'stream_elas',
 'slope_fdc',
 'baseflow_index',
 'hdf_mean',
 'Q5',
 'Q95',
 'high_q_freq',
 'high_q_dur',
 'low_q_freq',
 'low_q_dur',
 'zero_q_freq',
 'geol_prim',
 'geol_prim_prop',
 'geol_sec',
 'geol_sec_prop',
 'unconsoldted',
 'igneous',
 'silicsed',
 'carbnatesed',
 'othersed',
 'metamorph',
 'sedvolc',
 'oldrock',
 'claya',
 'clayb',
 'sanda',
 'solum_thickness',
 'ksat',
 'solpawhc',
 'elev_min',
 'elev_max',
 'elev_mean',
 'elev_range',
 'mean_slope_pct',
 'upsdist',
 'strdensity',
 'strahler',
 'elongratio',
 'relief',
 'reliefratio',
 'mrvbf_prop_0',
 'mrvbf_prop_1',
 'mrvbf_prop_2',
 'mrvbf_prop_3',
 'mrvbf_prop_4',
 'mrvbf_prop_5',
 'mrvbf_prop_6',
 'mrvbf_prop_7',
 'mrvbf_prop_8',
 'mrvbf_prop_9',
 'confinement',
 'lc01_extracti',
 'lc03_waterbo',
 'lc04_saltlak',
 'lc05_irrcrop',
 'lc06_irrpast',
 'lc07_irrsuga',
 'lc08_rfcropp',
 'lc09_rfpastu',
 'lc10_rfsugar',
 'lc11_wetlands',
 'lc14_tussclo',
 'lc15_alpineg',
 'lc16_openhum',
 'lc18_opentus',
 'lc19_shrbsca',
 'lc24_shrbden',
 'lc25_shrbope',
 'lc31_forclos',
 'lc32_foropen',
 'lc33_woodope',
 'lc34_woodspa',
 'lc35_urbanar',
 'prop_forested',
 'nvis_grasses_n',
 'nvis_grasses_e',
 'nvis_forests_n',
 'nvis_forests_e',
 'nvis_shrubs_n',
 'nvis_shrubs_e',
 'nvis_woodlands_n',
 'nvis_woodlands_e',
 'nvis_bare_n',
 'nvis_bare_e',
 'nvis_nodata_n',
 'nvis_nodata_e',
 'distupdamw',
 'impound_fac',
 'flow_div_fac',
 'leveebank_fac',
 'infrastruc_fac',
 'settlement_fac',
 'extract_ind_fac',
 'landuse_fac',
 'catchment_di',
 'flow_regime_di',
 'river_di',
 'pop_mean',
 'pop_max',
 'pop_gt_1',
 'pop_gt_10',
 'erosivity',
 'anngro_mega',
 'anngro_meso',
 'anngro_micro',
 'gromega_seas',
 'gromeso_seas',
 'gromicro_seas',
 'npp_ann',
 'npp_1',
 'npp_2',
 'npp_3',
 'npp_4',
 'npp_5',
 'npp_6',
 'npp_7',
 'npp_8',
 'npp_9',
 'npp_10',
 'npp_11',
 'npp_12']

[9]:

len(dataset.static_features)

[9]:

166

[10]:

mrvbf = 'proportion of catchment occupied by classes of MultiResolution Valley Bottom Flatness'
lc01 = 'land cover codes'
nvis = 'vegetation sub-groups'
anngro = 'Average annual growth index value for some plants'
gromega = 'Seasonality of growth index value'
npp = 'net primary productivity'

[11]:

static = dataset.fetch_static_features(stations=stations)
static.shape

[11]:

(222, 166)

[12]:

# EDA(data=static, save=False).heatmap()

[13]:

physical_features = []
soil_features = []
geological_features = []
flow_characteristics = []

static = static.dropna(axis=1)
static.shape

[13]:

(222, 153)

[14]:

coords = dataset.stn_coords()
coords

[14]:
lat long
station_id
912101A -18.643612 139.253052
912105A -18.970833 138.803329
915011A -20.177500 144.524445
917107A -18.133612 144.306671
919003A -16.471390 144.290558
... ... ...
312061 -41.419445 145.670563
314207 -41.250832 146.089996
314213 -40.872223 145.297775
315450 -41.611668 146.130554
318076 -41.494446 147.384171

222 rows × 2 columns

[15]:

dataset.plot_stations(color='area_km2')

Setting upper limit to 46848.34 for color scale

/home/docs/checkouts/readthedocs.org/user_builds/aquafetch/checkouts/latest/aqua_fetch/rr/utils.py:966: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise in a future error of pandas. Value '46848.34' has dtype incompatible with float32, please explicitly cast to a compatible dtype first.
  c[c>ul] = ul
../_images/auto_examples_camels_australia_16_2.svg 
[15]:

<Axes: title={'center': 'CAMELS_AUS Stations (n=222)'}, xlabel='Longitude', ylabel='Latitude'>

[16]:

dataset.plot_catchment(dataset.area().sort_values(ascending=False).index[0], show_outlet=True)

Using attribute 'CatchID' as default for boundary ID mapping.
../_images/auto_examples_camels_australia_17_1.svg 
[16]:

<Axes: xlabel='Longitude', ylabel='Latitude'>

[17]:

dataset.plot_num_observations()

Skipping aet_mm_silo_morton due to no missing values.
Skipping aet_mm_silo_morton_point due to no missing values.
Skipping et_morton_wet_SILO due to no missing values.
Skipping aet_mm_silo_short_crop due to no missing values.
Skipping aet_mm_silo_tall_crop due to no missing values.
Skipping evap_morton_lake_SILO due to no missing values.
All stations for evap_pan_SILO have 16436 observations.
Skipping evap_syn_SILO due to no missing values.
Skipping pcp_mm_awap due to no missing values.
Skipping pcp_mm_silo due to no missing values.
All stations for solrad_wm2_awap have 9131 observations.
Skipping airtemp_C_awap_max due to no missing values.
Skipping airtemp_C_awap_min due to no missing values.
Skipping vp_hpa_awap due to no missing values.
All stations for mslp_SILO have 21184 observations.
Skipping solrad_wm2_silo due to no missing values.
Skipping rh_%_silo_tmax due to no missing values.
Skipping rh_%_silo_tmin due to no missing values.
Skipping airtemp_C_silo_max due to no missing values.
Skipping airtemp_C_silo_min due to no missing values.
Skipping vp_deficit_SILO due to no missing values.
Skipping vp_hpa_silo due to no missing values.
Skipping airtemp_C_mean_silo due to no missing values.
Skipping airtemp_C_mean_awap due to no missing values.
../_images/auto_examples_camels_australia_18_1.svg 
[17]:

<Axes: xlabel='Number of observations', ylabel='Number of stations'>

[18]:

_, ax = plt.subplots()
for idx, period in enumerate([("19810101", "19901231"), ("19910101", "20001231"), ("20010101", "20101231")]):
    start, end = period
    ax = dataset.plot_num_observations(
        dynamic_features=['q_cms_obs'],
        ax=ax,
        start=start, end=end, show=False)
ax.lines[idx].set_label(f'{start} to {end}')
assert isinstance(ax, plt.Axes)
ax.legend()

[18]:

<matplotlib.legend.Legend at 0x734eea18e8d0>
../_images/auto_examples_camels_australia_19_1.svg 
[19]:

lat = coords['lat'].astype(float).values.reshape(-1,)
long = coords['long'].astype(float).values.reshape(-1,)

[20]:

idx = 0
ax_num = 0

fig, axes = plt.subplots(5, 5, figsize=(15, 12))
axes = axes.flatten()

while ax_num < 25:

    val = static.iloc[:, idx]
    idx += 1

    try:
        c = val.astype(float).values.reshape(-1,)

        en = 222
        ax = axes[ax_num]
        ax, sc = scatter(long[0:en], lat[0:en], c=c[0:en], cmap="hot", show=False, ax=ax)

        process_cbar(ax, sc, border=False, title=val.name, #title_kws ={"fontsize": 14}
                    )
        ax_num += 1
    except ValueError:
        continue

plt.tight_layout()
plt.show()
print(idx)
../_images/auto_examples_camels_australia_21_0.svg 
33

[21]:

idx = 32
ax_num = 0

fig, axes = plt.subplots(5, 5, figsize=(15, 12))
axes = axes.flatten()

while ax_num < 25:

    val = static.iloc[:, idx]
    idx += 1

    try:
        c = val.astype(float).values.reshape(-1,)

        en = 222
        ax = axes[ax_num]
        ax, sc = scatter(long[0:en], lat[0:en], c=c[0:en], cmap="hot", show=False, ax=ax)

        process_cbar(ax, sc, border=False, title=val.name, #title_kws ={"fontsize": 14}
                    )
        ax_num += 1
    except ValueError:
        continue

plt.tight_layout()
plt.show()
print(idx)
../_images/auto_examples_camels_australia_22_0.svg 
59

[22]:

idx = 59
ax_num = 0

fig, axes = plt.subplots(5, 5, figsize=(15, 12))
axes = axes.flatten()

while ax_num < 25:

    val = static.iloc[:, idx]
    idx += 1

    try:
        c = val.astype(float).values.reshape(-1,)

        en = 222
        ax = axes[ax_num]
        ax, sc = scatter(long[0:en], lat[0:en], c=c[0:en], cmap="hot", show=False, ax=ax)

        process_cbar(ax, sc, border=False, title=val.name, #title_kws ={"fontsize": 14}
                    )
        ax_num += 1
    except ValueError:
        continue

plt.tight_layout()
plt.show()
print(idx)
../_images/auto_examples_camels_australia_23_0.svg 
84

Dyanmic Features

[23]:

dataset.dynamic_features

[23]:

['q_cms_obs',
 'streamflow_MLd_inclInfilled',
 'q_mm_obs',
 'aet_mm_silo_morton',
 'aet_mm_silo_morton_point',
 'et_morton_wet_SILO',
 'aet_mm_silo_short_crop',
 'aet_mm_silo_tall_crop',
 'evap_morton_lake_SILO',
 'evap_pan_SILO',
 'evap_syn_SILO',
 'pcp_mm_awap',
 'pcp_mm_silo',
 'precipitation_var_AWAP',
 'solrad_wm2_awap',
 'airtemp_C_awap_max',
 'airtemp_C_awap_min',
 'vp_hpa_awap',
 'mslp_SILO',
 'solrad_wm2_silo',
 'rh_%_silo_tmax',
 'rh_%_silo_tmin',
 'airtemp_C_silo_max',
 'airtemp_C_silo_min',
 'vp_deficit_SILO',
 'vp_hpa_silo',
 'airtemp_C_mean_silo',
 'airtemp_C_mean_awap']

Streamflow

[24]:

streamflow = dataset.q_mm()

streamflow.shape

[24]:

(23376, 222)

[25]:

streamflow

[25]:
912101A 912105A 915011A 917107A 919003A 919201A 919309A 922101B 925001A 926002A ... 304499 305202 307473 308145 308799 312061 314207 314213 315450 318076
time
1951-01-01 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN 0.931382 NaN NaN NaN 0.571042
1951-01-02 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN 0.881380 NaN NaN NaN 0.387082
1951-01-03 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN 0.799637 NaN NaN NaN 0.343161
1951-01-04 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN 0.832386 NaN NaN NaN 0.313503
1951-01-05 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... NaN NaN NaN NaN NaN 0.897625 NaN NaN NaN 0.309466
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
2014-12-27 0.033409 0.032417 0.000000 0.010655 0.003816 0.0 0.0 0.0 0.036582 0.300530 ... 0.505348 0.447075 2.489122 1.391061 1.407268 0.547383 0.415754 0.093841 0.944415 0.243313
2014-12-28 0.037260 0.028840 0.000000 0.010369 0.002212 0.0 0.0 0.0 0.036308 0.293852 ... 0.506314 0.126054 2.198925 1.255558 1.330847 0.524695 0.386941 0.084121 0.881395 0.219409
2014-12-29 0.037653 0.027334 0.000000 0.010321 0.001903 0.0 0.0 0.0 0.036154 0.292314 ... 0.482631 0.057792 1.557511 1.061568 1.150340 0.501252 0.378870 0.075581 0.802402 0.207761
2014-12-30 0.032432 0.027785 0.000000 0.028903 0.001662 0.0 0.0 0.0 0.034776 0.293543 ... 0.548162 0.041354 1.354081 12.621201 17.800033 0.731634 0.387640 0.078101 7.128174 0.205275
2014-12-31 0.032191 0.169300 0.003321 0.042891 0.002262 0.0 0.0 0.0 0.034613 0.291518 ... 2.707964 0.135351 11.706610 10.158581 9.200824 1.745331 1.042905 0.134921 6.993985 0.223863

23376 rows × 222 columns

[26]:

# EDA(data=streamflow, save=False).heatmap()

[27]:

fig, axes = plt.subplots(7, 7, figsize=(10, 10), sharey="all")

for idx, ax in enumerate(axes.flat):

    hist(streamflow.iloc[:, idx].values.reshape(-1,),
         bins=20,
         ax=ax,
         show=False
        )

plt.show()
../_images/auto_examples_camels_australia_30_0.svg 
[28]:

_ = hist(streamflow.skew().values.reshape(-1,), bins=50)
../_images/auto_examples_camels_australia_31_0.svg 
[29]:

_, dynamic = dataset.fetch(stations=1, as_dataframe=True)
print(len(dynamic))
df = dynamic.popitem()[1] # the key in dynamic is the station name and value is stn dynamic data as DataFrame
df.shape

1

[29]:

(23376, 28)

[30]:

df

[30]:
dynamic_features q_cms_obs streamflow_MLd_inclInfilled q_mm_obs aet_mm_silo_morton aet_mm_silo_morton_point et_morton_wet_SILO aet_mm_silo_short_crop aet_mm_silo_tall_crop evap_morton_lake_SILO evap_pan_SILO ... mslp_SILO solrad_wm2_silo rh_%_silo_tmax rh_%_silo_tmin airtemp_C_silo_max airtemp_C_silo_min vp_deficit_SILO vp_hpa_silo airtemp_C_mean_silo airtemp_C_mean_awap
time
1951-01-01 NaN NaN NaN 5.860366 8.241464 7.040244 5.940244 7.056707 7.300610 NaN ... NaN 27.824390 47.578049 94.041466 30.325001 18.925001 16.125610 20.568903 24.625000 24.633965
1951-01-02 NaN NaN NaN 5.100000 6.053658 5.579878 4.456098 5.114025 5.756098 NaN ... NaN 20.976219 63.874390 94.298782 27.664635 21.156099 9.997561 23.678049 24.410366 24.555000
1951-01-03 NaN NaN NaN 6.300610 7.353659 6.840244 5.414024 6.253659 7.056707 NaN ... NaN 27.343903 55.140244 99.269516 28.398170 18.639025 12.189025 21.335976 23.518597 23.970032
1951-01-04 NaN NaN NaN 6.922561 6.922561 6.922561 5.096342 5.695732 7.141463 NaN ... NaN 27.210976 60.408535 100.000000 28.880487 17.879269 9.868293 24.020123 23.379879 23.637470
1951-01-05 NaN NaN NaN 2.076829 2.911585 2.515244 2.373780 2.929268 2.515854 NaN ... NaN 8.391464 65.278664 100.000000 29.021952 20.030489 9.010976 26.176830 24.526220 24.713627
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
2014-12-27 0.007206 0.622850 0.013366 6.317073 8.496952 7.380488 6.056707 7.096342 7.657317 9.412195 ... 1011.100037 27.117683 53.028049 89.912811 31.596342 22.590855 16.224390 24.638416 27.093597 27.322166
2014-12-28 0.005876 0.507893 0.010899 6.442073 8.035975 7.262805 5.796342 6.795732 7.480488 9.356708 ... 1011.200012 25.964025 55.365246 95.635368 32.095734 22.745123 15.404268 26.454878 27.420429 27.481434
2014-12-29 0.004511 0.389903 0.008367 4.660976 6.776219 5.696951 4.696342 5.696342 5.856708 7.829878 ... 1010.266479 19.011585 57.076832 93.168297 31.687195 23.301830 14.799391 26.659147 27.494513 28.037073
2014-12-30 0.003424 0.295926 0.006350 4.978049 7.873781 6.396341 5.395732 6.575610 6.556098 2.133537 ... 1010.299988 21.192684 52.496342 90.436584 33.655487 24.240244 18.373171 27.373781 28.947865 28.482439
2014-12-31 0.002834 0.244940 0.005256 6.726830 7.535366 7.139634 5.555488 6.410976 7.321951 8.815854 ... 1009.200012 24.302439 57.398781 100.000000 33.335365 22.997561 14.871342 29.420122 28.166462 27.534023

23376 rows × 28 columns

[31]:

# get name of all stations as list
stns = dataset.stations()
len(stns)

[31]:

222

get data of 10 % of stations as dataframe

[32]:

_, dynamic = dataset.fetch(0.1, as_dataframe=True)
len(dynamic)  # remember this is a dictionary of dataframes

[32]:

22

The keys in dynamic dictionary are names of stations

[33]:

dynamic.keys()

[33]:

dict_keys(['206018', '401012', '403209A', '401208', 'G8140040', '405274', '405226', '405263', '136202D', 'A0020101', '225110A', '105102A', '616002', '226222', '145018A', '308145', '308799', '230210', '102101A', '613002', 'G9070142', '616065'])

get data by station id

[34]:

dynamic = dataset.fetch(stations='224214A', as_dataframe=True)[1]
dynamic['224214A'].shape

[34]:

(23376, 28)

[35]:

dynamic['224214A']

[35]:
dynamic_features q_cms_obs streamflow_MLd_inclInfilled q_mm_obs aet_mm_silo_morton aet_mm_silo_morton_point et_morton_wet_SILO aet_mm_silo_short_crop aet_mm_silo_tall_crop evap_morton_lake_SILO evap_pan_SILO ... mslp_SILO solrad_wm2_silo rh_%_silo_tmax rh_%_silo_tmin airtemp_C_silo_max airtemp_C_silo_min vp_deficit_SILO vp_hpa_silo airtemp_C_mean_silo airtemp_C_mean_awap
time
1951-01-01 NaN NaN NaN 5.854414 7.344525 6.591062 5.199777 6.248827 6.805475 NaN ... NaN 31.693129 38.657600 99.223740 22.589106 7.179832 11.043966 10.625475 14.884469 14.630333
1951-01-02 NaN NaN NaN 3.720838 5.970670 4.841676 4.888380 6.280447 4.949106 NaN ... NaN 22.082012 34.147209 100.000000 28.960335 6.874693 15.285140 13.665586 17.917515 18.440363
1951-01-03 NaN NaN NaN 2.758045 7.028100 4.901564 5.288883 6.920726 5.006983 NaN ... NaN 21.208046 31.228270 100.000000 30.393967 10.257375 18.920113 13.559162 20.325670 21.916115
1951-01-04 NaN NaN NaN 1.872346 6.184246 4.023073 4.378045 5.689497 4.110280 NaN ... NaN 17.139608 38.984528 88.440506 26.720001 13.467207 15.200336 13.682403 20.093605 21.333338
1951-01-05 NaN NaN NaN 5.242626 7.473408 6.367821 5.259832 6.279441 6.540224 NaN ... NaN 28.298380 46.126926 88.514526 24.212961 13.754525 11.920447 13.948156 18.983744 19.464199
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
2014-12-27 0.963259 83.254898 0.188958 5.263129 6.776760 6.019442 4.915475 6.007598 6.219441 4.861006 ... 1019.592529 29.931396 36.573353 100.000000 22.333185 5.147654 10.853352 9.882738 13.740419 13.589680
2014-12-28 0.884752 76.469498 0.173558 5.425922 6.773743 6.106760 5.046369 6.066201 6.275363 5.650447 ... 1016.386047 28.012011 42.295364 100.000000 25.113855 9.529721 11.808492 13.501006 17.321789 17.031651
2014-12-29 0.840371 72.633598 0.164852 4.498380 7.134861 5.817374 5.214023 6.422570 5.948380 5.629944 ... 1003.352966 25.781174 40.154190 99.995697 26.736649 11.710727 14.221453 14.024302 19.223688 19.246988
2014-12-30 0.799492 69.100403 0.156833 4.197989 7.002067 5.593017 4.859721 5.987654 5.782179 5.312737 ... 1005.831726 28.189497 35.938995 87.377266 20.986034 7.259050 11.269665 8.918436 14.122542 13.945707
2014-12-31 0.764555 66.080803 0.149979 5.004805 7.923408 6.462682 5.426425 6.696928 6.690391 6.193520 ... 1014.303040 31.586760 32.787151 87.620003 22.560167 7.305643 12.661620 9.018715 14.932905 14.372753

23376 rows × 28 columns

get names of available dynamic features

[36]:

dataset.dynamic_features
# get only selected dynamic features
dynamic = dataset.fetch(1, as_dataframe=True,
dynamic_features=['airtemp_C_awap_max', 'pcp_mm_awap', 'aet_mm_silo_morton', 'q_cms_obs'])[1]
print(type(dynamic))
data = dynamic.popitem()[1]
data.shape

<class 'dict'>

[36]:

(23376, 4)

[37]:

data

[37]:
dynamic_features airtemp_C_awap_max pcp_mm_awap aet_mm_silo_morton q_cms_obs
time
1951-01-01 31.159369 0.000000 5.704088 NaN
1951-01-02 30.433960 0.000000 0.322263 NaN
1951-01-03 32.629593 0.000000 0.032299 NaN
1951-01-04 26.130142 1.494817 3.616058 NaN
1951-01-05 31.403191 0.785401 5.179489 NaN
... ... ... ... ...
2014-12-27 29.071814 0.000000 4.184708 0.0
2014-12-28 32.128502 0.000000 4.520219 0.0
2014-12-29 28.036566 0.000000 0.143504 0.0
2014-12-30 26.915073 0.000000 3.154416 0.0
2014-12-31 29.440561 0.000000 3.910219 0.0

23376 rows × 4 columns

get data of 10 random stations

[38]:

_, dynamic = dataset.fetch(10, as_dataframe=True)
len(dynamic)  # remember this is a dictioanry of dataframes

[38]:

10

static data is always a pandas DataFrame while dynamic is a dictionary of dataframes with keys as station names.

[39]:

static, dynamic = dataset.fetch(stations='224214A', static_features="all", as_dataframe=True)
static.shape, dynamic['224214A'].shape

[39]:

((1, 166), (23376, 28))

[40]:

static

[40]:
station_name drainage_division river_region notes lat long lat_centroid long_centroid map_zone area_km2 ... npp_3 npp_4 npp_5 npp_6 npp_7 npp_8 npp_9 npp_10 npp_11 npp_12
station_id
224214A Wentworth River at Tabberabbera (224214A) South East Coast (Victoria) MITCHELL-THOMSON RIVERS No notes -37.494444 147.393333 -37.372603 147.422449 55 440.6 ... 0.639791 0.552442 0.399512 0.304924 0.365059 0.528615 0.694625 0.868412 0.842424 0.76443

1 rows × 166 columns

[41]:

dynamic.keys()

[41]:

dict_keys(['224214A'])

get dynamic data data of all stations as xarray dataset

[42]:

static, dynamic = dataset.fetch(static_features="all")
static

[42]:
station_name drainage_division river_region notes lat long lat_centroid long_centroid map_zone area_km2 ... npp_3 npp_4 npp_5 npp_6 npp_7 npp_8 npp_9 npp_10 npp_11 npp_12
station_id
912101A Gregory River at Gregory Downs (912101A) Carpentaria Coast NICHOLSON-LEICHHARDT RIVERS No notes -18.643611 139.253056 -19.303638 138.553210 54 12577.3 ... 0.079221 0.028471 0.033520 0.022459 0.016778 0.002405 0.004844 0.016939 0.035641 0.046875
912105A Gregory River at Riversleigh No. 2 (912105A) Carpentaria Coast NICHOLSON-LEICHHARDT RIVERS No notes -18.970833 138.803333 -19.333672 138.508911 54 11392.9 ... 0.076032 0.027316 0.032327 0.021844 0.017243 0.002405 0.004756 0.016132 0.033410 0.044226
915011A Porcupine Creek at Mt Emu Plains (915011A) Carpentaria Coast FLINDERS-NORMAN RIVERS No notes -20.177500 144.524444 -20.054245 144.483641 55 565.3 ... 0.228245 0.178230 0.192825 0.137060 0.123925 0.084574 0.037268 0.100204 0.153955 0.172525
917107A Elizabeth Creek at Mount Surprise (917107A) Carpentaria Coast FLINDERS-NORMAN RIVERS Uncertain catchment boundary. The SRTM-derive... -18.133611 144.306667 -18.152298 144.524350 55 458.3 ... 0.255098 0.169087 0.163391 0.074736 0.064303 0.034086 0.014866 0.074020 0.154662 0.201448
919003A Mitchell River at O.K. Bridge (919003A) Carpentaria Coast MITCHELL-COLEMAN RIVERS (QLD) No notes -16.471389 144.290556 -16.646693 144.878729 55 7731.7 ... 0.322416 0.227765 0.189666 0.104363 0.092505 0.063088 0.036525 0.098279 0.184531 0.238028
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
312061 Hellyer River at Guilford Junction (312061) Tasmania ARTHUR RIVER No notes -41.419444 145.670556 -41.467375 145.651407 55 99.4 ... 0.451088 0.299095 0.176474 0.100124 0.135071 0.223450 0.332237 0.487613 0.569115 0.609835
314207 Leven River at at Bannons Bridge (314207) Tasmania SMITHTON-BURNIE COAST No notes -41.250833 146.090000 -41.403886 145.927003 55 499.6 ... 0.545703 0.390217 0.238852 0.151707 0.200632 0.331666 0.485940 0.669314 0.727175 0.738653
314213 Black River at South Forest (314213) Tasmania SMITHTON-BURNIE COAST No notes -40.872222 145.297778 -40.997264 145.345798 55 319.0 ... 0.593617 0.447931 0.278360 0.183765 0.244593 0.391560 0.567227 0.767863 0.813744 0.791578
315450 Forth River at U/S Lemonthyme (315450) Tasmania FORTH RIVER No notes -41.611667 146.130556 -41.733571 146.045131 55 309.6 ... 0.463656 0.309041 0.183652 0.094034 0.137295 0.238779 0.363403 0.520559 0.603897 0.662411
318076 North Esk River at Ballroom (318076) Tasmania TAMAR RIVER No notes -41.494444 147.384167 -41.458693 147.537101 55 374.2 ... 0.610545 0.511115 0.327000 0.218916 0.285644 0.461620 0.661953 0.848524 0.833786 0.871770

222 rows × 166 columns

[43]:

dynamic

[43]:

<xarray.Dataset> Size: 581MB
Dimensions:           (time: 23376, dynamic_features: 28)
Coordinates:
  * time              (time) datetime64[ns] 187kB 1951-01-01 ... 2014-12-31
  * dynamic_features  (dynamic_features) <U27 3kB 'q_cms_obs' ... 'airtemp_C_...
Data variables: (12/222)
    912101A           (time, dynamic_features) float32 3MB ...
    912105A           (time, dynamic_features) float32 3MB ...
    915011A           (time, dynamic_features) float32 3MB ...
    917107A           (time, dynamic_features) float32 3MB ...
    919003A           (time, dynamic_features) float32 3MB ...
    919201A           (time, dynamic_features) float32 3MB ...
    ...                ...
    308799            (time, dynamic_features) float32 3MB ...
    312061            (time, dynamic_features) float32 3MB ...
    314207            (time, dynamic_features) float32 3MB ...
    314213            (time, dynamic_features) float32 3MB ...
    315450            (time, dynamic_features) float32 3MB ...
    318076            (time, dynamic_features) float32 3MB ...