s3 storage — r6_storage

A s3 storage with flexible file format (default rds). The data format defines the data chunks per file. The data is cached locally. This local cache can be used as local storage. For performance enhancement it is recommended after acquiring all needed data from s3 to use the cache as local storage. See storage_local_rds() for more Informations.

Usage

storage_s3_rds(
  name,
  format,
  bucket,
  prefix = NULL,
  region = NULL,
  read.only = TRUE
)

Arguments

name: name of the store
format: data format of the store
bucket: name in aws s3
prefix: in aws s3
region: aws region
read.only: read only store. disables put, upload

Value

R6 class object of storage_s3

Fields

name: name of the store
format: data format of the store
bucket: s3 bucket containing the store
region: aws region of the bucket
prefix: of the s3 object keys. see Object Key and Metadata
path: local root of the store
data_path: local root of all chunks
data_s3: s3 root key of all chunks
content_path: local path to the rds file containing statistics of store content
content_s3: s3 object key to the rds file containing statistics of store content
columns_path: local path to the rds file containing the exact column types of the store content
columns_s3: s3 object key to the rds file containing the exact column types of the store content
meta_path: local root of all meta data files
meta_s3: s3 root key of all meta data files
read.only: flag for read.only usage of store. Default TRUE
ext: file extension for chunks. Default "rds"
read_function: function(file) for reading chunks from disk. Default base::readRDS()
write_function: function(object, file) for writing chunks to disk. Default base::saveRDS()

Methods

$get(filter=NULL, ...) get data from the store. The name of the arguments depend on the format. The filter argument is applied to each chunk.

$download(...) downloads data from s3 to the local cache. The dots arguments are used to filter the output of list_chunks(). Only the matching rows will be downloaded.

$put(data) puts the data into the store. Stops if store is read only

$upload() uploads content, meta data and all new and changed chunks to the s3 storage. For big additions to the store the recommend way is to use a local storage to modify the cache and use this function to apply the change.

$get_content() returns a tibble with the amount of data points per chunk per series

$list_chunks() get list of all chunks in s3 and local

$get_meta(key=NULL) get meta data. If key is omitted returns all the content of all files in a named list of tibbles, with the file name without extension as name. If key is supplied as argument only the list contains only the specified key.

$put_meta(...) puts meta data into the store. the name of the argument is used as file name and the value as data.

$get_local_storage() returns a storage to work with the cached data like a local storage

$destroy(confirmation) removes all files under path from the file system if "DELETE" is supplied as confirmation

Authentication

See the documentation of aws.signature for ways to provide the necessary informations. The simplest way is to use environment variables defined in a .Renviron file in the root directory of a RStudio Project:

AWS_ACCESS_KEY_ID = "xxxxxxxxxxxxxxxxxx"

AWS_SECRET_ACCESS_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

AWS_DEFAULT_REGION = "eu-central-1"

Examples

## init store, creates directory if necessary
format <- rOstluft::format_rolf()
store <- rOstluft::storage_s3_rds("s3_example", format, "rostluft", prefix = "aqmet")
#> store s3_example initialized under 'C:\Users\tom\AppData\Local/rOstluft/s3_example'

## get all data min30 for 2011 and 2012
store$get(site = "Zch_Stampfenbachstrasse", interval = "min30", year = 2011:2012)
#> # A tibble: 523,318 × 6
#>    starttime           site                    parameter interval unit    value
#>    <dttm>              <fct>                   <fct>     <fct>    <fct>   <dbl>
#>  1 2011-01-01 00:00:00 Zch_Stampfenbachstrasse CO        min30    ppm    0.525 
#>  2 2011-01-01 00:00:00 Zch_Stampfenbachstrasse Hr        min30    %Hr   91.9   
#>  3 2011-01-01 00:00:00 Zch_Stampfenbachstrasse NO2       min30    ppb   22.0   
#>  4 2011-01-01 00:00:00 Zch_Stampfenbachstrasse NO        min30    ppb   21.0   
#>  5 2011-01-01 00:00:00 Zch_Stampfenbachstrasse NOx       min30    ppb   43.0   
#>  6 2011-01-01 00:00:00 Zch_Stampfenbachstrasse O3        min30    ppb    1.10  
#>  7 2011-01-01 00:00:00 Zch_Stampfenbachstrasse PM10      min30    µg/m3 84.0   
#>  8 2011-01-01 00:00:00 Zch_Stampfenbachstrasse RainDur   min30    min    0     
#>  9 2011-01-01 00:00:00 Zch_Stampfenbachstrasse SO2       min30    ppb    1.90  
#> 10 2011-01-01 00:00:00 Zch_Stampfenbachstrasse StrGlo    min30    W/m2   0.0104
#> # ℹ 523,308 more rows

## get only data for O3
store$get(year = 2011:2012, site = "Zch_Stampfenbachstrasse", interval = "min30",
          filter = parameter == "O3")
#> # A tibble: 69,992 × 6
#>    starttime           site                    parameter interval unit  value
#>    <dttm>              <fct>                   <fct>     <fct>    <fct> <dbl>
#>  1 2011-01-01 00:00:00 Zch_Stampfenbachstrasse O3        min30    ppb    1.10
#>  2 2011-01-01 00:00:00 Zch_Stampfenbachstrasse O3        min30    µg/m3  2.19
#>  3 2011-01-01 00:30:00 Zch_Stampfenbachstrasse O3        min30    ppb    1.05
#>  4 2011-01-01 00:30:00 Zch_Stampfenbachstrasse O3        min30    µg/m3  2.10
#>  5 2011-01-01 01:00:00 Zch_Stampfenbachstrasse O3        min30    ppb    1.05
#>  6 2011-01-01 01:00:00 Zch_Stampfenbachstrasse O3        min30    µg/m3  2.10
#>  7 2011-01-01 01:30:00 Zch_Stampfenbachstrasse O3        min30    ppb    1.00
#>  8 2011-01-01 01:30:00 Zch_Stampfenbachstrasse O3        min30    µg/m3  2.00
#>  9 2011-01-01 02:00:00 Zch_Stampfenbachstrasse O3        min30    ppb    1.02
#> 10 2011-01-01 02:00:00 Zch_Stampfenbachstrasse O3        min30    µg/m3  2.04
#> # ℹ 69,982 more rows

## get NOx data from multiple stations
store$get(site = c("Zch_Stampfenbachstrasse", "Zch_Rosengartenstrasse"), interval = "min30",
          year = 2014, filter = parameter %in% c("NOx", "NO", "NO2"))
#> # A tibble: 104,547 × 6
#>    starttime           site                   parameter interval unit  value
#>    <dttm>              <fct>                  <fct>     <fct>    <fct> <dbl>
#>  1 2014-01-01 00:00:00 Zch_Rosengartenstrasse NO2       min30    ppb    18.6
#>  2 2014-01-01 00:00:00 Zch_Rosengartenstrasse NO        min30    ppb    42.8
#>  3 2014-01-01 00:00:00 Zch_Rosengartenstrasse NOx       min30    ppb    61.4
#>  4 2014-01-01 00:30:00 Zch_Rosengartenstrasse NO2       min30    ppb    21.3
#>  5 2014-01-01 00:30:00 Zch_Rosengartenstrasse NO        min30    ppb    55.9
#>  6 2014-01-01 00:30:00 Zch_Rosengartenstrasse NOx       min30    ppb    77.2
#>  7 2014-01-01 01:00:00 Zch_Rosengartenstrasse NO2       min30    ppb    28.2
#>  8 2014-01-01 01:00:00 Zch_Rosengartenstrasse NO        min30    ppb    88.1
#>  9 2014-01-01 01:00:00 Zch_Rosengartenstrasse NOx       min30    ppb   116. 
#> 10 2014-01-01 01:30:00 Zch_Rosengartenstrasse NO2       min30    ppb    39.8
#> # ℹ 104,537 more rows

## get n data points grouped by intervall, station, parameter, year in the store
store$get_content()
#> # A tibble: 220,692 × 6
#>     year interval site          parameter unit      n
#>    <dbl> <fct>    <fct>         <fct>     <fct> <int>
#>  1  2019 y1       8400_0040     NO2       µg/m3     1
#>  2  2022 y1       9658_0020     NH3       µg/m3     1
#>  3  2022 y1       8880_0030     NH3       µg/m3     1
#>  4  2022 y1       8535_0010     NH3       µg/m3     1
#>  5  2022 y1       WeF_Weid      NH3       µg/m3     1
#>  6  2022 y1       Tan_NABEL     NH3       µg/m3     1
#>  7  2022 y1       StE_Rödelbach NO2       µg/m3     1
#>  8  2022 y1       StE_Rödelbach NH3       µg/m3     1
#>  9  2022 y1       SNs_Wellrüti  NH3       µg/m3     1
#> 10  2022 y1       Hae_Buo       NH3       µg/m3     1
#> # ℹ 220,682 more rows

## get list of all chunks, show only local files
dplyr::filter(store$list_chunks(), !is.na(local.path))
#> # A tibble: 4 × 11
#>   chunk_name         interval site  year  s3.key s3.lastmodified s3.etag s3.size
#>   <fs::path>         <chr>    <chr> <chr> <chr>  <chr>           <chr>   <fs::b>
#> 1 …N0cmFzc2XCuzIwMTQ min30    Zch_… 2014  aqmet… 2023-04-26T18:… f9e721… 808.46K
#> 2 …zdHJhc3NlwrsyMDE0 min30    Zch_… 2014  aqmet… 2023-04-26T18:… aa917f…   1.22M
#> 3 …zdHJhc3NlwrsyMDEx min30    Zch_… 2011  aqmet… 2023-04-26T18:… 9a85a2…   1.22M
#> 4 …zdHJhc3NlwrsyMDEy min30    Zch_… 2012  aqmet… 2023-04-26T18:… 87e441…   1.22M
#> # ℹ 3 more variables: local.path <fs::path>, local.modification_time <dttm>,
#> #   local.size <fs::bytes>

## download all data for site Zch_Rosengartenstrasse before 2016
store$download(site == "Zch_Rosengartenstrasse", year < 2016)
#> list()

## now there should be 2 more local files
dplyr::filter(store$list_chunks(), !is.na(local.path))
#> # A tibble: 21 × 11
#>    chunk_name        interval site  year  s3.key s3.lastmodified s3.etag s3.size
#>    <fs::path>        <chr>    <chr> <chr> <chr>  <chr>           <chr>   <fs::b>
#>  1 …0cmFzc2XCuzIwMTM d1       Zch_… 2013  aqmet… 2023-04-26T18:… e06ac3…  19.84K
#>  2 …0cmFzc2XCuzIwMTQ d1       Zch_… 2014  aqmet… 2023-04-26T18:… 583d40…  76.76K
#>  3 …0cmFzc2XCuzIwMTU d1       Zch_… 2015  aqmet… 2023-04-26T18:… 55ebdf…  86.04K
#>  4 …0cmFzc2XCuzIwMTM h1       Zch_… 2013  aqmet… 2023-04-26T18:… 6a2647… 112.38K
#>  5 …0cmFzc2XCuzIwMTQ h1       Zch_… 2014  aqmet… 2023-04-26T18:… f89678… 447.54K
#>  6 …0cmFzc2XCuzIwMTU h1       Zch_… 2015  aqmet… 2023-04-26T18:… fafb4a… 567.64K
#>  7 …RyYXNzZcK7MjAxMw h8gl     Zch_… 2013  aqmet… 2023-04-26T18:… a7938a…  23.56K
#>  8 …RyYXNzZcK7MjAxNA h8gl     Zch_… 2014  aqmet… 2023-04-26T18:… 2cb961…  93.14K
#>  9 …RyYXNzZcK7MjAxNQ h8gl     Zch_… 2015  aqmet… 2023-04-26T18:… e05501…   92.2K
#> 10 …0cmFzc2XCuzIwMTM m1       Zch_… 2013  aqmet… 2023-04-26T18:… 287790…   2.08K
#> # ℹ 11 more rows
#> # ℹ 3 more variables: local.path <fs::path>, local.modification_time <dttm>,
#> #   local.size <fs::bytes>

## get all meta data
store$get_meta()
#> $all
#> # A tibble: 2,345 × 12
#>    site    site_short site_long      x      y  masl parameter_original parameter
#>    <chr>   <chr>      <chr>      <dbl>  <dbl> <dbl> <chr>              <chr>    
#>  1 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 Ox                 Ox       
#>  2 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 SO2                SO2      
#>  3 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 O3                 O3       
#>  4 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 NOx                NOx      
#>  5 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 NO                 NO       
#>  6 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 NO2                NO2      
#>  7 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 PM10               PM10     
#>  8 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 T                  T        
#>  9 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 Hr                 Hr       
#> 10 Arb_St… ARBS       Arbon St… 2.75e6 1.26e6   407 StrGlo             StrGlo   
#> # ℹ 2,335 more rows
#> # ℹ 4 more variables: unit <chr>, interval <chr>, timezone_original <chr>,
#> #   source <chr>
#> 
#> $ecmwf
#> # A tibble: 96 × 14
#>    site_short site          site_long   lon   lat masl  parameter_original unit 
#>    <chr>      <chr>         <lgl>     <dbl> <dbl> <lgl> <chr>              <chr>
#>  1 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    t2m                K    
#>  2 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    swvl1              m3/m3
#>  3 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    sp                 hPa  
#>  4 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    u10                m/s  
#>  5 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    v10                m/s  
#>  6 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    skt                K    
#>  7 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    ishf               W/m2 
#>  8 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    tp                 m    
#>  9 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    tco3               kg/m2
#> 10 ZH_EraI    Zürich - Era… NA         8.25  47.2 NA    sund               s    
#> # ℹ 86 more rows
#> # ℹ 6 more variables: interval <chr>, timezone_original <chr>, source <chr>,
#> #   x <dbl>, y <dbl>, parameter <chr>
#> 
#> $ethz
#> # A tibble: 28 × 14
#>    parameter_original unit     site_short site_long site         lon   lat  masl
#>    <chr>              <chr>    <chr>      <lgl>     <chr>      <dbl> <dbl> <dbl>
#>  1 p_air              hPa      ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#>  2 T_air              °C       ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#>  3 T_dew              °C       ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#>  4 RH                 %        ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#>  5 illum              log10lux ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#>  6 rain               mm       ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#>  7 wspd_vec           m/s      ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#>  8 wspd_scl           m/s      ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#>  9 wdir               °        ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#> 10 wspd_pk            m/s      ETHZ_CHN   NA        ETHZ_CHN-…  8.55  47.4   510
#> # ℹ 18 more rows
#> # ℹ 6 more variables: interval <chr>, timezone_original <chr>, source <chr>,
#> #   x <dbl>, y <dbl>, parameter <chr>
#> 
#> $hysplit
#> # A tibble: 2 × 12
#>   site_short      site  site_long lon   lat   masl  parameter_original parameter
#>   <chr>           <chr> <chr>     <lgl> <lgl> <lgl> <chr>              <chr>    
#> 1 ZH-Kaserne-hys… ZH_K… Zürich_K… NA    NA    NA    hysplit_cluster    hysplit_…
#> 2 SG-BP-hysplit   SG_B… St. Gall… NA    NA    NA    hysplit_cluster    hysplit_…
#> # ℹ 4 more variables: unit <lgl>, interval <chr>, timezone_original <chr>,
#> #   source <chr>
#> 
#> $hysplit_cluster_trajectories
#> # A tibble: 1,164 × 7
#> # Groups:   cluster [6]
#>    cluster hour.inc default                  lat   lon date                site 
#>      <dbl>    <dbl> <fct>                  <dbl> <dbl> <dttm>              <fct>
#>  1       1      -96 01 January 1980 to 31…  49.3 -16.6 2000-12-20 06:08:12 SG_B…
#>  2       1      -95 01 January 1980 to 31…  49.3 -16.3 2000-12-20 06:08:12 SG_B…
#>  3       1      -94 01 January 1980 to 31…  49.2 -16.0 2000-12-20 06:08:12 SG_B…
#>  4       1      -93 01 January 1980 to 31…  49.2 -15.6 2000-12-20 06:08:12 SG_B…
#>  5       1      -92 01 January 1980 to 31…  49.1 -15.3 2000-12-20 06:08:12 SG_B…
#>  6       1      -91 01 January 1980 to 31…  49.1 -15.0 2000-12-20 06:08:12 SG_B…
#>  7       1      -90 01 January 1980 to 31…  49.0 -14.6 2000-12-20 06:08:12 SG_B…
#>  8       1      -89 01 January 1980 to 31…  49.0 -14.3 2000-12-20 06:08:12 SG_B…
#>  9       1      -88 01 January 1980 to 31…  48.9 -14.0 2000-12-20 06:08:12 SG_B…
#> 10       1      -87 01 January 1980 to 31…  48.9 -13.6 2000-12-20 06:08:12 SG_B…
#> # ℹ 1,154 more rows
#> 
#> $meteoschweiz
#> # A tibble: 279 × 15
#>    site_short site    Länge Breite      x      y  masl parameter_original source
#>    <chr>      <chr>   <chr> <chr>   <dbl>  <dbl> <dbl> <chr>              <chr> 
#>  1 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 tre200s0           Meteo…
#>  2 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 ure200s0           Meteo…
#>  3 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 prestas0           Meteo…
#>  4 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 prestas0           Meteo…
#>  5 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 fkl010z1           Meteo…
#>  6 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 fkl010z0           Meteo…
#>  7 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 dkl010z0           Meteo…
#>  8 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 sre000z0           Meteo…
#>  9 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 sre000z0           Meteo…
#> 10 TAE        Aadorf… 8°54' 47°29' 2.71e6 1.26e6   539 gre000z0           Meteo…
#> # ℹ 269 more rows
#> # ℹ 6 more variables: unit <chr>, Beschreibung <chr>, timezone_original <chr>,
#> #   site_long <lgl>, interval <chr>, parameter <chr>
#> 
#> $meteotest
#> # A tibble: 2 × 12
#>   site_short site_long site       lon   lat    masl parameter_original parameter
#>   <chr>      <lgl>     <chr>      <lgl> <lgl> <dbl> <chr>              <chr>    
#> 1 SMA_23m    NA        Zürich - … NA    NA      581 ta1tows0           T        
#> 2 ZUEberg    NA        Zürich - … NA    NA      729 ta1tows0           T        
#> # ℹ 4 more variables: unit <chr>, interval <chr>, timezone_original <chr>,
#> #   source <chr>
#> 
#> $nabel
#> # A tibble: 12,348 × 39
#>    source mspbJahr  msID site        site_short msNameKurz site_long msKT  msOrt
#>    <chr>     <int> <int> <chr>       <chr>      <chr>      <chr>     <chr> <chr>
#>  1 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#>  2 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#>  3 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#>  4 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#>  5 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#>  6 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#>  7 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#>  8 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#>  9 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#> 10 NABEL      1991  1084 Zürich-Kas… NA         Zürich-Ka… Zürich-K… ZH    Züri…
#> # ℹ 12,338 more rows
#> # ℹ 30 more variables: msOrtsteil <chr>, x <int>, y <int>, masl <int>,
#> #   mmID <int>, ssID <int>, mmKurz <chr>, mspbPublic <lgl>, mspbWeb <lgl>,
#> #   mspbBafu <lgl>, scBisDatum <dttm>, scDTV <chr>, scDTVJahr <chr>,
#> #   scLKWProzent <int>, scStrassenabstand <int>, scEinwohner <int>,
#> #   scEinwohnerJahr <chr>, scEinDatum <dttm>, scBezugJahr <int>,
#> #   scVerkehrslage <chr>, scSiedlungsgroesse <chr>, …
#> 
#> $ostluft
#> # A tibble: 35,093 × 39
#>    source  mspbJahr  msID site       site_short msNameKurz site_long msKT  msOrt
#>    <chr>      <int> <int> <chr>      <chr>      <chr>      <chr>     <chr> <chr>
#>  1 OSTLUFT     2001     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  2 OSTLUFT     2003     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  3 OSTLUFT     1990     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  4 OSTLUFT     1991     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  5 OSTLUFT     1992     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  6 OSTLUFT     1993     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  7 OSTLUFT     1994     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  8 OSTLUFT     1995     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  9 OSTLUFT     1996     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#> 10 OSTLUFT     1997     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#> # ℹ 35,083 more rows
#> # ℹ 30 more variables: msOrtsteil <chr>, x <int>, y <int>, masl <int>,
#> #   mmID <int>, ssID <int>, mmKurz <chr>, parameter_original <chr>,
#> #   mspbPublic <lgl>, mspbWeb <lgl>, mspbBafu <lgl>, scBisDatum <dttm>,
#> #   scDTV <chr>, scDTVJahr <chr>, scLKWProzent <int>, scStrassenabstand <int>,
#> #   scEinwohner <int>, scEinwohnerJahr <chr>, scEinDatum <dttm>,
#> #   scBezugJahr <int>, scVerkehrslage <chr>, scSiedlungsgroesse <chr>, …
#> 
#> $seepolizei
#> # A tibble: 30 × 14
#>    parameter_original      unit  site_short site_long site       lon   lat  masl
#>    <chr>                   <chr> <chr>      <lgl>     <chr>    <dbl> <dbl> <dbl>
#>  1 air_temperature         °C    mythenquai NA        ZH_Myth…  8.54  47.4   407
#>  2 water_temperature       °C    mythenquai NA        ZH_Myth…  8.54  47.4   407
#>  3 wind_gust_max_10min     m/s   mythenquai NA        ZH_Myth…  8.54  47.4   407
#>  4 wind_speed_avg_10min    m/s   mythenquai NA        ZH_Myth…  8.54  47.4   407
#>  5 wind_force_avg_10min    bft   mythenquai NA        ZH_Myth…  8.54  47.4   407
#>  6 wind_direction          °     mythenquai NA        ZH_Myth…  8.54  47.4   407
#>  7 windchill               °C    mythenquai NA        ZH_Myth…  8.54  47.4   407
#>  8 barometric_pressure_qfe hPa   mythenquai NA        ZH_Myth…  8.54  47.4   407
#>  9 precipitation           mm    mythenquai NA        ZH_Myth…  8.54  47.4   407
#> 10 dew_point               °C    mythenquai NA        ZH_Myth…  8.54  47.4   407
#> # ℹ 20 more rows
#> # ℹ 6 more variables: interval <chr>, timezone_original <chr>, source <chr>,
#> #   x <dbl>, y <dbl>, parameter <chr>
#> 

## or a specific meta file
store$get_meta("ostluft")
#> $ostluft
#> # A tibble: 35,093 × 39
#>    source  mspbJahr  msID site       site_short msNameKurz site_long msKT  msOrt
#>    <chr>      <int> <int> <chr>      <chr>      <chr>      <chr>     <chr> <chr>
#>  1 OSTLUFT     2001     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  2 OSTLUFT     2003     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  3 OSTLUFT     1990     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  4 OSTLUFT     1991     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  5 OSTLUFT     1992     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  6 OSTLUFT     1993     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  7 OSTLUFT     1994     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  8 OSTLUFT     1995     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#>  9 OSTLUFT     1996     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#> 10 OSTLUFT     1997     1 Arb_Stadt… ARBS       Arbon Sta… Arbon St… TG    Arbon
#> # ℹ 35,083 more rows
#> # ℹ 30 more variables: msOrtsteil <chr>, x <int>, y <int>, masl <int>,
#> #   mmID <int>, ssID <int>, mmKurz <chr>, parameter_original <chr>,
#> #   mspbPublic <lgl>, mspbWeb <lgl>, mspbBafu <lgl>, scBisDatum <dttm>,
#> #   scDTV <chr>, scDTVJahr <chr>, scLKWProzent <int>, scStrassenabstand <int>,
#> #   scEinwohner <int>, scEinwohnerJahr <chr>, scEinDatum <dttm>,
#> #   scBezugJahr <int>, scVerkehrslage <chr>, scSiedlungsgroesse <chr>, …
#> 

## get the cache as local storage
local <- store$get_local_storage()
local$list_chunks()
#> # A tibble: 21 × 7
#>    chunk_name  interval site  year  local.path local.modification_t…¹ local.size
#>    <fs::path>  <chr>    <chr> <chr> <fs::path> <dttm>                 <fs::byte>
#>  1 …2XCuzIwMTM d1       Zch_… 2013  …IwMTM.rds 2023-12-25 16:12:02        19.84K
#>  2 …2XCuzIwMTQ d1       Zch_… 2014  …IwMTQ.rds 2023-12-25 16:12:02        76.76K
#>  3 …2XCuzIwMTU d1       Zch_… 2015  …IwMTU.rds 2023-12-25 16:12:03        86.04K
#>  4 …2XCuzIwMTM h1       Zch_… 2013  …IwMTM.rds 2023-12-25 16:12:03       112.38K
#>  5 …2XCuzIwMTQ h1       Zch_… 2014  …IwMTQ.rds 2023-12-25 16:12:03       447.54K
#>  6 …2XCuzIwMTU h1       Zch_… 2015  …IwMTU.rds 2023-12-25 16:12:03       567.64K
#>  7 …ZcK7MjAxMw h8gl     Zch_… 2013  …jAxMw.rds 2023-12-25 16:12:03        23.56K
#>  8 …ZcK7MjAxNA h8gl     Zch_… 2014  …jAxNA.rds 2023-12-25 16:12:03        93.14K
#>  9 …ZcK7MjAxNQ h8gl     Zch_… 2015  …jAxNQ.rds 2023-12-25 16:12:03         92.2K
#> 10 …2XCuzIwMTM m1       Zch_… 2013  …IwMTM.rds 2023-12-25 16:12:03         2.08K
#> # ℹ 11 more rows
#> # ℹ abbreviated name: ¹local.modification_time

## get cached data
local$get(site = "Zch_Stampfenbachstrasse", interval = "min30", year = 2011:2012)
#> # A tibble: 523,318 × 6
#>    starttime           site                    parameter interval unit    value
#>    <dttm>              <fct>                   <fct>     <fct>    <fct>   <dbl>
#>  1 2011-01-01 00:00:00 Zch_Stampfenbachstrasse CO        min30    ppm    0.525 
#>  2 2011-01-01 00:00:00 Zch_Stampfenbachstrasse Hr        min30    %Hr   91.9   
#>  3 2011-01-01 00:00:00 Zch_Stampfenbachstrasse NO2       min30    ppb   22.0   
#>  4 2011-01-01 00:00:00 Zch_Stampfenbachstrasse NO        min30    ppb   21.0   
#>  5 2011-01-01 00:00:00 Zch_Stampfenbachstrasse NOx       min30    ppb   43.0   
#>  6 2011-01-01 00:00:00 Zch_Stampfenbachstrasse O3        min30    ppb    1.10  
#>  7 2011-01-01 00:00:00 Zch_Stampfenbachstrasse PM10      min30    µg/m3 84.0   
#>  8 2011-01-01 00:00:00 Zch_Stampfenbachstrasse RainDur   min30    min    0     
#>  9 2011-01-01 00:00:00 Zch_Stampfenbachstrasse SO2       min30    ppb    1.90  
#> 10 2011-01-01 00:00:00 Zch_Stampfenbachstrasse StrGlo    min30    W/m2   0.0104
#> # ℹ 523,308 more rows

## destroy store (careful removes all files on the disk)
store$read.only = FALSE
store$destroy("DELETE")
#> Cache for Store s3_example destroyed

## No examples for write operations