--- title: API Usage date: "2026-09-05" vignette: > %\VignetteIndexEntry{API Usage} %\VignetteEngine{litedown::vignette} %\VignetteEncoding{UTF-8} --- ## Storr set up Create a 'storr' using one of the two construction paths: - use the specialised constructor `storr_tiledb()`: ``` r uri <- tempfile() sto <- storr_tiledb(uri, init = TRUE) ``` - or explicitly with the standard 'storr' interface: ``` r uri2 <- tempfile() dr <- driver_tiledb(uri2, init = TRUE) sto2 <- storr::storr(dr) ``` Or, use the convenient wrapper when the driver is configured with one of `qs2` formats: ``` r sto2 <- storr_tdb0(uri2) ``` The first approach generates a `TileDBStorr` object that represents the storr interface optimised for TileDB storage and provides additional functionality. The latter is the standard `storr` object. For the rest of the vignette, the `storr_tiledb()` constructor will be used. ## API Examples ### 1. get, set, del ``` r dat <- head(mtcars, 2) # set sto$set("a", dat) sto$set("b", list(name = "John")) sto$mset(c("a", "b", "c"), value = list(dat, dat, dat), namespace = "ns1") # get sto$get("b") # $name # [1] "John" sto$mget(c("b", "b"), namespace = c("objects", "ns1")) # [[1]] # [[1]]$name # [1] "John" # # # [[2]] # mpg cyl disp hp drat wt qsec vs am gear # Mazda RX4 21 6 160 110 3.9 2.620 16.46 0 1 4 # Mazda RX4 Wag 21 6 160 110 3.9 2.875 17.02 0 1 4 # carb # Mazda RX4 4 # Mazda RX4 Wag 4 # list keys sto$list("objects") # [1] "a" "b" sto$list("ns1") # [1] "a" "b" "c" # list namespaces sto$list_namespaces() # [1] "ns1" "objects" # list hashes sto$list_hashes() # [1] "c184c6034d956360b5bb682fcd4b6cb8" # [2] "fd441562b6f1ec33e42a4369820cb0ae" # del sto$del("a") sto$del("b") sto$exists("a") # [1] FALSE # Retrieve hashes and namespaces sto$list_namespaces() # [1] "ns1" sto$list_hashes() # [1] "c184c6034d956360b5bb682fcd4b6cb8" # [2] "fd441562b6f1ec33e42a4369820cb0ae" # Delete unused hash sto$gc() sto$list_hashes() # [1] "c184c6034d956360b5bb682fcd4b6cb8" # Delete by namespace sto$clear("ns1") # [1] TRUE sto$list("ns1") # character(0) ``` ### 2. get_keymeta, update_keymeta With a key-namespace pair, you can attached notes and expiration metadata. Worth noting that setting key-metadata will update/replace only the chosen metadata and will not overwrite the other. Set key metadata when creating a key: ``` r # Expires in 5 minutes sto$set("aa", 1, expires_at = Sys.time() + 60 * 5, notes = "{\"name\":\"John\",\"id\":12345}") # Expires in 1 second sto$set("bb", 1, expires_at = Sys.time() + 1) ``` Or update/retrieve with `$update_keymeta()` and `$get_keymeta()` methods: ``` r # Retrieve key metadata sto$get_keymeta("aa", use_cache = FALSE) # $expires_at # [1] "2026-09-05 12:38:22 EEST" # # $notes # [1] "{\"name\":\"John\",\"id\":12345}" sto$mget_keymeta(c("aa", "bb"), use_cache = TRUE) # [[1]] # [[1]]$expires_at # [1] "2026-09-05 12:38:22 EEST" # # [[1]]$notes # [1] "{\"name\":\"John\",\"id\":12345}" # # # [[2]] # [[2]]$expires_at # [1] "2026-09-05 12:33:23 EEST" # # [[2]]$notes # [1] NA # Update key metadata sto$update_keymeta("bb", notes = "Updated Note") sto$get_keymeta("bb") # $expires_at # [1] "2026-09-05 12:33:23 EEST" # # $notes # [1] "Updated Note" ``` Updating 'expires_at' metadata for key 'aa' will not overwrite 'notes' ``` r # Update 'expires_at' sto$update_keymeta("aa", expires_at = as.POSIXct(NA)) # Retains 'notes' value sto$get_keymeta("aa") # $expires_at # [1] NA # # $notes # [1] "{\"name\":\"John\",\"id\":12345}" ``` **Fetch single metadata** For getting only one metadata value, either expiration or notes, `storr.tiledb` offers efficient methods: ``` r # Get only notes for key 'aa' sto$get_keymeta_notes("aa") # [1] "{\"name\":\"John\",\"id\":12345}" # # Get only expiration date-times sto$mget_keymeta_expires_at(c("aa", "bb")) # [[1]] # [1] NA # # [[2]] # [1] "2026-09-05 12:33:23 EEST" ``` **Expiration management** ``` r # Retrieve keys with expiration sto$keys_with_expiration() # namespace key expires_at # # 1: objects bb 2026-09-05 12:33:23 Sys.sleep(1) # Retrieve expired keys sto$expired_keys() # namespace key expires_at # # 1: objects bb 2026-09-05 12:33:23 # Remove expired keys sto$clear_expired_keys() # [1] TRUE # Check for expired keys sto$has_expired_keys() # [1] FALSE # Or via gc() sto$gc(clear_expired = TRUE) ``` **Clear metadata** ``` r sto$clear_keymeta("aa") sto$get_keymeta("aa") # $expires_at # [1] NA # # $notes # [1] NA ``` ### 3. update Use `update()` and `mupdate()` to replace key-values and retain their key-metadata. ``` r # Add new key sto$set("mykey", 1, notes = "mykey") # Update "mykey" sto$update("mykey", 10) # Check new value sto$get("mykey") # [1] 10 # Check 'notes' metadata sto$get_keymeta_notes("mykey") # [1] "mykey" ``` ### 4. Async methods Set keys without blocking R session: ``` r Sys.sleep(1) sto$set_async("abc", 1) sto$set_async("abc2", 2) ``` ### 5. Fragments Management When writing data to TileDB, it generates multiple array fragments on disk. Over time, these files need to be consolidated in order to improve query performance by reducing I/O overhead. Note that the consolidation operation does **not** delete the old fragments; it only creates new consolidated fragments. The actual cleanup of old fragments is performed by the separate **vacuum** operation. To manage storr TileDB fragments use `storr_fragments()` which encapsulates the consolidation and vacuum operations. ``` r # Set up a new storr uri <- tempfile() sto <- storr_tiledb(uri, init = TRUE) # Set 3 keys and 2 objects sto$set("a", 1) sto$set("b", 2) sto$set("c", 1) ``` ``` r # Initialise StorrFragments class fosto <- storr_fragments(uri) # Total fragments (5, 3 for keys, 2 for hashes) fosto$frag_num() # [1] 5 # Total fragments to vacuum fosto$to_vacuum_num() # [1] 0 ``` Consolidate everything (both arrays) ``` r # Consolidate all fosto$consolidate() # [1] TRUE ``` ``` r # Total fragments (2, 1 for keys, 1 for hashes) fosto$frag_num() # [1] 2 # Total fragments to vacuum (5) fosto$to_vacuum_num() # [1] 5 ``` Now, let's clean up the old fragments. ``` r # Cleaup old fragments fosto$vacuum() # [1] TRUE # Total fragments to vacuum (0) fosto$to_vacuum_num() # [1] 0 ```