Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .Renviron.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
AZ_STORAGE_EP=[your azure blob storage endpoint url]
AZ_TABLE_EP=[your azure table storage endpoint url]
4 changes: 2 additions & 2 deletions R/get_auth_token.R
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@
#' to help provide an appropriate string or vector.
#' If setting version to 2, ensure that the `aad_version` argument is also set
#' to 2. Both are set to use AAD version 1 by default.
#' @param tenant A string specifying the Azure tenant. Defaults to
#' `"common"`. See [AzureAuth::get_azure_token] for other values.
#' @param tenant A string specifying the Azure tenant. Defaults to `"common"`.
#' See [AzureAuth::get_azure_token] for other values.
#' @param client_id A string specifying the application ID (aka client ID). If
#' `NULL`, (the default) the function attempts to obtain the client ID from the
#' Azure Resource Manager token, or prompts the user to log in to obtain it.
Expand Down
4 changes: 2 additions & 2 deletions R/list_files.R
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
list_files <- function(container, dir = "", ext = "", recursive = FALSE) {
stopifnot(rlang::is_character(c(dir, ext), 2))
stopifnot(rlang::is_bool(recursive))
pnf_msg <- ct_error_msg("Path {.val {path}} not found")
pnf_msg <- ct_error_msg("Path {.val {dir}} not found")
check_that(dir, \(x) AzureStor::blob_dir_exists(container, x), pnf_msg)

ext_rx <- ifelse(nzchar(ext), gsub("^\\.+", "\\.", ext), ".*") # nolint
Expand All @@ -40,7 +40,7 @@ list_files <- function(container, dir = "", ext = "", recursive = FALSE) {
if (nrow(tbl) == 0) {
fix_path <- \(p) sub("^/+$", "", sub("^([^/])(.*)", "/\\1\\2", p)) # nolint
ext <- if (nzchar(ext)) paste0(" ", ext)
msg <- "No{ext} files found in {.val [{container$name}]:{fix_path(path)}}"
msg <- "No{ext} files found in {.val [{container$name}]:{fix_path(dir)}}"
cli::cli_alert_info(msg)
invisible(character(0))
} else {
Expand Down
40 changes: 29 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,13 @@ blob and table storage and reading in data from files.

## Status

The package is in development.
Please [create an issue][issues] if you have ideas for its improvement.
The package is stable and useable.
Please [create a GitHub issue][issues] if you encounter problems, or have ideas
for its improvement.

## Installation

You can install the development version of `{azkit}` with:
You can install `{azkit}` with:

``` r
# install.packages("pak")
Expand All @@ -36,17 +37,25 @@ pak::pak("The-Strategy-Unit/azkit")
A primary function in `{azkit}` enables access to an Azure blob container:

```r
# This assumes you have Azure authentication working, and the correct
# Azure endpoint variables in your environment (see below).
# NB the file/data locations used here are invented examples.
data_container <- azkit::get_container("data-container")

```
Authentication is handled automatically by `get_container()`, but if you need
to, you can explicitly return an authentication token for inspection or re-use:
Authentication should be handled automatically by `get_container()`, but if you
need to, you can explicitly return an authentication token for inspection or
re-use:

```r
my_token <- azkit::get_auth_token()

```

(For issues with authentication, including mysterious error messages, please
see the Troubleshooting section below or the [Troubleshooting vignette][trblv].)

[trblv]: https://the-strategy-unit.github.io/azkit/articles/troubleshooting.html

```r
data_container <- azkit::get_container("data-container", token = my_token)
```
Expand All @@ -63,7 +72,6 @@ For example:

```r
pqt_data <- azkit::read_azure_parquet(data_container, "important_data.parquet")

```

To read in any file from the container in raw format, to be passed to the
Expand All @@ -73,6 +81,8 @@ handler of your choice, use:
raw_data <- azkit::read_azure_file(data_container, "misc_data.ext")
```

Currently these functions only read in a single file at a time.

You can map over multiple files by first using `azkit::list_files()` and then
passing the file paths to the `read*` function:

Expand All @@ -81,12 +91,10 @@ azkit::list_files(data_container, "data/latest", "parquet") |>
purrr::map(\(x) azkit::read_azure_parquet(data_container, x))
```

Currently these functions only read in a single file at a time.

You can also pass through arguments in `...` that will be applied to the
appropriate handler function (see documentation).
For example, `readr::read_delim()` is used under the hood by
`azkit::read_azure_csv`, so you can pass through a config argument such as
`azkit::read_azure_csv`, so you can pass through an argument such as
`col_types`:

```r
Expand All @@ -97,7 +105,7 @@ csv_data <- data_container |>

## Environment variables

To facilitate access to Azure Storage you may want to set some environment
To facilitate access to Azure Storage you should set some environment
variables.
The neatest way to do this is to include a [`.Renviron` file][posit_env] in
your project folder.
Expand All @@ -120,6 +128,16 @@ AZ_TABLE_EP=
Azure authentication is probably the main area where you might experience
difficulty.

> [!NOTE]
> Users at The Strategy Unit: when initially authenticating with Azure, you
> will be sent to the web.
> Since you may be logged into one of multiple Microsoft tenants, it is
> important that you authenticate with the same account (MLCSU) that you use
> for access to Azure storage.
> If you authenticate when logged into your nhs.net Microsoft account in your
> browser, your token will not be useable for access to Azure.


To debug, try running:

```r
Expand Down
4 changes: 2 additions & 2 deletions man/get_auth_token.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading