R/remoteMember.R
reIndex.RdoutTar() writes an uncompressed tar (R's utils::tar() defaults to
compression = "none"), so every member sits at a known byte offset. Google
Drive answers HTTP Range requests with 206 Partial Content, so a single
object can be pulled out of a multi-GB archive by fetching only its bytes.
For a lazily saved simList this is the difference between downloading the
whole tarball and reading the one object you want: measured on a 4.63 GB
archive, one 6.2 MB object took 0.83 s against ~304 s for the whole file.
reIndex() fetches the <runName>_index.rds written beside the tarball;
reGetMember() uses it to fetch a single member.
reIndex(gFile, destDir = tempdir())
reGetMember(gFile, member, index = NULL, file = NULL, progress = NULL)A Google Drive dribble row, file id, or URL for the tarball.
Directory for the downloaded index. Default tempdir().
Character. The archive member to fetch. Matched exactly, or as a regular expression when no exact match exists (it must then match one row).
Optional data.frame from reIndex(). Fetched if not supplied;
pass it when reading several members so the index is downloaded once.
Optional path to write the raw member to. When NULL (default)
the member is read back with readRDS() and returned.
Logical or NULL. Show a transfer progress bar. NULL
(default) shows one in an interactive session for members over ~5 MB,
where the wait is long enough to look like a hang.
reIndex(): a data.frame with name, offset and size, one row
per archive member.
reGetMember(): the deserialized object, or the path written when
file is supplied.
reGetMember(), outTar(), reGetUntarLoad()