Welcome to `baton`’s documentation¶

baton |bəˈtän|: noun, A short stick or staff or something resembling one.

Client programs and API for use with iRODS (Integrated Rule-Oriented Data System).

baton is intended as a supplement to the command line client programs (ils, imeta etc.) provided with a standard iRODS installation. Its provides the following features not included in iRODS:

A single JSON format for listing results, composing queries and performing updates.
Listing of data objects and collections as JSON, including their metadata (AVUs), file size, replicates, checksums, access control lists (ACLs) and creation and modification timestamps.
Queries on metadata, on access control lists (ACLs), creation and modification timestamps and timestamp ranges. The full range of iRODS query operators is supported.
Uploading local files to create data objects.
Fetching data object content inlined within JSON (text files only) or downloading data objects to local files.
Unbuffered option for IPC via pipes with fine-grained error reporting for batch operations.
Simplified API over the iRODS general query API to ease construction of new custom queries.

Obtaining and installing `baton`¶

baton may be downloaded as source code from its GitHub homepage . It is written using the iRODS C API and requires a C compiler to build.

Install iRODS and the Jansson JSON library ) as described in their documentation.
Generate the configure script. If you download a release tar.gz archive, you may skip this step because the configure script is included.

autoreconf -i

Generate the makefiles (see INSTALL for arguments to configure). iRODS may be installed from OS packages or from source.

./configure


If you have iRODS headers and libraries installed in a non-standard
place, you will need to set the CPPFLAGS and LDFLAGS environmment
variables appropriately.

CPPFLAGS="-I/path/to/irods/headers" \
  LDFLAGS="-L/path/to/irods/libraries" ./configure

Compile

make

Optionally, run the test suite

make check

If you have run configure with the optional –enable-coverage flag you can generate test coverage statistics with lcov.

make check-coverage

Install, including HTML manual and manpage.

make clean install

The `baton` suite of programs¶

baton comprises several command-line programs, all of which accept a stream of JSON objects on standard input (or from a file) and emit a stream of JSON objects on standard output i.e.

{ object 1 }  { object 2 }  ...  { object n }

The JSON stream may be in a single file (the filename given on the command line using the --file option) or in multiple files concatenated on STDIN. Any whitespace between the JSON objects is ignored.

The baton programs are:

baton-chmod

Modify the access control lists of collections and data objects.
baton-get

Download data objects as raw files or with metadata, embedded in JSON.
baton-put

Upload local files to create data objects.
baton-list

Print the paths, metadata and access control lists and timestamps of collections and data objects.
baton-metamod

Add or remove specfic AVU s in the metadata on collections and data objects.
baton-metaquery

Print the paths, metadata and access control lists of collections and data objects matching queries on metadata.
baton-do

Perform a mixture of “list”, “chmod”, “get”, “put”, “metamod” and “metaquery” baton operations specified by arguments supplied as JSON.

baton-do supports additional operations currently unavailable in the other programs, namely: “remove” (remove a data object), “mkdir” and “rmdir” (create and remove collections, optionally recursively).

All of the programs are designed to accept a stream of JSON objects, one for each operation on a collection or data object. After each operation is complete, the programs may be forced flush their output (See the --unbuffered command line option.) to ease their use in bidirectional communication via Unix pipes.

For details of how errors are handled, see Representing Errors.

baton-chmod¶

$ jq -n '{collection: "/unit/home/user",                            \
             access: [{owner: "public", level: "null"},  \
                      {owner: "admin",  level: "own"}]}' | baton-chmod --recurse

$ jq -n '{collection: "/unit/home/user", data_object: "a.txt"          \
              access: [{owner: "oscar",  level: "read"},    \
                       {owner: "victor", level: "write"}]}' | baton-chmod

Options¶

--connect-time <integer>¶: The duration in seconds after which a connection to iRODS will be refreshed (closed and reopened between JSON documents) to allow iRODS server resources to be released. Optional, defaults to 10 minutes.

--file <file name>¶: A JSON file describing the data objects and collections. Optional, defaults to STDIN.

--help¶: Prints command line help.

--recurse¶: Recurse into collections. Defaults to false.

--silent¶: Silence error messages.

--unbuffered¶: Flush output after each JSON object is processed.

--unsafe¶: Permit relative paths, which are unsafe in iRODS 3.x - 4.1.x

--verbose¶: Print verbose messages to STDERR.

--version¶: Print the version number and exit.

baton-get¶

Synopsis:

$ jq -n '{collection: "/unit/home/user/", data_object: "a.txt"}' | baton-get

$ jq -n '{collection: "/unit/home/user/", data_object: "a.txt", \
          directory: "/data/user/local" }'                      | baton-get --save

$ jq -n '{collection: "/unit/home/user/", data_object: "a.txt", \
          directory: "/data/user/local",  file: "a_copy.txt"}'  | baton-get --save

This program accepts JSON objects as described in Representing data objects and collections and prints results in the same format. The target path must represent a data object. Its default mode is to JSON encode the file content under the JSON property data in the output.

Some local path properties may be omitted from the input JSON and will be inferred from the iRODS paths. If the file property is omitted, the local file will take its name from the data object. If the directory property is omitted, the current working directory will be used for any saved downloads.

baton-get performs an on the-the-fly MD5 checksum of the data object content as it processes. It compares this with the expected MD5 checksum held in iRODS and will raise an error if they do not match. The program does not verify the checksum of the file(s) after they have been written to disk.

Options¶

--acl¶: Print access control lists in the output, in the format described in ACLs of data objects and collections.

--avu¶: Print AVU lists in output, in the format described in Metadata on data objects and collections.

--connect-time <integer>¶: The duration in seconds after which a connection to iRODS will be refreshed (closed and reopened between JSON documents) to allow iRODS server resources to be released. Optional, defaults to 10 minutes.

--file <file name>¶: A JSON file describing the data objects and collections. Optional, defaults to STDIN.

--help¶: Prints command line help.

--raw¶: Prints the contents of the data object instead of a JSON response. In this mode the program acts rather like the Unix program ‘cat’. This mode, or the –save mode must be used for any file that is not UTF-8 encoded text.

--save¶: Saves the contents of the data object to a local file. The local file path is defined by the JSON input in the format described in Representing Local Paths. Prints a JSON response to STDOUT for each file downloaded.

--silent¶: Silence error messages.

--size¶: Print data object sizes in the output. These appear as JSON integers under the property ‘size’. Where there are replicates, the size of the latest (highest numbered) replicate is reported.

--timestamp¶: Print data object timestamps in the output, in the format described in Representing timestamps.

--unbuffered¶: Flush output after each JSON object is processed.

--unsafe¶: Permit relative paths, which are unsafe in iRODS 3.x - 4.1.x

--verbose¶: Print verbose messages to STDERR.

--version¶: Print the version number and exit.

baton-put¶

Synopsis:

$ jq -n '{collection: "/unit/home/user/", data_object: "a.txt", \
          directory: "/data/user/local",  file: "a_copy.txt"}'  | baton-put

This program accepts JSON objects as described in Representing data objects and collections and prints results in the same format. The JSON must dsecribe a local file to upload and a target data object path to write. The target path may be an existing object, in which case it will overwrite.

baton-put performs an on the-the-fly MD5 checksum of the local file content as it is uploaded. It compares this with the eventual MD5 checksum held in iRODS and will raise an error if they do not match.

Options¶

--connect-time <integer>¶: The duration in seconds after which a connection to iRODS will be refreshed (closed and reopened between JSON documents) to allow iRODS server resources to be released. Optional, defaults to 10 minutes.

--file <file name>¶: A JSON file describing the data objects and collections. Optional, defaults to STDIN.

--help¶: Prints command line help.

--silent¶: Silence error messages.

--unbuffered¶: Flush output after each JSON object is processed.

--unsafe¶: Permit relative paths, which are unsafe in iRODS 3.x - 4.1.x

--verbose¶: Print verbose messages to STDERR.

--version¶: Print the version number and exit.

baton-list¶

Synopsis:

$ jq -n '{collection: "/unit/home/user/"}' | baton-list

$ jq -n '{collection: "/unit/home/user/", data_object: "a.txt"}' | baton-list

This program accepts JSON objects as described in Representing data objects and collections and prints results in the same format. If the target path is a data object, the printed result is also a data object. However, if the target path is a collection, the printed result is a JSON array of JSON objects containing one object representing the target collection followed by one for each collection or data object directly within the target.

Options¶

--acl¶: Print access control lists in the output, in the format described in ACLs of data objects and collections.

--connect-time <integer>¶: The duration in seconds after which a connection to iRODS will be refreshed (closed and reopened between JSON documents) to allow iRODS server resources to be released. Optional, defaults to 10 minutes.

--avu¶: Print AVU lists in output, in the format described in Metadata on data objects and collections.

--checksum¶: Print the current checksums of data objects. This does not recalulate checksums.

--contents¶: If the path is a collection print its contents, in the format described in Metadata on data objects and collections. Only the paths contained directly within a collection are printed.

--file <file name>¶: A JSON file describing the data objects and collections. Optional, defaults to STDIN.

--help¶: Prints command line help.

--replicate¶: Print data object replicate information, in the format described in Representing replicates.

--silent¶: Silence error messages.

--size¶: Print data object sizes in the output. These appear as JSON integers under the property ‘size’.

--timestamp¶: Print data object timestamps in the output, in the format described in Representing timestamps.

--unbuffered¶: Flush output after each JSON object is processed.

--unsafe¶: Permit relative paths, which are unsafe in iRODS 3.x - 4.1.x

--verbose¶: Print verbose messages to STDERR.

--version¶: Print the version number and exit.

baton-metamod¶

Synopsis:

$ jq -n '{collection: "/home/test", data_object: "a.txt",     \
               "avus": [{attribute: "x", value: "y"},   \
                        {attribute: "m", value: "n"}]}' | baton-metamod --operation add

$ jq -n '{collection: "/home/test", data_object: "a.txt",    \
                avus: [{attribute: "x", value: "y"},   \
                       {attribute: "m", value: "n"}]}' | baton-metamod --operation rem

This program accepts JSON objects as described in Representing data objects and collections and adds or removes matching metadata as described in Metadata on data objects and collections.

Options¶

--connect-time <integer>¶: The duration in seconds after which a connection to iRODS will be refreshed (closed and reopened between JSON documents) to allow iRODS server resources to be released. Optional, defaults to 10 minutes.

--file <file name>¶: A JSON file describing the data objects and collections. Optional, defaults to STDIN.

--help¶: Prints command line help.

--operation <operation name>¶: The operation to perform; one of add or remove.

--silent¶: Silence error messages.

--unbuffered¶: Flush output after each JSON object is processed.

--unsafe¶: Permit relative paths, which are unsafe in iRODS 3.x - 4.1.x

--verbose¶: Print verbose messages to STDERR.

--version¶: Print the version number and exit.

baton-metaquery¶

Synopsis:

$ jq -n '{avus: []}' | baton-metaquery

$ jq -n '{collection: "/unit/home/user", avus: []}' | baton-metaquery

$ jq -n '{avus: [{attribute: "x", value: "y"}]}' | baton-metaquery

$ jq -n '{avus: [{attribute: "x", value: "y"},   \
                 {attribute: "m", value: "n"}]}' | baton-metaquery

$ jq -n '{avus: [{attribute: "v", value: "100", operator: "n<"},     \
                 {attribute: "w", value: "n%",  operator: "like"}]}' | baton-metaquery

This program accepts JSON objects as described in Metadata queries and prints results in the format described in Representing data objects and collections . Additionally, it accepts an optional collection property which limits the scope of the associated metadata query to returning only those results that lie somewhere under that collection.

Options¶

--acl¶: Print access control lists in the output, in the format described in ACLs of data objects and collections.

--avu¶: Print AVU lists in the output, in the format described in Metadata on data objects and collections.

--connect-time <integer>¶: The duration in seconds after which a connection to iRODS will be refreshed (closed and reopened between JSON documents) to allow iRODS server resources to be released. Optional, defaults to 10 minutes.

--checksum¶: Print the current checksums of data objects. This does not recalulate checksums.

--coll¶: Limit the search to collection metadata only.

--file <file name>¶: A JSON file describing the data objects and collections. Optional, defaults to STDIN.

--help¶: Prints command line help.

--obj¶: Limit the search to data object metadata only.

--silent¶: Silence error messages.

--size¶: Print data object sizes in the output. These appear as JSON integers under the property ‘size’.

--timestamp¶: Print timestamp lists in the output, in the format described in Representing timestamps.

--unbuffered¶: Flush output after each JSON object is processed.

--unsafe¶: Permit relative paths, which are unsafe in iRODS 3.x - 4.1.x

--verbose¶: Print verbose messages to STDERR.

--version¶: Print the version number and exit.

--zone <zone name>¶: Query in a specific zone.

baton-do¶

Synopsis:

$ jq -n '{"operation": "metaquery",
          "arguments": {
              "avu": true,
              "acl": true,
              "replicate": true,
              "object": true
           },
           "target": {avus: [{attribute: "x", value: "y"},   \
                             {attribute: "m", value: "n"}]}' | baton-do

This program accepts JSON objects as described in Representing data objects and collections that have been wrapped in an additional JSON envelope describing a operation to be carried out, along with any parameters for that operation.

The JSON envelope has two mandatory properties; operation, whose value must be a string naming a baton operation to be performed (one of checksum, chmod, get, put, list, metamod, metaquery, move) and target which must be a baton-format JSON object. The envelope has one optional property arguments which, if present, must be a JSON object whose keys and values may be any of the command line options permitted for the standard baton clients supporting the previously named operations. Where command line options are boolean flags, a JSON true value should be used.

Options¶

--connect-time <integer>¶: The duration in seconds after which a connection to iRODS will be refreshed (closed and reopened between JSON documents) to allow iRODS server resources to be released. Optional, defaults to 10 minutes.

--file <file name>¶: A JSON file describing the baton operations and their parameters. Optional, defaults to STDIN.

--help¶: Prints command line help.

--silent¶: Silence error messages.

--no-error¶: Do not report iRODS errors by exiting with a non-zero error code. In this mode errors are reported only in-band of the JSON messages written to STDOUT.

--unbuffered¶: Flush output after each JSON object is processed.

--verbose¶: Print verbose messages to STDERR.

--version¶: Print the version number and exit.

--zone <zone name>¶: Operate in a specific zone.

Representing data objects and collections¶

Data objects and collections are represented as JSON objects. A collection is identified by the collection property or its shorter synonym c. For example, the current iRODS working collection may be represented by either of:

{"collection": "."}
{"coll": ""}

Warning

Relative paths in iRODS are dangerous!

iRODS 3.x - 4.1.x have a bug which means that a client’s current working collection may, in rare cases, change without notice.

See https://github.com/irods/irods/issues/2406

As a consequence, baton rejects all relative paths, unless the --unsafe command line argument is given.

The value associated with the collection property may be any iRODS path, absolute or relative, in UTF-8 encoding. A trailing slash on collections is permitted. This behaviour is different from iRODS icommands which do not recognise collections specified with a trailing slash:

{"collection": "/unit/home/user/λ/"}
{"collection": "/unit/home/",
   "contents": [{"collection": "/unit/home/user/λ/"}]}

A collection’s contents may be represented by the contents property. This is a JSON list which contains JSON objects representing other collections and data objects. A contents property on a data object JSON object has no meaning and will be ignored.

A data object is identified by having both a collection (or coll) property and a data_object property or its shorter synonym obj. For example, a file in the current iRODS working collection may be represented by either of:

{"collection:" "/unit/home/user", "data_object": "README.txt"}
{"coll:" "/unit/home/user", "obj": "README.txt"}

The value associated with the data_object property may be any iRODS data object name, in UTF-8 encoding. The full path of the data object may be recreated by concatenating the collection and data_object values. Data objects reported by listing or searches will may information on the file size under the size property. The value is a JSON integer indicating the file size in bytes as given in the ICAT database. If checksum reporting has been requested, data object checksums are given under the checksum property.

{"collection:" "/unit/home/user", "data_object": "README.txt", "size": 123456}

The above JSON representations of collections and data objects are sufficient to be passed to baton’s baton-list program, which fulfils a similar role to the iRODS ils program, listing data objects and collection contents.

baton ignores JSON properties that it does not recognise, therefore it is harmless to include extra properties in program input.

Representing replicates¶

In cases where data objects have replicates, the full details of replication may be displayed, including replicate identity numbers, status (valid or invalid), resource name, location and checksum.

{"collection": "/unit/home/user", "data_object": "README.txt",
 "replicates": [{"checksum": "2ebec02316ddc3ee64a67c89b5e3140c",
                 "location": "unit-a-1",
                 "resource": "test_resource",
                 "number": 0,
                 "valid": true}]}

Representing Local Paths¶

Copies of data objects on a filesystem local to the client are represented as JSON objects. A directory is identified by the directory property or is shorter synonym dir.

For example, the current working directory may be represented by either of:

{"directory": "."}
{"dir": "."}

The value associated with the directory property may be any path, absolute or relative, in UTF-8 encoding.

A file is identified by having both a directory (or dir) property and a file property. For example, a file in the current working directory may be represented by either of:

{"directory:" ".", "file": "README.txt"}
{"dir:" ".", "file": "README.txt"}

The value associated with the file property may be any iRODS data object name, in UTF-8 encoding. The full path of the local file may be recreated by concatenating the directory and file values.

A JSON object may have both iRODS path and local path properties, thereby indicating a mapping between a data object and a local file. The data_object and file values are not required to be identical.

{"collection": "/unit/home/user", "data_object": "README.txt",
 "directory:" ".",  "file": "README.txt"}

Representing metadata¶

Metadata on data objects and collections¶

Metadata attribute/value/units tuples are represented for both collections and data objects as a JSON array of objects under the avus property. Each AVU JSON object within the array must have at least an attribute and a value property. A units property may be used where an AVU has defined units. The values associated with all of these properties are strings. These properties have shorter synonyms a, v and u, respectively.

{"collection": "/unit/home/user",
       "avus": [{"attribute": "a", "value": "b", "units": "c"},
                {"attribute": "m", "value": "n"},
                {"attribute": "x", "value": "y", "units": "z"}]}

{"coll": "/unit/home/user",
 "avus": [{"a": "a", "v": "b", "u": "c"},
          {"a": "m", "v": "n"},
          {"a": "x", "v": "y", "u": "z"}]}

Any path that has no metadata may have an avus property with an empty JSON array as its value.

Metadata queries¶

This format for AVU s is used both in reporting existing metadata and in specifying baton queries. In the latter case a JSON AVU object acts as a selector in the query, with multiple AVU s being combined with logical AND and the comparison operator defaulting to =. As with the iRODS imeta program, units are ignored in queries.

For example, the following JSON may be passed to the baton baton-metaquery program:

{"avus": [{"attribute": "a", "value": "b", "units": "c"},
          {"attribute": "x", "value": "y"}]}

This is equivalent to a query using the iRODS imeta program of:

a = b and x = y

In cases where the = operator is not appropriate, an additional operator property may be supplied to override the default. The available operators are:

=
like
not like
in
>
n> (numeric coercion)
<
n< (numeric coercion)
>=
n>= (numeric coercion)
<=
n<= (numeric coercion)

The in operator requires that the attribute’s value in the query be a JSON list of possible strings to test. A successful match will be any metadata having one of those strings as a value for that attribute. e.g.

{"avus": [{"attribute": "a", "value": ["a", "b", "c"], "operator": "in"}]}

To modify the above query to return results only where the attribute x has values numerically less than 100, the JSON AVU would become:

{"avus": [{"attribute": "a", "value": "b", "units": "c"},
          {"attribute": "x", "value": "100", "operator": "n<"}]}

This is equivalent to a query using the iRODS imeta program of:

a = b and x n< y

The shorter synonym o may be used as an alternative operator property so that the query above becomes:

{"avus": [{"a": "a", "v": "b", "u": "c"},
          {"a": "x", "v": "100", "o": "n<"}]}

A query specified may include a collection property. The effect of this is to both direct the query to a specific zone and possibly to limit results to items contained in a specific collection. This property allows queries to different zones to be mixed in the input stream to a single invocation of the query program.

To direct the query to a specific zone, only the zone portion of the path is required:

{"avus": [{"a": "a", "v": "b"}],
 "coll": "/public"}

This will direct the query to the public zone. If the query program has been started with a --zone argument explicitly, that will take precedence over the zone provided in the query.

To limit results to a specific collection:

{"avus": [{"a": "a", "v": "b"}],
 "coll": "/public/seq/a/b/c"}

This will limit query results to those with AVU a = b located in the collection ‘/public/seq/a/b/c’. Beware that this type of query may have significantly poor performance in the ICAT generated SQL.

Representing timestamps¶

The ICAT database records timestamps indicating when collections and data objects were created and last modified. These are represented for both collections and data objects as a JSON array of objects under the timestamp property. Each timestamp within the array must have at least a created or a modified property. The values associated with these properties are ISO8601 datetime strings. These properties have shorter synonyms c, m, respectively.

{"collection": "/unit/home/user",
 "timestamps": [{"created":  "2014-01-01T00:00:00"},
                {"modified": "2014-01-01T00:00:00"}]}

Data objects may have replicates in iRODS. Where data object timestamps are reported by baton, the values for all replicates are reported. The replicate number is given in the timestamp object as a JSON integer.

{"collection": "/unit/home/user",
 "data_object": "foo.txt",
 "timestamps": [{"created":  "2014-01-01T00:00:00", "replicate": 0},
                {"modified": "2014-01-01T00:00:00", "replicate": 0}]}

Timestamp queries¶

The format described in Representing timestamps is used in both reporting existing timestamps and in refining baton metadata queries. In the latter case a timestamp acts as a selector in the query, with multiple timestamps being combined with logical AND and the comparison operator defaulting to =. Time ranges may be queried by using multiple timestamp objects having the same key (created or modified).

In most cases the the = operator is not appropriate and additional operator property should be supplied to override the default. The available operators are described in Metadata queries. The numeric coercing operators are preferred because the ICAT database stores timestamps as seconds since the epoch.

For example, to limit query results to anything created on or after 2014-01-01, the syntax would be:

{"timestamps": [{"created": "2014-01-01T00:00:00", "operator": "n>="}]}

To limit query results to anything created before 2014-01-01 and modified in 2014-03, the syntax would be:

{"timestamps": [{"created":  "2014-01-01T00:00:00", "operator": "n<"},
                {"modified": "2014-03-01T00:00:00", "operator": "n>="},
                {"modified": "2014-03-31T00:00:00", "operator": "n<"}]}

Representing Access Control Lists¶

ACLs of data objects and collections¶

ACL s are represented for both collections and data objects as a JSON array of objects under the access property. Each JSON ACL object within the array must have at least an owner and a level property.

{"collection": "/unit/home/user",
     "access": [{"owner": "user",   "zone": "unit", "level": "own"},
                {"owner": "public", "zone": "unit", "level": "read"}]}

The value associated with the owner property must be an iRODS user or group name. The value associated with the level property must be an iRODS symbolic access level string (i.e. “null”, “read”, “write” or “own”). Permissions read from iRODS will also have a zone property, indicating the zone to which the owner belongs.

ACL queries¶

The format described in Representing Access Control Lists is used in both reporting existing permissions and in refining baton metadata queries. In the latter case an ACL acts as a selector in the query.

To limit query results to anything owned by user1, the syntax would be:

{"access": [{"owner": "user1", "level": "own"}]}

However, care is required; there are two important points to bear in mind when composing ACL queries:

The ICAT database queries will match exact permissions only and will not return results for items whose permissions are subsumed under the query permission. For example, a query for items for which user user1 has permissions read will not return any items where user1 has permissions read-write or own, despite the fact that the latter subsume read.
Only one permission clause per user may be included in the query. This is because ICAT combines queries using AND which, combined with its limitations described above, would yield queries that always return an empty set. E.g. the following query will never return any results:

{"access": [{"owner": "user1", "level": "own"},
            {"owner": "user1", "level": "read"}]}

Representing Errors¶

When an error occurs while processing a particular JSON object in a stream, instead of printing the usual result object, the input JSON object is printed to standard output with an error property added to it. The value associated with this property is a JSON object that includes the iRODS error code under a code property and (where possible) and an error message under a message property.

{"collection": "test",
      "error": {"code": -310000,
                "message": "Path '/unit/home/user/test' does not exist (or lacks access permission)"}}

In this way, errors remain associated with the inputs that caused them.

The `baton` Cookbook¶

baton benefits from the lightweight command-line JSON processor ‘jq’ . In the following examples, jq is used to ease the de novo creation of JSON in the shell and to process the results returned by baton.

Perform operations on a large number of iRODS items

Perform operations on a large number of iRODS items, over a single connection.

jq -n -c '{coll: "/zone1/home/user/data", obj: "f1.txt"}'  > in.json
jq -n -c '{coll: "/zone1/home/user/data", obj: "f2.txt"}' >> in.json
jq -n -c '{coll: "/zone1/home/user/data", obj: "f3.txt"}' >> in.json

baton-list < in.json

Result:

{
   "data_object": "f1.txt",
   "collection": "/zone1/home/user/data"
}
{
   "data_object": "f2.txt",
   "collection": "/zone1/home/user/data"
}
{
   "data_object": "f3.txt",
   "collection": "/zone1/home/user/data"
}

List the contents of the current working collection

List the contents of the current working collection as flat paths suitable for passing to standard Unix filters.

jq -n '{coll: "/zone1/home/user/data"}' | baton-list | jq -r '.[] | .collection + "/" + .data_object'

Result:

/zone1/home/user/data/
/zone1/home/user/data/f1.txt
/zone1/home/user/data/f2.txt
/zone1/home/user/data/f3.txt
/zone1/home/user/data/a/
/zone1/home/user/data/b/
/zone1/home/user/data/c/

List the contents of a collection by metadata (I)

List all the collections and data objects located directly within the collection data, that have the attribute attr_a somewhere in their AVUs:

jq -n '{coll: "/zone1/home/user/data"}' | baton-list --avu | jq 'map(select(.avus[] | select(.attribute == "attr_a")))'

Result:

[
  {
    "avus": [
      {
        "value": "value_b",
        "attribute": "attr_b"
      },
      {
        "value": "value_a",
        "attribute": "attr_a"
      },
      {
        "value": "value_c",
        "attribute": "attr_c"
      }
    ],
    "data_object": "f1.txt",
    "collection": "/unit/home/user/data",
    "size": 0
  }
]

List the contents of a collection by metadata (II)

A similar operation to the previous one, except using an iRODS query with scope limited to the collection data or its subcollections, recursively.

jq -n '{coll: "/zone1/home/user", avus: [{a: "attr_a", v: "%", o: "like"}]}' | baton-metaquery --avu | jq '.'

[
  {
    "avus": [
      {
        "value": "value_b",
        "attribute": "attr_b"
      },
      {
        "value": "value_a",
        "attribute": "attr_a"
      },
      {
        "value": "value_c",
        "attribute": "attr_c"
      }
    ],
    "data_object": "f1.txt",
    "collection": "/zone1/home/user/data",
    "size": 0
  }
]

List the metadata on data objects

List the metadata on all data objects in the collection data:

jq -n '{coll: "/zone1/home/user/data"}' | baton-list --avu | jq 'map(select(.data_object))[]'

{
  "avus": [
    {
      "value": "value_b",
      "attribute": "attr_b"
    },
    {
      "value": "value_a",
      "attribute": "attr_a"
    },
    {
      "value": "value_c",
      "attribute": "attr_c"
    }
  ],
  "data_object": "f1.txt",
  "collection": "/zone1/home/user/data",
  "size": 0
}
{
  "avus": [
    {
      "value": "value_c",
      "attribute": "attr_c"
    }
  ],
  "data_object": "f2.txt",
  "collection": "/zone1/home/user/data",
  "size": 0
}
{
  "avus": [
    {
      "value": "value_c",
      "attribute": "attr_c"
    }
  ],
  "data_object": "f3.txt",
  "collection": "/zone1/home/user/data",
  "size": 0
}

Run a simple iRODS metadata query

Run a simple iRODS metadata query for collections and data objects having both ‘attr_a’ = ‘value_a’ and ‘attr_c’ == ‘value_c’. The final pass through jq simply pretty-prints the result. The result contains no AVU information because the --avu argument to baton-metaquery was not used:

jq -n '{avus: [{a: "attr_a", v: "value_a"},   \
               {a: "attr_c", v: "value_c"}]}' | baton-metaquery | jq '.'

Result:

[
  {
    "data_object": "f1.txt",
    "collection": "/zone1/home/user/data",
    "size": 0
  }
]

Run a simple iRODS metadata query with timestamp selection

Run an iRODS metadata query for collections and data objects having both ‘attr_a’ = ‘value_a’ and ‘attr_c’ == ‘value_c’ that were created before 2014-03-01T00:00:00 and modified between 2014-03-17T00:00:00 and 2014-03-18T00:00:00:

jq -n '{avus: [{a: "attr_a", v: "value_a"},           \
               {a: "attr_c", v: "value_c"}],          \
        time: [{c: "2014-03-01T00:00:00", o: "n<"},   \
               {m: "2014-03-17T00:00:00", o: "n>="},  \
               {m: "2014-03-18T00:00:00", o: "n<"}]}' | baton-metaquery | jq '.'

Result:

[
  {
    "data_object": "f1.txt",
    "collection": "/zone1/home/user/data",
    "size": 0
  }
]

ACL¶: Access Control List.
AVU¶: Attribute Value Unit metadata tuple.

Welcome to `baton`’s documentation¶

Obtaining and installing `baton`¶

The `baton` suite of programs¶

baton-chmod¶

Options¶

baton-get¶

Options¶

baton-put¶

Options¶

baton-list¶

Options¶

baton-metamod¶

Options¶

baton-metaquery¶

Options¶

baton-do¶

Options¶

Representing data objects and collections¶

Representing replicates¶

Representing Local Paths¶

Representing metadata¶

Metadata on data objects and collections¶

Metadata queries¶

Representing timestamps¶

Timestamp queries¶

Representing Access Control Lists¶

ACLs of data objects and collections¶

ACL queries¶

Representing Errors¶

The `baton` Cookbook¶

Indices and tables¶

Table of Contents

This Page

Welcome to baton’s documentation¶

Obtaining and installing baton¶

The baton suite of programs¶

baton-chmod¶

Options¶

baton-get¶

Options¶

baton-put¶

Options¶

baton-list¶

Options¶

baton-metamod¶

Options¶

baton-metaquery¶

Options¶

baton-do¶

Options¶

Representing data objects and collections¶

Representing replicates¶

Representing Local Paths¶

Representing metadata¶

Metadata on data objects and collections¶

Metadata queries¶

Representing timestamps¶

Timestamp queries¶

Representing Access Control Lists¶

ACLs of data objects and collections¶

ACL queries¶

Representing Errors¶

The baton Cookbook¶

Indices and tables¶

Welcome to `baton`’s documentation¶

Obtaining and installing `baton`¶

The `baton` suite of programs¶

The `baton` Cookbook¶