Command-line interface to openEQUELLA
Quick Background
equella-cli or eq is a set of command-line shortcuts written in node.js for HTTP requests to the REST API. It uses a configured .equellarc file with at least an OAuth token and "root" URL of an oE instance.
$ npm install --global equella-cli # installation
// .equellarc
{
"root": "https://openequella.school.edu",
"token": "1234abcd-4321-dcba-1234abcd4321"
}
First, an example
This prints the title and URL of the first item contributed by the internal user "audrelorde".
$ eq search --order=date --reverse --owner \
> (eq user --name 'audrelorde' | jq -r .id) \
> | jq '.results[0] | .name, .links.view'
Here is a relatively simple example that nonetheless will demonstrate several things about `eq`. This command prints the title and URL of the very first thing contributed by the internal user "audrelorde".
Let's step through each portion of the command. First of all, we use the "search" API route which lets us search for and filter through lists of contributed items. We are using three different flags to modify our search: we want to order results by date last modified, we want to reverse the order of the results so the oldest items come first, and we want to limit results to items owned by "audrelorde". Because "audrelorder" is an internal user, however, we need to use a UUID here and not a username, so we execute a sub-command also using the "user" API route to find the UUID of the user with this username. You can see, for the output of both the search and user API calls, we are using a UNIX JSON utility "jq" to parse them and inspect the properties we are interested in.
Motivations behind eq
Learn more about openEQUELLA
Learn more about node.js & writing a CLI
I'm happy when I'm in my terminal
What did I create eq? I knew that openEQUELLA had comprehensive APIs but needed an excuse to dive deep into them. I also wanted to learn more about node.js and writing a command-line tool, since those were skills I was certain I wanted to learn. Finally, I just really love spending time reading green font on a black background and I'd love to interact with our digital archive from the terminal.
That isn't a trite observation about the command-line; while I'm certain there are good systems people who use the command line rarely or not at all, to me it's the most efficient way to accomplish most tasks because of the way it facilitates automation. Because any suite of actions can be composed into a larger script, which can be written to be adapt to different contexts, you can work so much more efficiently if you can frame your tasks into commands.
Text is the universal interface
Basics of the Unix Philosophy
Elaborating on that last point, one of the beautiful things of the command-line is that you can stitch together complicated routines from a series of relatively simple pieces of software. `eq` is a good example of one of these pieces of software, as fundamentally it does very little: it abstracts over the REST APIs a little bit and pretty prints the JSON you receive back from them so it's easier to read. It is nothing more than syntactic sugar.
Consider our initial example: because eq is turning openEQUELLA API calls into text that can be passed around, it can integrate with other CLI tools. In this example, it's integrated _with itself_ essentially: the output of the user API data is used again in the search API call. This sort of composability is the power of the command line and, more broadly, the UNIX philosophy of computing.
The CLI is littered with wonderful tools that adhere to another component of the UNIX philosophy, "do one thing and do it well." That's the goal of eq, to be a proxy for the openEQUELLA APIs and make their text output pluggable into other programs.
openEQUELLA APIs
Comprehensive, you can do almost anything
Already support third-party tools (EBI)
Great documentation & built-in testing at apidocs.do
The openEQUELLA APIs are very strong. There is an API route for almost everything and you can perform a great number of tasks easily through the APIs. It's notable that there are already a few major tools that use the openEQUELLA APIs. I believe these both use the SOAP and not REST APIs, but here at CCA we make extensive use of the Python SOAP API toolkit to create taxonomies and also of the EBI "EQUELLA Bulk Importer" app to import large numbers of items at once using spreadsheet data.
What I love most about the APIs, and what's made creating a tool like eq possible, is the API documentation. It's not only detailed but contains a built-in testing interface. You can practice calls to the API and learn about each route's different options. For instance, when I fleshed out the CLI for the search API route it was vital to see the full list of filtering options like "owner", "order", and "reverse".
eq & jq, best friends for life
$ # syntax-highlighting for item JSON data
$ eq item a7edaf1b-f853-4b6d-b8a6-e402f0fd2f58 | jq
$ # open 10 most recent items in your web browser
$ eq search | jq '.results[].links.view' | xargs open
eq almost always prints raw JSON output to the terminal. While it's pretty printed so it's easier to read, it's still hard to use JSON in many circumstances. Most utilities want simple text or URLs to work with. That's why jq is so essential, it acts as a translator taking the openEQUELLA JSON and turning it into more specific, usable pieces of text.
Here are two simple examples. In the first one, I find the JSON output of an item but jq helps to syntax highlight the detailed output. In the second, I use jq as a bridge from openEQUELLA search results to open those results in my web browser with the "open" command.
fx is great, tooeq item $UUID | fx
For the use case of merely inspecting JSON data, `fx` is perhaps an even better tool. `fx` is another npm package which not only syntax highlights the JSON you pipe to it but also collapses large objects to make data more readable. You can interactively browse through a large, nested data set, opening up object details by clicking their ellipsis. This is great for exploring some of the more detailed API output, like the items route, for instance. I haven't used `fx` as much as `jq` but it has similar filtering functionality, too.
Example: course list taxonomies
We have a huge course lists project which converts course information into a series of departmental taxonomies, turning them into vocabularly lists and a hierarchical course structure that our users can browse when contributing items.
course list taxonomies pt. II
eq finds our taxonomy UUIDs, using a consistent naming convention to look them up
for dept in $depts
set tax "$dept - COURSE LIST"
set uuid (eq tax --name $tax | jq -r '.uuid')
if [ $uuid ]
uptaxo --tid $uuid --csv \
data/$dept-course-list-taxo.csv
end
end
Example: items without titles
eq search -l 50 | jq '.results[] | (.links.view, .name)' \
| ack '^null$' -A 1Above: last 50 items. Below: loop for last 1,000 items.
$ for $IDX in (seq 0 19)
> eq search --length 50 --start (math $IDX x 50) | \
> jq '.results[] | (.links.view, .name)' | \
> ack '^null$' --after-context 1 | sed '/null/d'
> end
We have very complex contribution forms which make it hard to fully troubleshoot every possible path through the form that end users might take. Item titles are also not always composed of the same elements, they might contain a work type, course title, creator name, or creator-provided title depending on a number of factors. Sometimes, a contribution goes through and ends up with no title so openEQUELLA shows the item's UUID as its title, which is an eyesore.
openEQUELLA's Manage Resources tool is great at finding items that match particular metadata patterns and then modifying them, but it has a blind spot: you can test for the existence of a metadata node but not its nullity. There's no way to find items without titles. With the REST API, however, we can look for items whose JSON data lacks a `name` property. This first line will look for items without names within the last fifty contributions. We can loop through items 50 at a time to search our whole instance; the expanded code below checks the last thousand contributions and prints the UUIDs of items without titles.
There is probably a more elegant way to do this but I think it demonstrates how stitching together `eq`, `jq`, and existing command-line tools can help with management.
Miscellaneous Shortcuts
$ eq launch # run the admin console launcher script
$ eq login # fill in login page in a web browser
$ eq settings diagnostics # open user diagnostics
$ eq apidocs # open the API documentation
Limitations & warnings
There are few tests but I'm writing more
Does not lock resources before modification (but you could manually)
Unable to download items complete with files
Rare blind spots in the APIs, many more in eq
¿Questions?
Thank you for having me! I can be reached at phette23 on
most platforms (Twitter, Gmail, GitHub).
