Orderly supports a centralised workflow, where a central server holds the canonical version of a report. In order to make this workflow work, we need a number of things:
To make this workflow work, we have developed some additional software.
First, we have a docker-based server system called OrderlyWeb which can be deployed onto a server. We will communicate with this server over https using a package
orderlyweb which will follow
orderly onto CRAN.
The OrderlyWeb server software provides a number of things
orderlythat can run reports as needed using the
A diagram may help show how the pieces fit together
Getting OrderlyWeb installed requires a little work, and you will need access to an appropriate server running docker, our python-based deployment tool, and TLS certificates for the server that you are using. We recommend using vault for storing certificates and any secrets used in deployment.
OrderlyWeb will use GitHub for authentication, the setup for which is described here.
orderlyto talk to OrderlyWeb
Once OrderlyWeb is running, we can tell
orderly about it by adding an appropriate setting to the
orderly_config.yml file. For example, if you had one testing and one production instance
remote: testing: driver: orderlyweb::orderlyweb_remote args: host: testing.example.com port: 443 token: $GITHUB_TOKEN production: driver: orderlyweb::orderlyweb_remote args: host: production.example.com port: 443 token: $GITHUB_TOKEN
$GITHUB_TOKEN is an environment variable that holds a GitHub token with “user” scope. The
testing remote is listed first, and so will be the default one used in any remote-using command.
Several commands can interact with the remote server:
orderly::orderly_run_remotewill run a report on a remote server, committing it immediately. This will save the log alongside the report.
orderly::orderly_pull_archivewill pull a version of a report from a remote server
orderly::orderly_pull_dependencieswill pull all the dependencies of a report from the remote server (i.e., versions of reports declared in the
depends:section of a report)
We hope that this system will become easier to deploy in future. We use it internally on two projects with orderly archives that hold gigabytes of analyses, and it has evolved to meet our needs. If you think it might work for you but can't get started, please get in touch.