Provenance & Publishing

Every tool s4n creates, and every connection it (or SciWIn-Studio) draws between tools, becomes part of a provenance graph — think of it as an automatically-kept lab notebook: a node for every data or code artifact involved, and an edge for every command that turned one into another. That graph is what makes a workflow re-executable somewhere else, not just readable: you, or anyone else, can trace any result back to exactly what produced it, and rerun that chain from scratch.

Automatic semantic annotation

s4n create recognizes common file types by extension — CSV, JSON, images, Python scripts, and more — and tags each recognized input or output with its matching EDAM ontology format term automatically: a standardized label for the kind of data a file holds, so other tools and repositories can tell what it is without you writing that down yourself. No separate annotation step, no manual lookup.

Provenance Run Crates

Add --rocrate to s4n execute run and the run’s provenance graph is packaged as an RO-Crate — a standard way of bundling a folder of files with one machine-readable metadata file describing what each file is and how they relate, so the folder explains itself to anyone (or any tool) that opens it later, without you around to walk them through it. Concretely, that bundle holds: the workflow, a record of what actually ran — following the Workflow Run RO-Crate profile, which models a run the way PROV does (an activity using and producing artifacts, an agent orchestrating it) but expresses it as plain schema.org Actions rather than PROV-O’s own vocabulary — and structured (schema.org/JSON-LD) metadata, instead of a folder of loose files. Two layouts are available:

• --rocrate files (what a bare --rocrate means) keeps the original CWL files, directly re-executable.
• --rocrate packed writes one packed JSON document holding the whole workflow graph instead.

Running on REANA (--engine reana) always produces a packed crate — that’s the only layout REANA hands back.

flowchart LR
  Run["s4n execute run<br/>--rocrate"]
  Graph["provenance graph<br/>artifacts + commands"]
  Crate["Provenance Run Crate<br/>RO-Crate + schema.org Actions"]

  Run --> Graph --> Crate

Publishing

A Provenance Run Crate is already in the format WorkflowHub expects for a citable, FAIR-registered workflow — what turns “I ran this once” into something a colleague can find, cite with a DOI, and rerun themselves, the way you’d deposit code on GitHub or data on Zenodo, but for the whole runnable pipeline. s4n doesn’t push a crate to WorkflowHub automatically yet; direct integration is a stated but not-yet-built direction for the project. Publishing today means uploading the generated crate there yourself.

Remote execution specifics

Running against REANA has one compatibility wrinkle worth knowing about: REANA can only pull a pre-built container image, it can’t build a Dockerfile the way --engine docker or --engine local can. The code to work around this already exists in sciwin — build the tool’s Dockerfile locally and push the result to ttl.sh, a public, self-expiring registry (images live for one hour), so REANA has somewhere to pull it from — but it isn’t currently called from s4n execute run (tracked in #314). Today, a tool whose DockerRequirement only carries a dockerFile will simply fail against --engine reana; give it a dockerPull reference instead if you need to run it there.