πŸ“‹ Documentation Version Notice

You're reading the documentation for a development version (master). For the latest released version, please have a look at v1.7.

Experimental BIOMERO shallow OME-Zarr storage

Warning

This is an experimental, private BIOMERO storage contract. It is inspired by the shallow-copy use case in OME-NGFF RFC 8, but .biomero-shallow.json is not an RFC 8 Collection and is not a portable OME-NGFF standard. The current wire schema is version 1 and will evolve through versioned readers and upcasters.

One managed result, several OMERO views

The most useful way to understand this design is as one managed data graph with several views, similar to database views over shared stored data. BIOMERO.importer keeps the shallow result once, in place, in .analyzed. OMERO then registers lightweight objects whose PixelBuffer routes to the appropriate part of that result graph:

  • a source-backed Plate view displays the managed original intensity pixels;

  • an optional label-backed Plate view displays one selected label at the same well and field positions; and

  • Image results can expose each newly produced label as a separate mask Image.

These views do not each receive another copy of the Zarr arrays. They add OMERO database objects, hierarchy, and provenance, but reuse the managed source and label storage. The same shallow result can therefore look like the original Plate in one OMERO view and like a segmentation Plate in another, while still remaining one authoritative workflow result for later reconstruction.

        flowchart TB
    S[Read-only managed source pixels]
    R[One in-place shallow result<br/>metadata + local labels + managed references]
    S --> R
    R --> P[Source-backed OMERO Plate<br/>PixelBuffer displays intensity pixels]
    R --> L[Optional label-backed OMERO Plate<br/>PixelBuffer displays one selected label]
    R --> I[OMERO label Images<br/>PixelBuffer displays individual new labels]
    S --> F[Temporary reconstructed full Zarr]
    R --> F
    F --> W[Next Zarr-native workflow]

The Plate views are cheap in pixel-storage terms, but not completely free: each additional view still creates another OMERO Plate/Well/WellSample/Image hierarchy. BIOMERO therefore creates one source-backed Plate by default and at most one explicitly requested label-backed preview, rather than automatically registering a Plate for every label.

Standards status at a glance:

  • RFC 8 is a proposal, not a released OME-NGFF Collections specification;

  • BIOMERO emulates its composition idea with a private managed sidecar rather than claiming standards compliance;

  • the current workflow interchange profile is OME-NGFF 0.4 on Zarr v2, not the newest draft NGFF/Zarr feature set, because the Glencoe exporter and OMERO Zarr PixelBuffer must both be able to serve the result; and

  • generic readers are only expected to read a BIOMERO result after it has been reconstructed into a conventional full Zarr.

BIOMERO can store a label-producing Zarr workflow result without keeping a second copy of the unchanged input pixels. The retained result contains its labels, the structural metadata needed to describe them, and managed references to a full source Zarr. When that result is selected for a Zarr-consuming workflow, BIOMERO reconstructs an ordinary, self-contained OME-Zarr before transfer.

The feature deliberately applies only to derived workflow results. It never removes or rewrites the user’s original raw data or the managed full source. When BIOMERO cannot establish that returned pixels are unchanged, it keeps the returned Zarr in full. A workflow result can also be regenerated from its original input and parameters, so the optimization has a smaller risk boundary than deduplicating primary data.

Why shallow results exist

A typical segmentation workflow receives a full OME-Zarr, copies its image arrays to the output, and adds one or more NGFF labels. Persisting every such output duplicates the largest part of the data. This becomes costly for Plates: several segmentation and analysis runs can otherwise create several copies of the same hundreds of gigabytes or terabytes of intensity data.

The shallow result is a managed composition:

full managed source pixels ───────────────┐
                                          β”œβ”€ reconstruct ─> full workflow input
derived result metadata + local labels β”€β”€β”€β”˜

This is primarily a storage optimization, not an interchange format. Zarr-consuming workflows normally receive the reconstructed full Zarr so that generic tools can use intensity pixels, physical metadata, inherited labels, and new labels without understanding BIOMERO. A TIFF-consuming workflow is the intentional exception: its temporary Zarr conversion material represents the exact OMERO Image pixels the user selected, such as one registered mask Image, rather than the complete shallow collection.

Which Zarr is which?

There can be several Zarr directories during one workflow, but they have different owners and lifetimes. They must not all be interpreted as the same scientific object.

Artifact

Typical location

What it contains

Who consumes it

Managed source or canonical Zarr

Original managed location or .processed

Complete image pixels and NGFF structure; it may already contain labels

BIOMERO as the read-only pixel source

Temporary transfer Zarr

Workflow-specific transfer directory

A complete, ordinary Zarr assembled for this workflow

A Zarr-native workflow on HPC

Full returned Zarr

Temporary result, then .analyzed

Whatever the workflow produced, often copied input pixels plus labels

BIOMERO.importer before normalization

Stored shallow result

.analyzed

Result metadata, locally new or changed labels, and managed references to unchanged pixels and inherited labels

BIOMERO, OMERO registration, and later reconstruction

OMERO label Image

OMERO metadata plus PixelBuffer path

A view of one label node, not another copy of the whole collection

iViewer, thumbnails, ROI conversion, and user selection

Reconstructed follow-up input

Temporary transfer directory

Source pixels plus all inherited and local labels, materialized as one conventional Zarr

The next Zarr-native workflow

Note

PROCESSED_DATA_FOLDER is an importer-library setting (default .processed). Set it on the importer container to change preprocessing output. With BIOMERO_SHALLOW_ZARR=true and a custom folder, also set the same value on biomeroworker: Image Transfer calls the importer’s CanonicalStore there to choose destinations for new canonical Zarr copies. The processor forwards the name automatically from biomero.constants.slurm_env; the worker needs BIOMERO and importer versions supporting this option. With shallow-Zarr disabled, no worker setting is needed. Existing locations are read from stored metadata and are not migrated when this value changes.

The stored shallow directory is therefore often not byte-for-byte the Zarr that the workflow received. It is the compact, authoritative result in BIOMERO-managed storage. At the workflow boundary BIOMERO turns it back into a normal Zarr.

        flowchart LR
    A[OMERO selection] --> B{Workflow input format}
    B -->|Zarr-native| C[Full temporary Zarr]
    B -->|TIFF / BIAFLOWS| D[Disposable Zarr from selected PixelBuffer]
    D --> E[TIFF input]
    C --> F[Workflow]
    E --> F
    F --> G{Returned a Zarr?}
    G -->|no| H[Legacy result import]
    G -->|yes| I[Verify image and label identities]
    I -->|image pixels changed or uncertain| J[Keep full returned Zarr]
    I -->|image pixels unchanged| K[Store shallow result]
    K --> L[Keep new or changed labels locally]
    K --> M[Reference unchanged pixels and inherited labels]
    L --> N[Create new OMERO label views]
    M --> O[Reconstruct when selected for another Zarr workflow]
    L --> O

What the workflow actually receives

User selection and workflow

Data delivered to the workflow

Ordinary Image selected for a Zarr-native workflow

A complete Zarr copied from an existing managed Zarr or reusable canonical conversion

Shallow label result selected for a Zarr-native workflow

A newly materialized full Zarr containing the original intensity pixels, every inherited label, and every label stored by the selected result

Ordinary or label Image selected for a TIFF/BIAFLOWS workflow

TIFF converted from the selected OMERO PixelBuffer; for a label Image this means mask pixels, not reconstructed intensity pixels

Plate selected for a workflow

A complete Plate Zarr; Plates remain Zarr-only and are never flattened into the TIFF exception

Zarr uploaded directly through BIOMERO.importer

The submitted Zarr is imported normally; no workflow input snapshot exists, so BIOMERO does not automatically shallow it

The temporary Zarr used before a TIFF conversion is an implementation detail of the older transfer path. The TIFF workflow never receives that Zarr. It is also excluded from canonical promotion and return-side Zarr matching.

Labels across workflow generations

BIOMERO identifies label pixels independently from image pixels. A follow-up workflow may receive five existing label layers and add four more. On return:

  • the five unchanged inherited labels remain members of the logical result but become references to their existing managed locations;

  • the four new labels remain physically stored in the new shallow result;

  • only the four new labels become new OMERO mask Images by default; and

  • the next reconstruction contains all nine labels exactly once.

This is label deduplication as well as image-pixel deduplication. Labels are not forgotten merely because their chunks are not copied into every result. If a workflow changes an existing label at the same logical path, its ISCC-BIO identity changes and BIOMERO stores that changed label as a new component.

Lifecycle

  1. Image Transfer resolves or creates a full managed Zarr for every selected OMERO Image or Plate. A non-Zarr original may therefore acquire a reusable canonical representation in .processed; an already managed Zarr can be used directly without making another canonical copy.

  2. BIOMERO calculates per-image and per-label pixel identities and stores the authoritative ordered input snapshot in its workflow event store.

  3. Each task-local input copy receives .biomero-input.json. This small marker identifies the selected input even when several inputs contain identical pixels or a workflow renames its result. It is not a workflow-provider contract and is removed from the stored result.

  4. The workflow runs against a normal, full OME-Zarr. It may ignore and simply copy BIOMERO metadata.

  5. On return, BIOMERO.importer recomputes the decoded pixel identities. If the image pixels match the corresponding source and useful labels are present, it transactionally removes the duplicated image arrays and writes .biomero-shallow.json. Changed or uncertain results stay full.

  6. OMERO registers viewable projections of the result while the authoritative shallow collection remains in managed .analyzed storage.

  7. Selecting a shallow result for a later Zarr workflow causes Image Transfer to materialize a temporary full Zarr containing the source pixels, inherited labels, and locally retained labels. For a workflow that will convert its inputs to TIFF, Image Transfer instead uses the established OMERO CLI Zarr export route for the selected OMERO Image. The registered PixelBuffer then remains the authority, so selecting a label Image exports that label rather than reconstructing and accidentally converting the original image. This conversion artifact is not promoted as a canonical source and carries no returned-Zarr matching contract. Temporary inputs are removed after transfer.

Return-side identity work and normalization belong to BIOMERO.importer. They are not tied to the lifetime of the OMERO.web request that submitted the workflow. The OMERO script currently waits for the import status, but an ended web session does not terminate importer-owned processing.

Relationship to OME-NGFF RFC 8

RFC 8 proposes Collections and, as a motivating use case, shallow copies of images with segmentations. BIOMERO mirrors the following design ideas:

  • unchanged image data can remain in a separately managed source;

  • a derived collection can retain labels and refer back to that source;

  • collection members can be composed into a complete view at a system boundary; and

  • a Plate can refer to source images per field rather than inventing one Plate-wide pixel checksum.

BIOMERO currently adds private machinery that RFC 8 does not define: OMERO object IDs and generations, logical storage roots, relative managed paths, workflow and transfer identifiers, an event-store input snapshot, ISCC-BIO pixel identities, and registration projections for the current OMERO PixelBuffer.

Consequently, a BIOMERO shallow result must not be presented as a standardized RFC 8 Collection. Generic OME-Zarr readers are not expected to follow its managed references. Once a compatible Collections model is released and supported by the surrounding OMERO stack, the private schema can be migrated or adapted behind its versioned reader.

NGFF label source

A retained label still has the standard NGFF 0.4 relationship:

{
  "image-label": {
    "source": {"image": "../../"},
    "version": "0.4"
  }
}

That relative path describes the label’s logical image inside the reconstructed Zarr. While the result is shallow, the local image arrays may be absent, so the BIOMERO sidecar is the authority for locating the externally managed pixels. Reconstruction makes the ordinary relative NGFF relationship valid again. We do not overload image-label.source.image with a deployment-specific absolute filesystem or object-store path.

The BIOMERO storage contract

The Pydantic models in biomero-schema are the shared contract between BIOMERO, BIOMERO.importer, and the OMERO scripts. Services must use those models instead of independently constructing JSON. The principal markers are:

Marker

Lifetime and purpose

.biomero-canonical.json

Identifies a committed reusable full source representation.

.biomero-input.json

Task-local input hint, validated against the workflow event snapshot and removed on return.

.biomero-shallow.json

Authoritative manifest for a stored derived result whose image arrays were omitted.

An abbreviated Image result looks like this:

{
  "schema": 1,
  "model": "rfc8-shallow-copy",
  "workflowId": "<workflow-id>",
  "transferArtifact": "segmentation-result.ome.zarr",
  "interchangeProfile": "ngff-0.4-zarr-v2",
  "images": [{
    "imageNodePath": ".",
    "source": {
      "storageRoot": "group-0-data",
      "relativePath": ".processed/canonical-image.ome.zarr",
      "sourceObjectType": "Image",
      "sourceObjectId": 42,
      "sourceGeneration": 1,
      "nodePath": ".",
      "pixelIdentity": {"method": "iscc-bio/imagewalk", "role": "image"}
    },
    "returnedPixelIdentity": {
      "method": "iscc-bio/imagewalk",
      "role": "image"
    },
    "labelNodePaths": ["labels/labels_nuclei"],
    "labelComponents": [{
      "logicalNodePath": "labels/labels_nuclei",
      "source": null,
      "pixelIdentity": {"method": "iscc-bio/imagewalk", "role": "label"}
    }]
  }]
}

The actual identity objects also contain the ISCC codes, shape, dtype, axes, coordinate transformations, tool version, and IMAGEWALK revision. A Plate has one images entry for every retained field such as A/1/0; each entry points to that field in the managed source Plate.

source: null on a label component means the label is stored locally in this result. A managed source on a label component means it is inherited from an earlier shallow result. This distinction lets multi-stage workflows derive new masks from existing masks and later analyze the combined label set without repeatedly storing unchanged label pixels.

The Zarr root also carries a small biomero pointer to the manifest, but the sidecar is authoritative. A shallow root is not a synthetic black image: its duplicated multiscale image arrays are absent. This avoids storing even a fake pixel pyramid and prevents readers from mistaking zeros for scientific data.

Pixel identity with ISCC-BIO

The BIOMERO Schema documentation provides the normative, field-by-field PixelIdentity reference, including the exact equality predicate and a complete JSON example. This page focuses on how that contract participates in shallow storage.

BIOMERO uses the experimental ISCC-BIO IMAGEWALK implementation. IMAGEWALK traverses decoded level-0 bioimage planes deterministically and is designed to identify the logical pixels independently of their container, chunking, compression, extra pyramid levels, labels, and ordinary metadata changes. In principle this also lets BIOMERO recognize the same pixels in a raw format and in its canonical OME-Zarr representation.

For every image or label node BIOMERO records:

  • the combined ISCC value, Data-Code, and Instance-Code;

  • shape, dtype, axes, and coordinate transformations;

  • the node role (image or label) and logical node path; and

  • the ISCC-BIO version and IMAGEWALK implementation revision.

The current exact equality predicate compares the Instance-Code together with the role, shape, dtype, axes, and coordinate transformations. Node paths and aggregate/Data-Codes are not used to disambiguate otherwise identical selected images. The task-local marker and ordered event snapshot provide that mapping.

An embedded code is a claim, not proof that a workflow preserved the pixels. BIOMERO therefore records the input identity before execution and recomputes the returned pixels before removing anything. Copying a stale metadata field does not make changed pixels eligible. This is not intended as adversarial cryptographic attestation; it is a conservative decision about whether to keep more or less of a reproducible derived result.

Why TREEWALK is not the equality check

An ISCC-SUM TREEWALK over an entire Zarr answers whether the stored fileset is bit-identical. It changes when chunks are recompressed or rechunked, metadata or scales change, or labels are addedβ€”the exact changes a Zarr workflow may make without modifying the original image pixels. It is therefore the wrong signal for shallow eligibility.

TREEWALK remains interesting for future whole-store integrity, deduplication, version tracking, or citation. Its convention excludes a .iscc.json sidecar and supports .isccignore, avoiding a circular whole-store identifier. The schema already leaves room for a separate storeIdentity, but BIOMERO does not currently require one.

Embedded attrs.iscc

The intended portable direction is to publish an IMAGEWALK identity in the user attributes of each Image groupβ€”for Zarr v3, attributes.iscc as a sibling of the versioned attributes.ome namespace. That makes a derived Zarr carry a path-independent identity for its source. Current BIOMERO matching does not assume this attribute exists: identities are held in the managed markers and event provenance, and returned pixels are recomputed. Embedding and consuming the group attribute consistently is remaining interoperability work and may change with ISCC-BIO and NGFF guidance.

Eligibility and failure behavior

The normalizer is intentionally conservative:

Returned result

Storage outcome

Source pixels match and at least one local or inherited label exists

Store shallow collection.

Source pixels changed

Keep full returned Zarr.

Identity, source, field mapping, or schema is missing/ambiguous

Keep full returned Zarr.

Label-free pass-through duplicates only the input

Do not create a useless derived shallow result.

Feature flag disabled

Preserve the legacy full-result import path.

Importer integration disabled

Preserve the independent Get Results path.

Normalization uses a same-filesystem rollback journal. Duplicate array directories are moved into the journal first, the sidecar and remaining metadata are committed, and only then is the journal deleted. A failure before commit restores the result. The full managed source is read-only throughout.

Images, Plates, and OMERO representation

For an Image result, BIOMERO can register retained label nodes as separate viewable OMERO Images. This makes masks available to today’s PixelBuffer and iViewer and permits optional conversion to ROIs. The authoritative shallow collection stays in .analyzed; the OMERO objects carry compact managed references and provenance rather than a copy of the entire manifest.

For a chained result, the importer projects locally new or changed labels by default. Unchanged inherited labels remain in the shallow manifest and are included in later reconstruction, but do not create duplicate OMERO mask Images. An explicit re-projection option may be added later for users who need another OMERO view of an inherited mask.

For a Plate, labels live below each Plate image/field in NGFF. Importing every label from a large Plate as unrelated Images would lose the useful Plate organization and could create thousands of OMERO objects. BIOMERO therefore keeps one authoritative derived Plate representation. It can register:

  • a source-backed Plate whose ordinary pixels come from the original managed Zarr; and

  • optionally, a label-backed Plate preview when one requested label name is present consistently across the fields.

The preview is a convenience for current OMERO viewing, not another authority. Native label overlays and complete RFC 8 traversal depend on future OMERO and viewer support. Per-field identities and mappings remain in the storage sidecar; OMERO gets one compact Plate-level reference instead of hundreds or thousands of repeated key-value annotations.

Compatibility profile

BIOMERO currently exchanges OME-NGFF 0.4 on Zarr v2. This is determined by the deployed Glencoe exporter/importer tooling and the OMERO Zarr PixelBuffer that must serve registered pixels; accepting a newer, valid NGFF version in one component would not help if the rest of the OMERO path could not read it.

Workflows do not need to know about BIOMERO’s shallow-storage representation. BIOMERO inspects and optimizes ordinary workflow results only after the workflow has finished. The workflow-facing image and label contract is documented under Developing OME-Zarr workflows. BIOMERO will advance the profile as Glencoe and OMERO releases add compatible support. The private shallow reader remains versioned so older managed results can be reconstructed during such a transition.

Operational trade-off: storage versus import time

Shallow normalization trades importer CPU and storage I/O for lower persistent storage use. It is opt-in because a deployment with small results or a slow, metadata-heavy filesystem may value latency more than the saved capacity.

The current production-path Plate benchmark used an 18-field Plate returned by a Zarr-to-Zarr segmentation workflow, with one new label per field and 1,722 files. It ran inside the Linux importer container against the real /data mount; preparing disposable benchmark copies was excluded.

Measurement

Result

Full returned Plate

146,143,912 bytes

Stored shallow Plate

10,775,929 bytes

Storage removed

135,367,983 bytes (92.6%)

Read-only ISCC-BIO verification

12.653 s mean

Transactional normalization

12.729 s mean

Added importer processing

about 25.4 s

The first diagnostic implementation took roughly 196 seconds to normalize the same Plate. Same-filesystem moves, avoiding a copy of the retained label tree, and avoiding recursive before/after byte scans reduced normalization to 12.7 seconds.

The following observations put the Plate benchmark alongside the live Image paths tested so far. Estimated full means the size of the pixels and label components if they were materialized together; production deliberately skips an exact recursive pre-normalization size scan because that scan can cost more than normalization itself.

Scenario

Logical content

Full or estimated full

Stored shallow

Storage avoided

Return-path shallow processing

Outbound reconstruction

18-field Plate benchmark

18 fields, one new label per field

146,143,912 bytes

10,775,929 bytes

135,367,983 bytes (92.6%)

25.4 s mean

not measured

Five-Image live batch

five Images with new and inherited labels

28.463 MiB estimated

2.319 MiB

26.144 MiB (91.9%)

about 10 s total

not measured

Multi-generation Image chain

one Image, five inherited labels and four new labels

8,956,291 bytes estimated

990,300 bytes

7,965,991 bytes (88.9%)

19.7 s observed

7.6 s observed

Earlier individual Image

one Image result

6,848,883 bytes

185,072 bytes

6,663,811 bytes (97.3%)

not measured

not measured

The multi-generation observation started from one shallow Image referencing canonical intensity pixels and five inherited label layers. A Zarr-to-Zarr segmentation reconstructed that complete collection and appended four new labels. Reconstruction took 7.6 seconds. On return, the importer spent 18.3 seconds evaluating image and label identities with four workers and 1.4 seconds transactionally retaining only the four new layers. The five inherited label directories were absent from the new physical store but remained present as managed components in .biomero-shallow.json. Only the four new labels were registered as new OMERO Images. The estimated 8,956,291-byte full footprint is the sum of the referenced canonical pixels and all nine logical label components, not a recursive pre-normalization tree scan. This is one warm-system observation rather than a statistically stable benchmark.

A later live run processed five Image results with both new and inherited labels. Their estimated full size was 28.463 MiB and their stored shallow size was 2.319 MiB: 26.144 MiB, or 91.9%, was avoided. Importer identity and normalization work took approximately 10 seconds in total with four workers. One earlier individual Image example occupied 185,072 bytes shallow versus a 6,848,883-byte full source Zarr, a 97.3% size difference; that individual observation did not include a comparable end-to-end timing.

Parallel identity workers and scaling

BIOMERO_SHALLOW_ZARR_WORKERS controls a bounded importer thread pool for per-image and per-label identity generation. It defaults to 4 in the NL-BIOMERO deployment. Discovery, transactional moves, and journal deletion are not parallelized.

A preliminary read-only sweep of the 18-image/18-label Plate produced:

Workers

Verification time

1

14.524 s mean

2

11.988 s mean

4

9.401 s mean

8

12.569 s observed

16

13.857 s observed

32

13.201 s observed

Four workers performed best on this development mount. Higher counts increased I/O contention, and variance was material. Sites should benchmark their own storage with 1, 2, and 4 workers before increasing the value.

The 18-field result does not establish linear scaling. A deliberately crude linear extrapolation of 25.4 seconds would be about 24 minutes for 1,000 equally sized fields. Actual time depends on decoded pixel volume, label and file counts, chunking, cache state, filesystem metadata latency, and concurrent I/O; a very large or badly chunked Plate could still take hours. A representative large-Plate benchmark remains necessary before broad production enablement.

Enabling and observing the feature

The NL-BIOMERO Compose deployment uses:

IMPORTER_ENABLED=true
BIOMERO_SHALLOW_ZARR=true
BIOMERO_SHALLOW_ZARR_WORKERS=4

BIOMERO_SHALLOW_ZARR defaults off, preserving the old export/import behavior. The worker flag crosses the OMERO processor environment allow-list; the worker count belongs to the importer service. Shallow processing also requires the BIOMERO.importer identity extra, which supplies ISCC-BIO. The shipped NL-BIOMERO importer image installs that extra. A custom importer installation must use pip install "biomero-importer[identity]"; when it is absent, the importer reports the missing capability and rejects only shallow lifecycle orders while ordinary imports continue. Monitor image/field count, label count, bytes before and after, identity time, normalization time, and total import time. Disable the feature if its measured latency is not justified by the storage saved.

Useful logs distinguish:

  • calculation versus reuse of canonical source identities;

  • the selected source and transfer marker;

  • eligible (input-image-unchanged) versus a conservative keep-full reason;

  • identity generation and normalization durations; and

  • stored full versus shallow outcomes.

Validation status

The feature branch has verified the following live paths:

  • full canonical creation and later reuse during Image Transfer;

  • ordered event provenance and task-local markers, including five selected Images with identical pixels and renamed outputs;

  • five Image results normalized to 91.9% smaller shallow collections while retaining multiple and inherited labels;

  • label-Image registration, non-image attachments, and ROI creation for those Image results;

  • an 18-field Plate normalized to 92.6% smaller storage, with one compact derived Plate reference and optional label-backed preview;

  • a complete chained Zarr workflow: five inherited labels were reconstructed, four appended labels were retained, unchanged inherited label chunks were referenced rather than copied, and only the four new labels became OMERO Images;

  • focused reconstruction of a shallow Image into a temporary full Zarr;

  • a live and unit-covered TIFF-bound exception, where a selected shallow label Image is exported from its registered OMERO PixelBuffer instead of being reconstructed with the original pixels; and

  • compatibility readers/tests for older event streams and absent optional shallow settings.

The following remain release gates or scale validation:

  • a live changed-pixel Image and changed-pixel Plate must remain full;

  • a full canonical Zarr that already contains labels must be repeated live after the canonical-input inventory regression fix; unit coverage verifies that an empty preliminary inventory now triggers label discovery;

  • feature-off behavior and importer-disabled Get Results need live controls;

  • unsupported/newer NGFF input must fail or fall back clearly; and

  • a representative large high-content Plate needs storage-local timing and capacity measurements.

Expected evolution

The present contract fixes schema: 1, model: "rfc8-shallow-copy", and the ngff-0.4-zarr-v2 interchange profile. Recorded identities also pin the ISCC-BIO version and IMAGEWALK revision because ISCC-BIO is itself early-stage.

Viewer support can evolve independently of this storage contract. Today, BIOMERO registers source-backed and label-backed PixelBuffer views so that OMERO.iViewer can show either the intensity image or a selected mask as an ordinary Image or Plate. A label-aware viewer can instead follow the shallow references and present labels as overlays on the source pixels, which is the more natural NGFF experience; OME’s napari-ome-zarr already exposes NGFF labels as label layers. As native label support reaches OMERO viewers, and as the viewer integration being developed alongside BIOMERO matures, users should be able to open one managed result and switch or overlay its labels directly. That may make some convenience mask registrations optional, but it does not need to change the stored result, provenance, pixel identities, or reconstruction contract.

Likely future changes include migration toward a released NGFF Collections model, newer Zarr/NGFF profiles as OMERO PixelBuffer support advances, standardized embedded Image identities, object-store-aware references, and asynchronous or differently scheduled processing for very large Plates. Those changes should be introduced through new schema/profile versions and upcasters, not by silently changing the meaning of existing managed results.

Further reading