📋 Documentation Version Notice

You're reading the documentation for a development version (master). For the latest released version, please have a look at v1.7.2.

BIOMERO.analyzer + BIOMERO.importer Integration

This guide covers connecting BIOMERO.analyzer’s SLURM result import to BIOMERO.importer so that analysis results land on your managed remote storage rather than on OMERO server storage.

New in version 1.3.0: Requires BIOMERO 2.3.0 or newer. The NL-BIOMERO demo enables this integration; existing deployments opt in with IMPORTER_ENABLED=true on the workflow worker and the shared-storage configuration below.

Note

Summary for system administrators:

• Enable with IMPORTER_ENABLED=true and provide a running importer service.

• Share the result-storage paths, group mappings and tracking database configuration; the worker needs write access to the target group folders.

• This is a prerequisite for BIOMERO Zarr workflows and Shallow OME-Zarr Storage. Configure it before those features.

• With the flag disabled, analysis uses classic Get Results rather than this importer path. Disabling it does not migrate existing results.

For full technical detail see Analyzer → Importer Integration.

Overview

Without this integration, results produced by analysis workflows (e.g., segmentation masks) are transferred back to OMERO via the standard OMERO API and end up on the OMERO server volume.

With this integration enabled:

1. The biomeroworker copies results to the shared remote storage in a dedicated .analyzed/<workflow-uuid>/<timestamp>/ folder.

2. It creates an upload order in the BIOMERO.importer tracking database.

3. BIOMERO.importer picks up the order and performs an in-place import — the same pipeline used for incoming research data.

4. The biomeroworker polls for completion and adds provenance metadata to the imported images.

This keeps analysis results under the same storage and permission model as raw data, avoids bloat on OMERO server storage, and provides a consistent in-place import experience.

Required configuration changes

.env

IMPORTER_ENABLED=true

Use the component versions supplied with your selected NL-BIOMERO release. For source builds, keep BIOMERO_IMPORTER_VERSION aligned with that release; custom deployments need compatible worker and importer packages.

docker-compose.yml — biomeroworker service

Add the following to the biomeroworker service:

Build args (so the Python library is installed in the container):

build:
  args:
    BIOMERO_IMPORTER_VERSION: ${BIOMERO_IMPORTER_VERSION}

Environment variables:

environment:
  IMPORTER_ENABLED: ${IMPORTER_ENABLED}
  INGEST_TRACKING_DB_URL: postgresql+psycopg2://${BIOMERO_POSTGRES_USER}:${BIOMERO_POSTGRES_PASSWORD}@database-biomero:5432/${BIOMERO_POSTGRES_DB}

Note

INGEST_TRACKING_DB_URL must point to the same database used by the biomero-importer service so that upload orders are visible to the importer.

Volume mounts:

volumes:
  - "./config/biomero-importer:/opt/omero/server/config-importer:ro"
  - "./web/biomero-config.json:/opt/omero/server/biomero-config.json:ro"

• config/biomero-importer — directory containing settings.yml for BIOMERO.importer. The worker reads base_dir from this file to determine where to write result files.

• web/biomero-config.json — used by OMERO.biomero to define per-group folder mappings. The worker uses these to resolve the correct destination path for each OMERO group.

Storage and file-system setup

Shared mount

The three containers below must all mount the same storage at the same container path:

Container	Purpose	Mount
biomeroworker	Writes .analyzed/ directory tree	./web/L-Drive:/data
biomero-importer	Reads files for in-place import and writes .processed/ directory	./web/L-Drive:/data
omeroserver	Resolves symlinks after in-place import	./web/L-Drive:/data

Note

The .processed subfolder name can be changed with the PROCESSED_DATA_FOLDER environment variable in the importer container; leaving it unset preserves the default. See BIOMERO.importer Container for details. This does not change the separate .analyzed results folder.

For a custom folder with BIOMERO_SHALLOW_ZARR=true, also set the same value on biomeroworker so generated canonical Zarrs use that folder. No worker setting is needed when shallow-Zarr is disabled.

Result folder structure

Results are placed under the OMERO group’s subfolder on the shared drive:

<group_base_path>/
└── .analyzed/
    └── <workflow-uuid>/
        └── <YYYYMMDD_HHMMSS>/
            ├── <job_id>_out.zip          # results archive
            ├── <job_id>_out/             # extracted images + CSVs
            │   ├── data/out/
            │   │   └── result.tiff
            │   └── metadata.csv          # workflow provenance
            └── omero-<job_id>.log        # SLURM job log

Group path resolution

The <group_base_path> is resolved in order:

1. Explicit mapping in biomero-config.json: if the user’s OMERO group has a folder mapping configured in the OMERO.biomero UI, that folder is used.

2. Fallback: <base_dir>/<group_name> from settings.yml. Example: group team-a with no mapping and base_dir=/data → /data/team-a/.

The group is the active OMERO group of the user who launched the analysis workflow.

Permissions

The biomeroworker process (runs as the omero-server user and group, uid=1000 gid=994, by default) needs:

• Write permission on the group’s base path.

• Permission to create the group subfolder if it does not exist yet (first workflow run for that group when no explicit mapping is configured).

Verify in the container:

# Check ownership of the data root
ls -la /path/to/L-Drive/
touch /path/to/L-Drive/test_write && echo "Write OK" || echo "Write failed"

# If needed, grant write access to UID 1000 and GID 994 or map containers to a different user with appropriate permissions.

Non-Docker deployments

If you are not using the NL-BIOMERO Docker Compose stack:

1. Install the biomero-importer Python library in the worker’s virtual environment:

   pip install biomero-importer==<version>
   

Note

You can ignore Python requirement and Zarr version warnings. Those checks apply only to preprocessing or custom pipelines, which are not used in this setup.

2. Set environment variables before starting the OMERO processor:

   export IMPORTER_ENABLED=true
   export INGEST_TRACKING_DB_URL=postgresql+psycopg2://user:pass@db:5432/biomero
   

3. Place configuration files at:

   • /opt/omero/server/config-importer/settings.yml

   • /opt/omero/server/biomero-config.json

4. Ensure the biomeroworker process has write access to the group folders in shared storage. See BIOMERO.analyzer SLURM Integration for the baseline worker deployment prerequisites.

Validating the setup

After rebuilding and restarting the biomeroworker container, verify the library is available:

docker-compose exec biomeroworker \
    /opt/omero/server/venv3/bin/python -c \
    "import biomero_importer; print('biomero-importer OK')"

Run a test analysis workflow. Afterwards check:

• A .analyzed/<uuid>/<timestamp>/ directory appears on the shared drive.

• The upload order appears in the importer tracking database:

  SELECT uuid, stage, created_at FROM imports ORDER BY created_at DESC LIMIT 5;
  

• Both the importer and the analyzer tabs from metabase show successful operations.

• You can find the imported images in OMERO.web using either UUID from the importer or analyzer.

• The imported images have Fileset Info in OMERO with info about Imported with: –transfer=ln_s Imported from:` and Paths on server:.

Zarr Workflow Support

From BIOMERO v2.4.0 / NL-BIOMERO v1.4.0 onward, this integration is also a prerequisite for Zarr-based analysis workflows (both regular Zarr workflows and Zarr plate workflows). BIOMERO uses the same shared remote storage pathway to stage Zarr input data for SLURM jobs and to retrieve results.

Without IMPORTER_ENABLED=true, the Zarr workflow type toggles in the Analyzer Admin UI have no effect and workflows will fall back to standard TIFF-based processing.

For per-workflow configuration of Zarr types, see Zarr and Plate Workflow Types in OMERO.biomero Plugin Administration.

Shallow OME-Zarr Storage

With this integration configured, shallow storage can avoid duplicate pixels in eligible workflow results. The demo enables it; other deployments opt in with BIOMERO_SHALLOW_ZARR=true. See Shallow OME-Zarr Storage for setup, local/remote processing choices and storage/time trade-offs, and Working with shallow OME-Zarr results for how the data is represented and reconstructed.

Troubleshooting

biomero-importer not found

Rebuild the biomeroworker image — the library is installed at build time. Confirm the BIOMERO_IMPORTER_VERSION build arg is present in the Compose file.

Upload order created but import never runs

Confirm the INGEST_TRACKING_DB_URL on both biomeroworker and biomero-importer services point to the same PostgreSQL database.

Results still appear on OMERO server storage

Confirm IMPORTER_ENABLED=true is set on the biomeroworker (not just omeroweb or biomero-importer).

PermissionError writing to .analyzed

The omero-server user (UID 1000) lacks write access. See Permissions above.

OMERO images not created after import succeeds

The shared storage must be mounted at the same path in all three containers. Check that the paths in docker-compose are consistent.

Related documentation

• BIOMERO.analyzer SLURM Integration — baseline SLURM setup

• Analyzer → Importer Integration — technical deep-dive

• BIOMERO.importer Container — BIOMERO.importer container reference

• BIOMERO Worker Container — biomeroworker container reference

• OMERO.biomero Plugin Administration — Zarr workflow type configuration in the Analyzer Admin UI

• Working with shallow OME-Zarr results — experimental shallow Zarr storage