Skip to content

Fix pipeline for large .tiff files

Current pipeline works for large .tif files but not for large .tiff files, check BIGTIGGTOTIFF process:

(base) [torchia@login2 calcium_imaging_pipeline_with_nextflow]$ nextflow run main.nf -profile embl_hd,slurm -params-file params.yml -resume -bg
Nextflow 25.04.6 is available - Please consider updating your version to it
(base) [torchia@login2 calcium_imaging_pipeline_with_nextflow]$ N E X T F L O W  ~  version 24.10.4
Launching `main.nf` [hopeful_gauss] DSL2 - revision: 6ccf9396cb
[7e/abdad0] Submitted process > CAIMAN_MOTION_CORRECTION:TESTVIDEOFORMAT (1)
[bc/b9ef5b] Submitted process > CAIMAN_MOTION_CORRECTION:BIGTIFFTOTIFF (1)
ERROR ~ Error executing process > 'CAIMAN_MOTION_CORRECTION:BIGTIFFTOTIFF (1)'

Caused by:
  Process `CAIMAN_MOTION_CORRECTION:BIGTIFFTOTIFF (1)` terminated with an error exit status (1)


Command executed:

  bigtiff_to_tiff.py "FLPGAT250423B_d3.tiff"

Command exit status:
  1

Command output:
  (empty)

Command error:
  Traceback (most recent call last):
    File "/g/gross/Sofia/nextflow_examples/calcium_imaging_pipeline_with_nextflow/bin/bigtiff_to_tiff.py", line 25, in <module>
      dtype = converted_images.dtype
    File "/usr/lib/python3/dist-packages/tifffile/tifffile.py", line 664, in imwrite
      return tif.save(data, shape, dtype, **kwargs)
    File "/usr/lib/python3/dist-packages/tifffile/tifffile.py", line 1554, in save
      raise ValueError('data too large for standard TIFF file')
  ValueError: data too large for standard TIFF file

Work dir:
  /g/gross/Sofia/nextflow_examples/calcium_imaging_pipeline_with_nextflow/work/bc/b9ef5b50ec795ea497fa6ecff3c8d7

Container:
  /g/gross/nextflow/apptainer/biocontainers-tifffile-v20181128-1-deb_cv1.img

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
Execution cancelled -- Finishing pending tasks before exit
^C
(base) [torchia@login2 calcium_imaging_pipeline_with_nextflow]$