singularity data package symlink handling

[nathanweeks]

Before you report an issue...

• Have you searched for a duplicate report?
• Have you replicated the problem on the latest release?
• Have you checked the latest documentation?
• Are you using an up-to-date system?

Only issues that can be replicated on the latest release, or development branch, of SingularityCE will be investigated and fixed. The open source project does not maintain long-term stable branches or fix bugs in prior versions. If you require LTS support then please see the Sylabs website.

Version of Singularity

What version of Singularity are you using? Run:

$ singularity --version
singularity-ce version 4.2.2-1.el8

Describe the bug

The behavior of singularity data package with respect to symbolic links may be counterintuitive (see below). If this is intended, it would be helpful to document, and perhaps introduce command-line options to alter this behavior.

To Reproduce

When creating a data container from a directory with singularity data package, symlinks to files are followed, with the target file included in the data container:

$ tree dir1
dir1
├── singularity -> /usr/bin/singularity
└── test.1g

0 directories, 2 files
$ singularity exec --oci --data dir1.oci.sif:/dir1  alpine_latest.oci.sif ls -hs /dir1
...
total 1G    
  44.5M singularity    1.0G test.1g

Symlinks to directories result in an empty directory in the data container; the contents of the directory that was linked to are not includedr:

$ tree dir2
dir2
└── dir1 -> ../dir1

1 directory, 0 files
$ singularity data package dir2 dir2.oci.sif
INFO:    Converting layers to SquashFS
$ singularity exec --oci --data dir2.oci.sif:/dir2  alpine_latest.oci.sif ls -hs /dir2
INFO:    System configuration does not support cgroup management - starting container in current cgroup
INFO:    No /run/user session directory for user. Using "/tmp/singularity-oci-14516" for runtime state.
total 0    
      0 dir1

If a symlink points to a non-existent path, singularity data package errors:

$ tree dir3
dir3
└── apptainer -> /usr/bin/apptainer

0 directories, 1 file
$ ls /usr/bin/apptainer
ls: cannot access '/usr/bin/apptainer': No such file or directory
$ singularity data package dir3 dir3.oci.sif
FATAL:   open apptainer: no such file or directory

Expected behavior

I initially expected any symbolic links to be added to the data container as symbolic links (i.e., not followed). But if the current handling is correct, it would be useful to have an option to explicitly disable following symlinks (especially if creating a data container from a directory that other users have write access to). This option could possibly be made consistent some other command-line utility (e.g., cp -P / cp --no-deference, rsync -l / rsync --links),

OS / Linux Distribution

$ cat /etc/os-release
NAME="Rocky Linux"
VERSION="8.9 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.9"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.9 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.9"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.9"

Installation Method

RPM from EPEL

Additional context