Regression: btrfs driver stopped to work on Fedora 35 Workstation

[PavelSosin-320]

btrfs driver failed to create container's artifact on Fedora 35 workstation with error:
podman create docker.io/theiaide/theia
Error: error creating container storage: error creating read-write layer with ID "8ae14a5f8ca628b446710af2a5d0b5864b9de8066e519eb9574c063190b360a0": stat /home/pavelsosin/.local/share/containers/storage/btrfs/subvolumes/a549a6e957e2fa8db7ecfe3741211453b76c0f7809d79e5871ae9566cb4c7b71: no such file or directory

or any other container ...
The possible reason is wrong calculation of the absolute storage root based on environment variables. The "home" area /home/username is created and managed by the Homed service and mounted dynamically on GUI session logon. If homed is ignored significant portion of systemd functionality is missed and GNOME "Terminal application" and all CLI tools running inside GNOME terminal stop working correctly.
Human users created with help of homed service and services running via Systemd units can't determine user's home area path correctly. The systemd-homed service exposes its interfaces via varlink API. According my test on the "native" Fedora 35 and WSL image Homed works correctly and uniformly hiding underlying FS differences.
The necessary user information can be achieved very reliable using userdbctl and its varlink API:
userdbctl user -j pavelsosin
{
"userName" : "pavelsosin",
"uid" : 1000,
"gid" : 1000,
"realName" : "Pavel Sosin",
"homeDirectory" : "/home/pavelsosin",
"shell" : "/bin/bash",
"privileged" : {
"hashedPassword" : [
"xxxxxxxxxx"
]
},
"passwordChangeNow" : false,
"lastPasswordChangeUSec" : 1629936000000000,
"passwordChangeMaxUSec" : 8639913600000000,
"passwordChangeWarnUSec" : 604800000000,
"status" : {
"51283388c010465db40092ad75fe21af" : {
"service" : "io.systemd.NameServiceSwitch"
}
}
}
Please, pay attention to the words in the documentation:
home-dir=PATH, -dPATH
Takes a path to use as home directory for the user. Note that this is the directory the user's home directory is mounted to while the user is logged in. This is not where the user's data is actually stored, see --image-path= for that. If not specified defaults to /home/$USER. Systemd-homed service can mount it according its own configuration. In btrfs filesystem where /home is subvolume and home area is portable "Homedir" can be located elsewhere.
P.S. Please track btrfs TODO Fedora workstation

[PavelSosin-320]

I've carried checks of SELinux files and directories labelling of the the Human user home in my Fedora 35 and everything meet my expectations learned from the Fedora's SELinux documentation and the article about SELinux on Fedora written by Dan Walsh.
Also the creation of read-write subvolume by home area owner itself is not an issue:
_btrfs subvolume create $HOME/.local/share/containers/storage/btrfs-containers/svHerman
Create subvolume '/home/pavelsosin/.local/share/containers/storage/btrfs-containers/svHerman'
stat /home/pavelsosin/.local/share/containers/storage/btrfs-containers/svHerman
File: /home/pavelsosin/.local/share/containers/storage/btrfs-containers/svHerman
Size: 0 Blocks: 0 IO Block: 4096 directory
Device: 48h/72d Inode: 256 Links: 1
Access: (0775/drwxrwxr-x) Uid: ( 1000/pavelsosin) Gid: ( 1000/pavelsosin)
Context: system_u:object_r:unlabeled_t:s0
Access: 2021-12-20 19:38:32.994291055 +0200
Modify: 2021-12-20 19:38:32.994291055 +0200

Change: 2021-12-20 19:38:32.994291055 +0200
Birth: 2021-12-20 19:38:32.994291055 +0200_
But comparing btrfs driver code and man Stat(2) documentation and the source code of the btrfs driver I I'm questioning if the error message
Error: error creating container storage: error creating read-write layer with ID "14f53c321e5943ad6cb88584522cc540642614a4aac2ca8de65f900721cc8e6b": stat /home/pavelsosin/.local/share/containers/storage/btrfs/subvolumes/9f54eef412758095c8079ac465d494a2872e02e90bf1fb5f12a1641c0d1bb78b: no such file or directory
that comes from the stat call can be caused by the wrong choice of the /f/l .. stat or insufficient privileges on the created subvolume path to get stat information? Maybe, Stat of the subvolume has to be the same as directory stat or caching has to be suppressed?
Delegation to Overlay graphgdriver will obviously fails because graph driver expects SELinux label "system_u:object_r:container_file_t:s0". It may never be achieved.

[PavelSosin-320]

BTRFS has been significantly upgraded in Fedora 36 WS. btrfs-fuse appears for example. Who handles this issue can freely close it due to EOL and create the new one based on podman 4 testing. Now some files inside rootless user storage have not labels at all. P.S. Fedora 36 WS is one of 2 most popular Linux workstation distros today according to number of references in Youtube, please!

[rhatdan]

This issue is fixed in podman 4.1