Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

export_sourcedir plugin #304

Open
pvandyken opened this issue Jun 2, 2023 · 2 comments
Open

export_sourcedir plugin #304

pvandyken opened this issue Jun 2, 2023 · 2 comments
Labels
enhancement New feature or request plugin Feature suitable for, or an issue with, a snakebids plugin

Comments

@pvandyken
Copy link
Contributor

This proposal will supersede #180 .

The plugin will allow the copying or symlinking of directories in the app to the output dir. This will allow the snakemake app to access contained files via relative files, solving the provenance issues caused by reinstalling the app in a different location

def export_sources(
    paths: str | Path | list[str | Path],
    *,
    root: str | Path = ".",
    timestamp: bool = True,
    checksum: bool = False,
    version: str | tuple[str, str, str] | packaging.version.Version | None = None,
    symlink: bool = False,
):
    """Recursively copy given directories and files into the output dir.

    Allows workflow to access files in the directories via relative paths. All exported
    paths must be subdirectories of the root, which defaults to the snakemake_dir given
    to SnakeBidsApp. Paths will be normalized relative to the root, then exported as-is.

    If exported files are already in the output dir, the export will only be performed
    on outdated files. By default, outdated files are determined by comparing the mtime
    of the previously exported file to the file in the app. If a ``version``
    specifier is used, checks for outdated files will only be made when the version 
    changes.

    Parameters
    ==========
    paths :
        Paths of directories and files to export. Directories are exported recursively.
    root :
        Root of exported directories. All paths will be exported as relative to this
        root.
    timestamp :
        Allow the default behaviour of updating exports whenever the exported copy is
        older than the file to be exported. Defaults to ``True``.
    checksum :
        Find outdated exports by computing the file checksum. This is the most
        robust solution to deal with snakemake's automatic provenance, but may be slow
        for large files. If set, this takes precendence over ``timestamp``.
    version :
        A version string, tuple, or ``Version`` object to be recorded with the output
        data. Outdated exports will be updated only when the version changes.
    symlink :
        If ``True``, symlinks will be made to exported files, rather than copying. This
        method is faster when large files are being exported, but may not be compatible
        with some distributed modes of snakemake.
    """
@pvandyken
Copy link
Contributor Author

I think this feature would be quite useful, but not a priority before 1.0.

@pvandyken pvandyken added enhancement New feature or request plugin Feature suitable for, or an issue with, a snakebids plugin labels Jun 2, 2023
@pvandyken
Copy link
Contributor Author

See also #109, use of a plugin like this would probably supersede any reference to those variables

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request plugin Feature suitable for, or an issue with, a snakebids plugin
Projects
None yet
Development

No branches or pull requests

1 participant