Skip to content

Commit

Permalink
typos and update content for neurolibre
Browse files Browse the repository at this point in the history
  • Loading branch information
htwangtw committed May 10, 2023
1 parent ad37d37 commit 26eecfd
Show file tree
Hide file tree
Showing 15 changed files with 112 additions and 425 deletions.
4 changes: 2 additions & 2 deletions content/_config.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Book settings
# Learn more at https://jupyterbook.org/customize/config.html

title: "A reproducible benchmark of resting-state fMRI denoising strategies using fMRIPrep and Nilearn"
title: "A reproducible benchmark of denoising strategies in resting-state fMRI connectivity using fMRIPrep and Nilearn"
author: Hao-Ting Wang, Steven L Meisler, Hanad Sharmarke, Natasha Clarke, Nicolas Gensollen, Christopher J Markiewicz, François Paugam, Bertrand Thirion, Pierre Bellec
copyright: "2023"
logo: logo.png
Expand Down Expand Up @@ -36,7 +36,7 @@ repository:
html:
use_issues_button: true
use_repository_button: true
announcement: "⚠️This is a working draft for discussion and feedback!⚠️"
# announcement: "⚠️This is a working draft for discussion and feedback!⚠️"

launch_buttons:
notebook_interface: "classic" # The interface interactive links will activate ["classic", "jupyterlab"]
Expand Down
3 changes: 1 addition & 2 deletions content/_toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ root: index.md
parts:
- caption: Research report
chapters:
# - file: research_report/software_implemetation.md
# - file: research_report/benchmark_methods.md
- file: research_report/results-group.md
- file: research_report/results-atlas.md
# - file: research_report/contributions.md
Expand All @@ -18,6 +16,7 @@ parts:
- file: supplementary_materials/ohbm2022abstract.md
- caption: Workflow documentation
chapters:
- file: docs/software_implemetation.md
- file: docs/tldr.md
- file: docs/setup.md
- file: docs/fmriprep.md
Expand Down
4 changes: 2 additions & 2 deletions content/docs/fmriprep.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ If you wish to use the produced metrics to generate the book, feel free to skip
We need to build the fMRIPrep singularity containers version `20.2.1` (fMRIPrep-slurm default) and `20.2.5`.
For details, please find the documentation on [niprep](https://www.nipreps.org/apps/singularity/).
Alternatively, you can execute the script prepared `get_fmriprep.sh`.
We recommand building the container in a tmux session.
We recommend building the container in a tmux session.
This will take a few hours!
Run it before going home.

2. `preprocessing/get_datasets.sh`:

While runing Step 1 you can get the OpenNeuro datasets in parallel.
While running Step 1 you can get the OpenNeuro datasets in parallel.
Run the script in a tmux session.
This will take an hour or so.

Expand Down
2 changes: 1 addition & 1 deletion content/docs/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ If not otherwise specified, the scripts were executive from the root of the proj
If you are a member of the SIMEXP team, you can run the script as is;
otherwise, please install other software and modify the path and variables accordingly.

We recommand to understand how to create and detach a tmux session before further reading.
We recommend to understand how to create and detach a tmux session before further reading.

- [tmux cheat sheet](https://tmuxcheatsheet.com/)
- [How to leave your script running in a tmux session](https://stackoverflow.com/questions/25331471/tmux-detach-while-running-script)
Expand Down
44 changes: 44 additions & 0 deletions content/docs/software_implemetation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Software implementation

For the full report, please see our preprint.

The information here aims to provide context to understand the documentations.

## Denoising workflow

The denoising workflow is implemented through `nilearn`.
{numref}`fig-fmriprep-nilearn-denoise` presents the graphic summary of the workflow.

The Python-based workflow describes the basic procedure to generate functional connectomes from fMRIPrep outputs with a Nilearn data loading routine
(e.g., `NiftiMapsMasker` or `NiftiLabelsMasker`),
fMRIPrep confounds output retrieval function (e.g., `load_confounds_strategy`),
and connectome generation routine (`ConnectivityMeasure`).
Path to the preprocessed image data is passed to load_confounds_strategy and the function fetches the associated confounds from the `.tsv` file.
The path of an atlas and the path of the preprocessed image file is then passed to the masker, along with the confounds, for time series extraction.
The time series are then passed to `ConnectivityMeasure` for generating connectomes.


```{figure} ../images/fig-1-masker.png
---
height: 500px
name: fig-fmriprep-nilearn-denoise
---
Example workflow of extracting denoised timeseries and functional connectomes from fMRIPrep outputs using nilearn.
```

## Benchmark workflow

{numref}`fig-denoise-benchmark-workflow` presents the graphic summary of the benchmark workflow.

The denoising benchmark workflow expands on the workflow in {numref}`fig-fmriprep-nilearn-denoise` (represented by the Nilearn logo in this figure).
We retrieved the datasets from OpenNeuro through DataLad and all steps indicated with the arrows are implemented with bash scripts written for the SLURM scheduler.
Atlases were either retrieved from the TemplateFlow archive or reformatted to fit the TemplateFlow format.
The extracted time series, denoising metrics, and all metadata for generating the report are available on [Zenodo](https://doi.org/10.5281/zenodo.6941757).

```{figure} ../images/fig2-benchmark.png
---
height: 500px
name: fig-denoise-benchmark-workflow
---
Workflow of generating the full denoising benchmark.
```
4 changes: 2 additions & 2 deletions content/docs/timeseires_metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
SLURM account for job submission (default: rrg-pbellec)
```
We created two separate scripts for descrete and probability atlas due to different memory requrement.
We created two separate scripts for discrete and probability atlas due to different memory requirement.
You will find the output under:
```
/scratch/${USER}/fmriprep-denoise-benchmark/giga_timeseries/{DATASET_NAME}/{FMRIPREP_VERSION}/{UNIXTIME}/.slurm
Expand All @@ -55,7 +55,7 @@
4. `generate_metrics/*/slurm/metrics*.sh`:
Caculate metrics on denoising quality per atlas.
Calculate metrics on denoising quality per atlas.
Use this line to submit all jobs at once.
```bash
Expand Down
33 changes: 15 additions & 18 deletions content/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,26 +13,23 @@ kernelspec:

# Summary

Reducing contributions from non-neuronal sources is a crucial step in functional magnetic resonance imaging (fMRI) analyses.
Reducing contributions from non-neuronal sources is a crucial step in functional magnetic resonance imaging (fMRI) connectivity analyses.
Many viable strategies for denoising fMRI are used in the literature,
and practitioners rely on denoising benchmarks for guidance in the selection of an appropriate choice for their study.
However, fMRI denoising software is an ever-evolving field, and the benchmarks can quickly become obsolete as the techniques or implementations change.
In this work, we present a fully reproducible denoising benchmark featuring a range of denoising strategies and evaluation metrics,
and practitioners rely on denoising benchmarks for guidance in the selection of an appropriate choice for their study.
However, fMRI denoising software is an ever-evolving field, and the benchmarks can quickly become obsolete as the techniques or implementations change.
In this work, we present a fully reproducible denoising benchmark featuring a range of denoising strategies and evaluation metrics for connectivity analyses,
built primarily on the fMRIPrep {cite:p}`fmriprep1` and Nilearn {cite:p}`nilearn` software packages.
We apply this reproducible benchmark to investigate the robustness of the conclusions across two different datasets and two versions of fMRIPrep.
The majority of benchmark results were consistent with prior literature.
Scrubbing, a technique which excludes time points with excessive motion,
combined with global signal regression, is generally effective at noise removal.
Scrubbing however disrupts the continuous sampling of brain images and is incompatible with some statistical analyses,
e.g. auto-regressive modeling. In this case, a simple strategy using motion parameters,
average activity in select brain compartments, and global signal regression should be preferred.
Importantly, we found that certain denoising strategies behave inconsistently across datasets and/or versions of fMRIPrep,
or had a different behavior than in previously published benchmarks, especially ICA-AROMA.
These results demonstrate that a reproducible denoising benchmark can effectively assess the robustness of conclusions across multiple datasets and software versions.
Technologies such as BIDS-App {cite:p}`bidsapp`, the Jupyter Book {cite:p}`jupyter` and Neurolibre {cite:p}`neurolibre` provided the infrastructure to publish the metadata and report figures.
Readers can reproduce the report figures beyond the ones reported in the published manuscript.
With the denoising benchmark, we hope to provide useful guidelines for the community,
and that our software infrastructure will facilitate continued development as the state-of-the-art advances.
We apply this reproducible benchmark to investigate the robustness of the conclusions across two different datasets and two versions of fMRIPrep.
The majority of benchmark results were consistent with prior literature.
Scrubbing, a technique which excludes time points with excessive motion, combined with global signal regression, is generally effective at noise removal.
Scrubbing however disrupts the continuous sampling of brain images and is incompatible with some statistical analyses, e.g. auto-regressive modeling.
In this case, a simple strategy using motion parameters, average activity in select brain compartments,
and global signal regression should be preferred.
Importantly, we found that certain denoising strategies behave inconsistently across datasets and/or versions of fMRIPrep,
or had a different behavior than in previously published benchmarks.
These results demonstrate that a reproducible denoising benchmark can effectively assess the robustness of conclusions across multiple datasets and software versions.
In addition to reproducing core computations, interested readers can also reproduce or modify the figures of the article using the Jupyter Book project and the Neurolibre reproducible preprint server {cite:p}`neurolibre`.
With the denoising benchmark, we hope to provide useful guidelines for the community, and that our software infrastructure will facilitate continued development as the state-of-the-art advances.

![Overview of API.\label{top_level_fig}](./images/api_summary.png)

Expand Down
16 changes: 8 additions & 8 deletions content/references.bib
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
@article{biswal_2010,
author = {Bharat B. Biswal and Maarten Mennes and Xi-Nian Zuo and Suril Gohel and Clare Kelly and Steve M. Smith and Christian F. Beckmann and Jonathan S. Adelstein and Randy L. Buckner and Stan Colcombe and Anne-Marie Dogonowski and Monique Ernst and Damien Fair and Michelle Hampson and Matthew J. Hoptman and James S. Hyde and Vesa J. Kiviniemi and Rolf Kötter and Shi-Jiang Li and Ching-Po Lin and Mark J. Lowe and Clare Mackay and David J. Madden and Kristoffer H. Madsen and Daniel S. Margulies and Helen S. Mayberg and Katie McMahon and Christopher S. Monk and Stewart H. Mostofsky and Bonnie J. Nagel and James J. Pekar and Scott J. Peltier and Steven E. Petersen and Valentin Riedl and Serge A. R. B. Rombouts and Bart Rypma and Bradley L. Schlaggar and Sein Schmidt and Rachael D. Seidler and Greg J. Siegle and Christian Sorg and Gao-Jun Teng and Juha Veijola and Arno Villringer and Martin Walter and Lihong Wang and Xu-Chu Weng and Susan Whitfield-Gabrieli and Peter Williamson and Christian Windischberger and Yu-Feng Zang and Hong-Ying Zhang and F. Xavier Castellanos and Michael P. Milham },
title = {Toward discovery science of human brain function},
journal = {Proceedings of the National Academy of Sciences},
volume = {107},
number = {10},
pages = {4734-4739},
year = {2010},
doi = {10.1073/pnas.0911855107},
author = {Bharat B. Biswal and Maarten Mennes and Xi-Nian Zuo and Suril Gohel and Clare Kelly and Steve M. Smith and Christian F. Beckmann and Jonathan S. Adelstein and Randy L. Buckner and Stan Colcombe and Anne-Marie Dogonowski and Monique Ernst and Damien Fair and Michelle Hampson and Matthew J. Hoptman and James S. Hyde and Vesa J. Kiviniemi and Rolf Kötter and Shi-Jiang Li and Ching-Po Lin and Mark J. Lowe and Clare Mackay and David J. Madden and Kristoffer H. Madsen and Daniel S. Margulies and Helen S. Mayberg and Katie McMahon and Christopher S. Monk and Stewart H. Mostofsky and Bonnie J. Nagel and James J. Pekar and Scott J. Peltier and Steven E. Petersen and Valentin Riedl and Serge A. R. B. Rombouts and Bart Rypma and Bradley L. Schlaggar and Sein Schmidt and Rachael D. Seidler and Greg J. Siegle and Christian Sorg and Gao-Jun Teng and Juha Veijola and Arno Villringer and Martin Walter and Lihong Wang and Xu-Chu Weng and Susan Whitfield-Gabrieli and Peter Williamson and Christian Windischberger and Yu-Feng Zang and Hong-Ying Zhang and F. Xavier Castellanos and Michael P. Milham },
title = {Toward discovery science of human brain function},
journal = {Proceedings of the National Academy of Sciences},
volume = {107},
number = {10},
pages = {4734-4739},
year = {2010},
doi = {10.1073/pnas.0911855107},
}

@article{sporns_organization_2004,
Expand Down
Loading

0 comments on commit 26eecfd

Please sign in to comment.