Hierarchical downsampling #46
Conversation
Non-homogeneous way to performa downsampling of input image according to the sigma to noise ratio. Optimal macropixel dimension is thus explored in a tree search method according to the distribution of signal intensities (a shapiro test is implemented, different analysis are planned to be included in future work). Three different exit condition are implemented: - when at least N nodes in the same tree branch are proved to be normally distributed - when N consecutive nodes are proved to be normally distributed - when the fraction of normally distributed children is higher to a certain threshold NB The output of this block is compliant to all blocks in stage 03 but it is not yet usable to run stage04 and stage05 analysis. Allignment for at least some of the methods implemented at these two further stages are planned to be implemented in the next steps.
rename variabple to a more coherent way.
center of mass of non filled macropixels is computed only according to non-nan channels and saved in the neo analogsignal array annotations.
Add utils is not a parameter defined in the config file, Snakefile updated accordingly.
There was a problem hiding this comment.
I just had a quick read over the code.
The main challenge remains to be that the pipeline stages and blocks need to have well-defined inputs and outputs, and this block - if applied - changes the structure of the stage output, which causes errors for various combinations of block and stage executions. Therefore, to include this method, it needs to either generate an output consistent with the stage 02 (i.e. channels arranged on a grid) or it needs to be branched off. The off-branching could be for example in the form of a new stage, following stage 02, but which can't be followed by stage03 but instead e.g an adapted stage 05.
| # 'spatial_downsampling', 'roi_selection', 'logMUA_estimation', 'phase_transform', | ||
| # 'zscore', 'subsampling' | ||
| # 'spatial_downsampling', 'hierarchical_spatial_sampling', 'roi_selection', 'logMUA_estimation', 'phase_transform', | ||
| # 'zscore', 'subsampling', 'smart_spatial_downsampling' |
There was a problem hiding this comment.
what is smart_spatial_downsampling?
| def next_power_of_2(n): | ||
| if n == 0: | ||
| return 1 | ||
| if n & (n - 1) == 0: | ||
| return n | ||
| while n & (n - 1) > 0: | ||
| n &= (n - 1) | ||
| return n << 1 |
There was a problem hiding this comment.
This function is really hard to read and understand. It could be probably just replaced by
exponent = np.ceil(np.log2(n))
return int(2**exponent)
| NewImage = np.empty([np.shape(original_img)[0], np.shape(original_img)[0]])*np.nan | ||
| for macro in MacroPixelCoords: | ||
| # fill pixels belonging to the same macropixel with the same signal | ||
| NewImage[macro[0]:macro[0] + macro[2], macro[1]:macro[1] + macro[2]] = np.mean(np.nanmean(original_img[macro[0]:macro[0] + macro[2], macro[1]:macro[1]+macro[2]], axis = (0,1))) |
There was a problem hiding this comment.
this line needs to be separated into multiple reasonable expressions
| config.ADD_UTILS = f"export PYTHONPATH='$PYTHONPATH:{utils_path}'" | ||
| ADD_UTILS = f"export PYTHONPATH='$PYTHONPATH:{utils_path}'" | ||
| #config.ADD_UTILS = f"export PYTHONPATH='$PYTHONPATH:{utils_path}'" |
There was a problem hiding this comment.
This is a change that should be consistent throughout all rules and stages, i.e. should not be part of this PR
There was a problem hiding this comment.
Yeah, sorry, must have done some error in the cherry-pick process. This was not supposed to be here.
| CLI.add_argument("--output_array", nargs='?', type=none_or_str, | ||
| help="path of output numpy array", default=None) |
HIerarchical sampling name update coherently: 1.Remove "Smart-downsampling" (obsolete HOS version) from usable blocks applicable in stage 02 listed in the config_template file. 2. HOS block description in README
Changes in the Snakefile of stage05_wave_characterization have been removed. Not involved in this PR
lines 111-114 have been separated into multiple reasonable expressions
lines 111-114 have been separated into multiple reasonable expressions
x and y coordinates are now coherently defined solving IndexErrors in subsequent stages. Conflicts: pipeline/stage02_processing/scripts/hierarchical_spatial_sampling.py
Conflicts: pipeline/stage02_processing/scripts/hierarchical_spatial_sampling.py
Hierarchical spatial sampling implemented in stage 02.