Clean up straylight due to residual cosmic ray shower spots

[stscijgbot-jp]

Issue JP-3830 was created on JIRA by David Law:

One of the limiting issues for MIRI MRS performance at faint signal levels is the large-scale blotchiness caused by residual cosmic ray showers that are too low-level to be detected at the group stage.

Left panel of splotches1.png attached shows such a Ch1A/2A detector image after pixelwise subtraction of a dedicated background; the negative/positive residuals can be tens to hundreds of pixels across, and act as a strongly correlated scattered light signal that can artificially produce features in the extracted spectra similar to faint extended sources (left panel of splotches2.png).

For sufficiently faint targets though, it may be possible to use the non-science inter-slice pixels to build a map of the splotches to be subtracted from the science data.  Right panel of these two figures suggests that this approach has significant promise to warrant further investigation.

Such a correction step would most logically happen in the straylight step, an MRS-only step already designed to remove straylight components from the data.  This location is ideal since it is after pixel-based background subtraction and self-calibrated bad pixel correction, and prior to application of the flatfield that loses the inter-slice pixels.

Initial tests use the astropy 2d gaussian convolution method with NaN-valued science pixels, and should reject some high/low percentile of the non-science pixels to avoid bad pixel propagation when the custom bad pixel mask has not been performed.  Will likely require the straylight step to read the 'regions' reference files.

Assigning to David Law for further investigation.

[stscijgbot-jp]

Comment by David Law on JIRA:

Additional testing looks very successful; experimenting with a very low surface brightness lensed PAH (Spilker+23) that previously required significant out-of-pipeline tweaking of the cube.

rateim.png illustrates the cleanup of background-subtracted rate images, and b113.png the difference between pipeline reductions for the lensed PAH before/after this change.

[stscijgbot-jp]

Comment by David Law on JIRA:

Initial branch implementation: https://github.com/drlaw1558/jwst/tree/JP-3830

[stscijgbot-jp]

Comment by David Law on JIRA:

Note this will also need an update to the MRS regions files to include the trimmed wavelength ranges; potentially as negative slice numbers to avoid confusing the WCS routine.