Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust initialization of the total quality index (qi_total) #41

Open
guntherhaase opened this issue May 22, 2024 · 1 comment
Open

Adjust initialization of the total quality index (qi_total) #41

guntherhaase opened this issue May 22, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@guntherhaase
Copy link
Contributor

Currently, qi_total is initialized with "0". If no additional quality control is applied, this prevents e.g. NWP from using the corresponding quantity. Hence, qi_total needs to be initialized with "1".

@guntherhaase guntherhaase added the enhancement New feature or request label May 22, 2024
@MikoKNMI
Copy link

MikoKNMI commented Jun 7, 2024

For NIMBUS it is now SOLVED and being TESTED at GeoSphere.

SOLUTION for qi-total problem in BALTRAD

In NIMBUS we have identified and solved a serious bug in BALTRAD.
The primary problem has been identified here: rave_qitotal_quality_plugin.py

This file has not been touched in the last 5 years.
https://github.com/baltrad/rave/blob/master/Lib/rave_qitotal_quality_plugin.py

For NIMBUS we had to modify this file this way and this is the solution:

https://github.com/opera-radar/opera-nimbus-test-env/blob/master/toolbox/nimbus-rave/Lib/rave_qitotal_quality_plugin.py

This is the full commit:
https://github.com/opera-radar/opera-nimbus-test-env/commit/de0ec35cdd60cd1f194b5ec23439d9ad4a28c0e0#diff-44a71e4224fa00d2e658ebb0b790812034e98bca00e1cc0b51ff7d6ae7d8c5e4

This is the problematic spot:
https://github.com/opera-radar/opera-nimbus-test-env/commit/de0ec35cdd60cd1f194b5ec23439d9ad4a28c0e0#diff-44a71e4224fa00d2e658ebb0b790812034e98bca00e1cc0b51ff7d6ae7d8c5e4L85

    if nr_qitotal_fields > 0:
      if hasattr(qitotal, QITOTAL_METHOD):
        method = getattr(qitotal, QITOTAL_METHOD)
        result = method(qitotalfields)
        scan.addOrReplaceQualityField(result)

Now ALL NIMBUS QC-PVOL files contain qi-total quality field. Even the radar data from sources which are only treated with "distance,scansun,qi-total".
Note that "distance,scansun" filters do leave traces (quality fields) in the HDF data so qi-total filter does not see them and the BALTRAD BUG was not creating even an empty qi-total quality field.

This is now SOLVED.

Testing details below.

  1. Testing node=plgda; QC-FILTERS based on modules_by_radar.xml: <plgda modules="distance,scansun,qi-total"></plgda>
    Quality-fields: node=plgda; scan_qc_list=[] before computing qi-total.

  2. Testing node=nosta; QC-FILTERS based on modules_by_radar.xml: <nosta modules="distance,scansun,qi-total"></nosta>
    Quality-fields: node=nosta; scan_qc_list=['no.met.prorad.qi_total', 'no.met.prorad.beamblockage'] before computing qi-total.

  3. Testing node=nldhl; QC-FILTERS based on modules_by_radar.xml: <default modules="distance,scansun,hac-increment,ropo,beamb,satfilter,hac-filter, qi-total"></default>

    Quality-fields: node=nldhl; scan_qc_list=['fi.fmi.ropo.detector.classification', 'se.smhi.detector.beamblockage', 'fr.mf.satfilter'] before computing qi-total.

Full testing-report can be found here:

https://github.com/opera-radar/opera-nimbus-test-env/blob/master/testing/NIMBUS-qc-qi-total-prod-testing-2024-06.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants