Cannot find field with valid time stamp error

[qiujx20]

Your name

Jiaxin Qiu

Your affiliation

Tsinghua University

What happened? What did you expect to happen?

GC.log
gc.run.txt
HEMCO.LOG
HEMCO_Config.rc.gmao_metfields.txt
HEMCO_Config.rc.txt
HEMCO_Diagn.rc.txt
HISTORY.rc.txt

I am running GEOS-Chem version 14.1.1 on a server and simulating from June 1, 2023. However, the model crashed on August 1, 2023. The error message in the GC.log indicates an issue with the file MERRA2.20230731.A1.2x25.nc4. I have confirmed that the simulation works without issues using GEOS-Chem version 12.3.0 on the same day, so the problem does not appear to be related to the MERRA2 data. I also re-downloaded the MERRA2 data for July 31, 2023, and August 1, 2023, and restarted the simulation from August 1, 2023, but the same error persists. My colleague is also experiencing the same issue with the simulation on the same day.

Below is the error that occurred in GC.LOG:

HEMCO: Opening /public/home/qiujiaxin/GC_14.2.1/ExtData/HEMCO/VOLCANO/v2015-02/2009/volc_so2_gen.1x1.47L.200908.nc
HEMCO: Opening /public/home/qiujiaxin/GC_14.2.1/ExtData/HEMCO/MULTI_ICE/v2021-07/multiyearice.merra2.05x0625.2020.nc
HEMCO: Opening /public/data/geoschem/ExtData/GEOS_2x2.5/MERRA2/2023/07/MERRA2.20230731.A1.2x25.nc4

HEMCO ERROR: Cannot find field with valid time stamp in /public/data/geoschem/
 ExtData/GEOS_2x2.5/MERRA2/2023/07/MERRA2.20230731.A1.2x25.nc4 - Cannot get fiel
 d ALBEDO. Please check file name and time (incl. time range flag) in the config
 . file

HEMCO ERROR: Error encountered in routine HCOIO_Read!

HEMCO ERROR: Error in HCOIO_DATAREAD called from HEMCO ReadList_Fill: ALBEDO

 --> LOCATION: ReadList_Fill (HCO_ReadList_Mod.F90)

HEMCO ERROR: Error in ReadList_Fill (4) called from HEMCO ReadList_Read
 --> L
 OCATION: ReadList_Read (HCO_ReadList_Mod.F90)
 Error in ReadList_Read called from hco_run
===============================================================================
GEOS-Chem ERROR: Error encountered in "HCO_Run"!
 -> at HCOI_GC_Run (in module GeosCore/hco_interface_gc_mod.F90)

THIS ERROR ORIGINATED IN HEMCO!  Please check the HEMCO log file for
additional error messages!
===============================================================================

===============================================================================
GEOS-Chem ERROR: Error encountered in "HCOI_GC_Run"!
 -> at Emissions_Run (in module GeosCore/emissions_mod.F90)
===============================================================================

===============================================================================
GEOS-CHEM ERROR: Error encountered in "Emissions_Run", Phase 0

This error can indicate a missing file. Please check the HEMCO log file for
additional error messages!

STOP at  -> at GEOS-Chem (in GeosCore/main.F90)
===============================================================================
real 44.27
user 305.80
sys 10.48
srun: error: comput26: task 0: Exited with exit code 159
srun: Terminating job step 28344.0

What are the steps to reproduce the bug?

I started running GEOS-Chem version 14.1.1 on August 1, 2023.

Please attach any relevant configuration and log files.

No response

What GEOS-Chem version were you using?

14.1.1

What environment were you running GEOS-Chem on?

Local cluster

What compiler and version were you using?

gcc (GCC) 4.8.5

Will you be addressing this bug yourself?

Yes, but I will need some help

In what configuration were you running GEOS-Chem?

GCClassic

What simulation were you running?

Full chemistry

As what resolution were you running GEOS-Chem?

2×2.5

What meterology fields did you use?

MERRA-2

Additional information

No response

[yantosca]

Thanks for writing @qiujx20. Could you run the following command:

$ ncdump -cts /public/data/geoschem/ExtData/GEOS_2x2.5/MERRA2/2023/07/MERRA2.20230731.A1.2x25.nc4 >a1.log

And then attach the log file here? I must want to rule out that the file itself is corrupted.

[yantosca]

@qiujx20, I noticed that you also said you are using GCC 4.8.5. This version of the compiler may be too old for GEOS-Chem Classic. You might also try a more recent compiler and see if it works.