-
Notifications
You must be signed in to change notification settings - Fork 62
Transfer grib2 files under UPP to Jet Hera for the MATS verification
# go to the COMROOT
cd /gpfs/f6/bil-fire10-oar/world-shared/gge/OPSROOT/cyc_noDA/com/rrfs/v2.1.0
tar -cvf cyc_noDA.tar rrfs.*/*/upp
The tarball file is around 114 GB. There are a few ways to transfer the file. But it is recommended to use Globus. You may find the Globus instruction at the RDHPCS wiki here
cd cyc_noDA
tar -xvf cyc_noDA.tar
rm cyc_noDA.tar
Follow the Upper Air/ RAOBs, surface, Upper Air/ AMDAR, SURFRAD, C&V(ceiling and visibility), and MRMS QPE instructions to generate verification statistics and load them into the MATS system.
The following steps are examples for the verification of the above experiment (be sure to replace it with your experiment name).
4. Sudo to role.amb-verif, enter corresponding directories, create links, and then run verification scripts.
You will need to provide the start and end epochs (or start and end seconds since a base epoch) for the scripts.
The base epoch used here is January 1, 1970, at 00:00:00 UTC.
To compute the correct epochs for your retro, follow the examples here:
date -d '2024-05-06 00:00:00 UTC' +%s
1714953600
date -d '2024-05-12 23:00:00 UTC' +%s
1715554800
Now run the following commands (replace retroDIR and expName with yours):
(note: role.amb-verif uses tcsh)
sudo su - role.amb-verif
set retroDIR="/lfs6/BMC/wrfruc/gge/MATS_verification/cyc_noDA"
set expName="RRFSv2_conus_12km_ret_May2024_my_exp_name"
set start_epoch=1714953600
set end_epoch=1715554800
# Upper Air Radiosonde
cd ~role.amb-verif/RR_uaverif/RR_retro
ln -snf ${retroDIR} ${expName}
cd ..
process_retro4.pl ${expName} ${start_epoch} ${end_epoch} FW-to-Wobus 1
# Surface
cd ~role.amb-verif/ruc_madis_surface/RR_retro
ln -snf ${retroDIR} ${expName}
cd ..
retro2.pl ${expName} ${start_epoch} ${end_epoch} 1
# Aircraft
cd ~role.amb-verif/acars_RR/retro_runs
ln -snf ${retroDIR} ${expName}
cd ..
cp fcst_lens_template ${expName}_fcst_lens
./gen_retro_acars_stats3.pl ${expName} ${start_epoch} ${end_epoch} 37 1
./load_acars_model_data_retro2.pl
# SURFRAD
set max_forecast_hours=12
cd ~role.amb-verif/surfrad3/retro
ln -snf ${retroDIR} ${expName}
cd ..
./gen_surfrad.sh ${start_epoch} ${end_epoch} 1 ${max_forecast_hours} ${expName}
# Ceiling
/home/role.amb-verif/ceiling/get_ceil_obs_retro.pl ${start_epoch} ${end_epoch}
cd /home/role.amb-verif/ceiling/retro_data
ln -snf ${retroDIR} ${expName}
cd ..
./ceil_driver_retro.pl ${expName} ${start_epoch} ${end_epoch} 1
#./ceil_driver_retro_exp_diag.pl ${expName} ${start_epoch} ${end_epoch} 1 #experimental ceiling diagnostic #1
#./ceil_driver_retro_exp_diag2.pl ${expName} ${start_epoch} ${end_epoch} 1 #experimental ceiling diagnostic #2
# Visibility
/home/role.amb-verif/visibility/get_vis_obs_retro.pl ${start_epoch} ${end_epoch}
cd /home/role.amb-verif/visibility/retro_data
ln -snf ${retroDIR} ${expName}
cd ..
./retro_driver.tcsh ${expName} ${start_epoch} ${end_epoch} 1
~/mats_metadata/update_metadata_upperair.py ${expName}
~/mats_metadata/update_metadata_surface.py ${expName}
~/mats_metadata/update_metadata_aircraft.py ${expName}
~/mats_metadata/update_metadata_surfrad.py ${expName}
~/mats_metadata/update_metadata_visibility.py ${expName}
~/mats_metadata/update_metadata_ceiling.py ${expName}
and then visit the following urls to refresh corresponding MATS metadata:
UpperAir, Surface, SURFRAD, C&V and MRMS QPE
Now you should be able to see your experiment name in the https://gsl.noaa.gov/mats/ data source drop-down list.
6. Log your experiment in the RRFSv2 retro Google sheet if you have not done so.
FYI, here is the MAIN google sheet RRFSv2 retro Google sheet and here is the RRFSv2 retro planning documentation