Code Sharing
In addition to *.mat files archived according to the GOAT MAT archiving format, GOAT can be ‘linked’ to the following data sources:
- OPeNDAP sources
- NetCDF files
- Downloadable NetCDF (accessible via http or ftp)
GOAT archives that include any of the above sources are referred to as LINK archives. LINK archives follow the GOAT Auto Link Protocol (ALP) described in the GOAT_GUIDE.
CMIP5 LINK
The GOAT function GOAT_CMIP5_LINK links CMIP5 NetCDF files to GOAT. For a given main directory this function performs the following:
- Automatically finds all CMIP5 NetCDF files
- Automatically identifies variable, realm, experiment (including reanalyses), frequency, calendar (e.g., Gregorian, non-leapyear or irregular calendars such as 360 days a year), ensemble and temporal span of each file
- creates GOAT LINK archive
The LINK archive allows users to browse through the linked data using the GOAT GUI, visualize data, perform and export analysis with the same ease as data installed in *.mat format.
The automated linking capability can be further extended to observational datasets, as a growing number of observational datasets are made available at the CMIP5 archiving standard via the Obs4Mips and ana4MIPs projects.
LINKed CMIP5 archives can also be created for data stored on servers accessible via ssh, by mounting these servers using an ssh file-system utility (e.g., Mac: FUSE for OSX, Windows: Dokan for windows, Linux: FUSE and SSHFS).
The GOAT_CMIP5_LINK function is part of the GOAT CMIP5 toolbox which includes additional functions such as
- Auto parsing of CMIP5 file names into variable, experiment, realm, frequency, model, ensemble and temporal span
- Auto survey of CMIP5 files attributes (variable, model, etc.) in a given folder
- Variable filter for wget fies
OPeNDAP LINK archives
OPeNDAP (Open-source Project for a Network Data Access Protocol) is a popular data transport protocol, allowing users to inquire about and retrieve subsets of files.
Using the GOAT Auto Link Protocol (ALP), data available via OPeNDAP sources can be browsed, visualized and analyzed with same ease as *.mat installed data.
Listed below are several downloadable LINK archives. Additional LINK archives (or requests for LINK archives) can be submitted here.
| DATASET NAME/link | DESCRIPTION |
|---|---|
| ATMOSPHERIC REANALYSES | |
| 20CR | ESRL-PSD 20th Century Reanalysis (V2) (Pressure/Monolevel/Subsurface) |
| ERA-40 | By ECMWF. A partial dataset via the OPeNDAP server at Asia Pacific Data Research Center (IPRC). |
| MERRA | Entire (~80TB) MERRA dataset (excluding Diurnal datasets) via its three OPeNDAP portals: 1. http://goldsmr1.sci.gsfc.nasa.gov/dods/ (1.25° x 1.0° x 72 surface and 3D fields for chemical transport modelling) 2. http://goldsmr2.sci.gsfc.nasa.gov/dods/ (0.667° x 0.5° surface prognostic and diagnostic fields) 3. http://goldsmr3.sci.gsfc.nasa.gov/dods/ (1.25° x 1.25° x 42 analysis and assimilation fields) |
| NCEP-I | CDAS NCEP/NCAR Reanalysis via NOAA's non-operational NOMADS 3 server. The archive includes links to Rotating datasets (continuously updated most recent reanalysis output) |
| NCEP-II | NCEP-DOE Reanalysis via NOAA's non-operational NOMADS 3 server. The archive includes links to Rotating datasets (continuously updated most recent reanalysis output) |
| ERA-Interim | ECMWF ERA-Interim Reanalysis via ECMWF MARS server. The archive includes 0.75X0.75 and 1.5X1.5 degree resolutions. Monthly synoptic means are not included. Pre-requisites: 1. ECMWF API key. Follow instructions here. 2. Updated JSON package. An updated JSON package version can be downloaded from CPAN. |
| OCEAN REANALYSES | |
| ORA-S4 | By ECMWF. OPeNDAP server at Asia Pacific Data Research Center (IPRC). |
| PRECIPITATION | |
| TRMM | OPeNDAP and downloaded NetCDF file LINK archive of the TRMM datasets: a. 3B31 (version 7) b. 3B42 (version 7) |
| SURFACE TEMPERATURES | |
| ERSST (V3b) | NOAA Extended Reconstructed Sea Surface Temperatutures v3b. |