OPeNDAP and CMIP5

LINK archives of NetCDF files and OPeNDAP sources

In addition to *.mat files archived according to the GOAT MAT archiving format, GOAT can be ‘linked’ to the following data sources:

  1. OPeNDAP sources
  2. NetCDF files
  3. Downloadable NetCDF (accessible via http or ftp)

GOAT archives that include any of the above sources are referred to as LINK archives. LINK archives follow the GOAT Auto Link Protocol (ALP) described in the GOAT_GUIDE.

CMIP5 LINK

The GOAT function GOAT_CMIP5_LINK links CMIP5 NetCDF files to GOAT. For a given main directory this function performs the following:

  • Automatically finds all CMIP5 NetCDF files
  • Automatically identifies variable, realm, experiment (including reanalyses), frequency, calendar (e.g., Gregorian, non-leapyear or irregular calendars such as 360 days a year), ensemble and temporal span of each file
  • creates GOAT LINK archive

The LINK archive allows users to browse through the linked data using the GOAT GUI, visualize data, perform and export analysis with the same ease as data installed in *.mat format.

The automated linking capability can be further extended to observational datasets, as a growing number of observational datasets are made available at the CMIP5 archiving standard via the Obs4Mips and ana4MIPs projects.

LINKed CMIP5 archives can also be created for data stored on servers accessible via ssh, by mounting these servers using an ssh file-system utility (e.g., Mac: FUSE for OSX, Windows: Dokan for windows, Linux: FUSE and SSHFS).

The GOAT_CMIP5_LINK function is part of the GOAT CMIP5 toolbox which includes additional functions such as

  • Auto parsing of CMIP5 file names into variable, experiment, realm, frequency, model, ensemble and temporal span
  • Auto survey of CMIP5 files attributes (variable, model, etc.) in a given folder
  • Variable filter for wget fies

OPeNDAP LINK archives

OPeNDAP (Open-source Project for a Network Data Access Protocol) is a popular data transport protocol, allowing users to inquire about and retrieve subsets of files

Using the GOAT Auto Link Protocol (ALP), data available via OPeNDAP sources can be browsed, visualized and analyzed with same ease as *.mat installed data.

Listed below are several downloadable LINK archives. Additional LINK archives (or requests for LINK archives) can be submitted here.

DATASET NAME/link DESCRIPTION
ATMOSPHERIC REANALYSES
20CR  ESRL-PSD 20th Century Reanalysis (V2) (Pressure/Monolevel/Subsurface)
ERA-40  By ECMWF. A partial dataset via the OPeNDAP server at Asia Pacific Data Research Center (IPRC).
MERRA  Entire (~80TB) MERRA dataset (excluding Diurnal datasets) via its three OPeNDAP portals:
1. http://goldsmr1.sci.gsfc.nasa.gov/dods/   (1.25° x 1.0° x 72 surface and 3D fields for chemical transport modelling)
2. http://goldsmr2.sci.gsfc.nasa.gov/dods/   (0.667° x 0.5° surface prognostic and diagnostic fields)
3. http://goldsmr3.sci.gsfc.nasa.gov/dods/   (1.25° x 1.25° x 42 analysis and assimilation fields)
NCEP-I  CDAS NCEP/NCAR Reanalysis via NOAA's non-operational NOMADS 3 server. The archive includes links to Rotating datasets (continuously updated most recent reanalysis output)
NCEP-II   NCEP-DOE Reanalysis via NOAA's non-operational NOMADS 3 server. The archive includes links to Rotating datasets (continuously updated most recent reanalysis output)
ERA-Interim   ECMWF ERA-Interim Reanalysis via ECMWF MARS  server. The archive includes 0.75X0.75 and 1.5X1.5 degree resolutions. Monthly synoptic means are not included.
Pre-requisites: 
1. ECMWF API key. Follow instructions here.
2. Updated JSON package. An updated JSON package version can be downloaded from CPAN.
OCEAN REANALYSES
ORA-S4 By ECMWF. OPeNDAP server at Asia Pacific Data Research Center (IPRC).
PRECIPITATION
TRMM  OPeNDAP and downloaded NetCDF file LINK archive of the TRMM datasets:
a. 3B31 (version 7)
b. 3B42 (version 7)
SURFACE TEMPERATURES
ERSST (V3b) NOAA Extended Reconstructed Sea Surface Temperatutures v3b.