Vous êtes sur la page 1sur 24

HYCOM Data Service

www.hycom.org/dataserver
An overview
Ashwanth Srinivasan, (FSU)
Steve Hankin (NOAA/PMEL)
Contributors:
Jon Callahan (Mazama Consulting)
Roland Schweitzer (Weathertop Consulting),
Ansley Manke (NOAA/PMEL)
Jeremy Malczyk (UW/JISAO)
Peter Cornillon (URI)
1

HYCOM Data Service: History and


Current Status

Data Service was launched in 2002 in


Miami and served HYCOM Outputs via FTP,
LAS and OPeNDAP servers
Started Serving Near Real Time Atlantic
data in 2003
Hosted about 3 TB of Data until Dec, 2006
in Miami.
Jan -2007 service moved to FSU and
hosted on a 100 TB SAN
Currently serving approximately 60 TB of
data

New Hardware

100 TB Fibre Channel SAN

Three 8 CPU machines

32 GB RAM/machine

Software
Red Hat Linux
Red Hat cluster suite and
Global File System
Apache Web Server
THREDDS
LAS Server
Vsftpd server
3

Operational Enhancements

All components (LAS,OPENDAP,FTP)


co-located in the same physical
machine.
Subset of files are uncompressed for
faster access
In memory, decompression by using
RAM scratch disks
High-availability architecture
minimal downtime during updates
4

Currently Available Datasets

Daily outputs from the real time Global


HYCOM+NCODA analysis (Apr 2007
current)
Global HYCOM+NCODA analysis (2004-2005
available now; 2005-2007 will be available
shortly)
Four years of Global Non-Assimilative
Simulation (2003-2007)
Near real-time 1/12 Atlantic Ocean
prediction system output (June 2003
Present)
Monthly mean 1/12 Pacific Ocean
simulation output (1978-2003)

Global HYCOM Data Serving


Challenges

Very large data/file sizes each day of


output is 12-19 G consists of 26 (15+9) 2d
and 3d variables (4500 x 3298 x 32 pts)
Curvilinear, staggered and hybrid vertical
coordinates makes analysis and visualization
on large files resource intensive and time
consuming
Need to handle model products in nonstandard native HYCOM format and in
standard format like NetCDF (B.Cs are
needed on native grids and in both NetCDF
and HYCOM IO formats)
6

HYCOM Ocean Prediction System


Data Flow
Global run at NAVO daily
5 day hindcast and
5 day forecast in
curvilinear and hybrid
coordinates

50 G pulled to FSU
~ 2-3 hrs

*Approximately 4-5 hrs of


data processing daily

Conversion to
NetCDF and
re-gridding
to Z levels

THREDDS
and
LAS Update

Means of Data Access

THREDDS
Provides a dataset catalog and OPeNDAP access
to all datasets.

Live Access Server


Visualization and downloads in different
formats, batch access etc.

OPeNDAP enabled clients


Choice of common applications or user written
programs to access data

FTP
NetCDF files for each variable

HYCOM LAS:

http://hycom.coaps.fsu.edu/las/

Provides NetCDF data in


both native HYCOM
(staggered) and regular
(non-staggered) grids
On-the-fly conversion to
fixed Z levels (but limited
to pre-configured Z levels)
tested HYCOM format IO
as a product but needs
better integration
Full resolution global
outputs not available
currently via LAS
9

Access to native coordinates and


regridded fields

10

Hybrid-Z
regridding

fixed-Z
native-Z

11

HYDAE model
intercomparison

12

13

Synthetic Data Used in the


Experiments

14

Truth and Forecasts on Day 50

15

RMS Error in 1-Day Forecasts

16

SSH 1-Day Forecast Error


Distribution

17

HYCOM specific Matlab tools from the


OPeNDAP Developers
The Matlab GUI developed for the 1/12o North
Atlantic Model Runs has been significantly modified.
Access the GUI from:

http://opendap.org/download/ml-toolbox.html

The OPeNDAP-IDL client is now robust and can be used


to access HYCOM data (no GUI yet).
Many modifications to the netCDF-OPeNDAP libraries.

Server4 - supporting GridFTP is now available.

18

The HYCOM Matlab GUI

19

Access the OPeNDAP Matlab GUIs from the


OPeNDAP download webpage

http://opendap.org/download
Or go directly to the GUI web page

http://opendap.org/download/m
l-toolbox.html
20

FTP Access
FTP://www.hycom.org/datasets

21

Access Details for June 2007

23

Community Collaborations

Working with John Caron at Unidata


on testing the Forecast Aggregation
server with large operational
datasets
Building a HYCOM .[ab] format IO
service provider (ISOP) to be
incorporated into the JAVA NetCDF
2.2 library
Partnership with IOOS data
management Caucus

24

Other Addition/Improvements

Forcing data
Reference observational data sets
A completely redesigned website is
being planned along the lines of
GODAE server
Searchable mailing list, FAQs
Links to other data repositories
25