13.2 GIDB METOC Data Integration
In this section we describe the integration of Meteorological and Oceanographic (METOC) data into the GIDB System.
13.2.1 Background
METOC data is remotely retrieved via Metcast (DII 2001), which is a system for the distribution of METOC data to the navy developed by the Fleet Numerical Meteorology and Oceanography Center (FNMOC). Metcast is a client/server architecture in which the Metcast server application serves data from a METOC database containing a variety of products, including satellite imagery and model outputs of weather and ocean conditions describing the current and future state of the environment around the globe. The server publishes a catalog of currently available products. This catalog is in XML format and describes the database's hierarchy of products and an appropriate set of metadata parameters for each product. A Metcast client may request a catalog from any known Metcast server and then, using the catalog to identify desired products, request those products. Products are requested using a Metcast-specific request language, the Metcast Broker Language (MBL), which, although not XML itself, closely mirrors the structure of the XML catalog. The server returns the requested products to the client in their native formats.
The products available from the Metcast server are arranged according to the Metcast data model?that is, they're arranged hierarchically, grouping products into sets and subsets as shown in Figure 13.2. A real catalog would typically be populated with a much larger selection of products.
Figure 13.2. Products Arranged According to the Metcast Data Model
The products are grouped first according to type, whether they are satellite images or grids. Gridded products are outputs from meteorological and oceanographic models, which describe current and future conditions of the atmosphere or oceans by describing their state at gridded intervals over some area. Images are grouped according to area and then, within that, are described by the specific product type such as an infrared or a visible image. Grids are first grouped by the production center (the agency that generated it), then by the model that produced it, and then by the particular parameter it describes such as winds or temperature. Finally, grids are fully specified by the specific level and tau. Level represents the specific altitude within the atmosphere, and tau represents the number of hours into the future relative to the time of the model run that the grid describes.
The Metcast catalog is hierarchically structured according to this scheme. It describes elements corresponding to the nodes of this hierarchy with their appropriate attributes as specified in the Metcast Catalog Document Type Definition (DTD). For example, the model element has associated attributes describing its resolution in the x and y directions, and the parameter element has an attribute specifying the units.
When a user selects a product or set of products, the Metcast client software must generate an MBL request specifying the product or products. MBL is a relatively simple text-based request language in which each request is in the form of an S-expression (DII 2001). Each request specifies the product or products by designating an area of interest for which a data product is desired and a set of attributes fully describing the desired product for that area.
13.2.2 Implementation
In the existing Metcast system, a standalone Metcast client application uses Metcast's protocols for acquiring data products from the remote server. Our goal was to add Metcast connectivity to the GIDB System, extending its functionality so that it behaved as a Metcast client, allowing for the seamless integration of METOC data. The user would be able to browse up-to-date information on atmospheric and oceanic conditions side by side with data extracted from other local and remote geospatial data sources. To accomplish this we developed a GIDB Metcast driver, essentially a software bridge between the two systems. The GIDB System requires a standard interface to access any one of its data stores. The driver provides this common interface, hiding the underlying details of how the data source is accessed.
The general process of acquiring products from Metcast (or any other data source) is illustrated in Figure 13.3 and involves the following:
Available products are discovered (a catalog is acquired from Metcast).
Desired products are requested.
Requested products are delivered.
Figure 13.3. Complete Set of Steps to Acquire a Data Product from Metcast
During this process, the Metcast driver has three principal translations to perform:
To translate the organization of products in the Metcast catalog into the organizational form in which they are delivered through the GIDB portal
To translate GIDB mapping application requests for products to Metcast's MBL requests for products
To translate the resulting products from the native formats in which they are delivered to the GIDB portal's common data transfer format
Since the Metcast catalog is an XML document, we will consider the first translation in detail. We will also touch upon the second. As Metcast data products are returned in a non-XML format, we will not discuss the third translation here.
First, let us consider the translation from the Metcast catalog to the GIDB portal's catalog. This portal's catalog structures data products into the following hierarchy: Database?Library (or scale)?Coverage (or thematic layer)?Feature Class?Feature. An example of this structure is shown in Figure 13.4.
Figure 13.4. Metcast Data Products Organized According to the GIDB Data Model
The GIDB mapping application allows the user to select products of interest by navigating through the hierarchy, selecting a database, then selecting a library, and so on, down to individual products. For databases that the GIDB System maintains locally within Ozone, the data can actually be structured in this hierarchy. For Metcast, however, we have no control over how the data are structured. A catalog is received and organized according to the Metcast structure. Therefore, the GIDB Metcast driver must transform this hierarchy into the GIDB hierarchy. The user navigates the new GIDB hierarchy until a product is selected.
Translating the Metcast catalog is a two-step process. Since good third-party XML parsers already exist, we do not wish to directly parse the XML catalog ourselves. Instead we use the Apache Xerces XML parser to generate a Document Object Model (DOM) tree. The DOM tree is a generic object-oriented Java tree embodying the same structure and information as the original XML document. This DOM tree forms the input to our mapping. The driver applies a series of transformations to the tree that navigate down through its hierarchy. As the navigation proceeds, a new GIDB catalog tree is constructed, embodying the same information but in a new structure. The new tree could also be a DOM tree. In fact, for convenience's sake, we developed our own tree node objects with convenient helper methods, including methods useful for navigating geospatial data (such as queries for the minimum-bounding-rectangle circumscribing the data under a node). Figure 13.5 shows an example of the DOM-to-GIDB mapping process. The XML sample is slightly simplified from the original. The actual catalog contains some additional information that is not relevant to our purposes.
Figure 13.5. Mapping from a Catalog Entry to a DOM Tree Branch to a GIDB Tree Branch
For example, the Center element of the Metcast DOM tree is mapped to a Database node in the GIDB tree. The database node is given the center ID and Name attributes from the Center. When the driver navigates down to a DOM tree Model node, a new GIDB Library node is built. The Grid ID, Process ID, and resolution attributes from the original Model are added to the new Database. This process continues down each level of the tree and continues until each branch of the DOM tree has been navigated down to each leaf and a completely new GIDB tree has been constructed representing the equivalent information. This new tree is the GIDB catalog that the Metcast driver will use to inform the GIDB mapping application user of the products that are available.
Once the user has navigated through the GIDB catalog tree and identified a particular product of interest, another translation must occur. This translation entails taking the user's selection from the GIDB catalog tree and constructing an MBL request that is meaningful to the Metcast system. The selection of a product in the GIDB catalog tree is the request, and the complete branch that leads to this product is the full specification of the product and forms the basis of the translation from the GIDB request to the Metcast MBL request (see Figure 13.6). The "bounding-box" in the MBL request is the area for which data is requested. It must be a subset of the boundaries of the requested product.
Figure 13.6. Mapping from a GIDB Tree Branch to an MBL Request
When generating the MBL request, the driver traverses the given branch from its leaf node (the product) to the root, extracting relevant attributes about the product and using them to construct a request. We will not go into the details of the S-expression syntax of the MBL statement, but a quick visual inspection (see Listing 13.1) shows that it embodies information in the original XML document describing the product (see Listing 13.2).
Listing 13.1 MBL Request
(GIDB_Request
(products
(grib
(source 58)
(process-id 82)
(grid-id 20)
(resolution 0.2)
(product-GRIB-code 61)
(layer surface surface)
(tau 12)))
(bounding-box 32.152206 -92.98574 28.03591 -85.4269))
Listing 13.2 Original Catalog Entry
<GRIDS Timestamp="1003338720">
<CENTER
NAME="US Navy - FNMOC"
CENTERID="58">
<MODEL
NAME="COAMPS_CONUS"
GRIDID="20"
PROCID="82"
BOUNDARIES="50.0,-126.0,24.0,-66.0"
XRESDEG="0.200000"
YRESDEG="0.200000">
<PARAMETER ID="61"
NAME="12hr Precipitation">
<LEVELS>
<LE TYPE="surface">surface</LE>
</LEVELS>
<TAUS>
<TE>12</TE>
</TAUS>
</PARAMETER>
</MODEL>
</CENTER>
</GRIDS>
The request is sent to the server, which in turn delivers the appropriate product back to the client. This last step completes the process for retrieving a product. Figure 13.7 shows a display screen of data retrieved through this process. The data in the figure was retrieved via Metcast and displayed in the GIDB application.
Figure 13.7. A View of the Winds off the West Coast of France
The Metcast catalog in XML provides fundamental information about the available products and their organization, forming the basis of the product retrieval process. The GIDB Metcast driver uses this information, transforming it into another scheme. Products are selected and mapped back to Metcast's scheme. These translations all occur behind the scenes, allowing a user of the GIDB System to access Metcast data without any knowledge of the disparate data models and protocols.
| Top |
