|Copyright © 1997 - 2019 Science Tools Corporation All rights reserved|
NASA's EOSDIS and The BigSur System
The context of this paper is to articulate the vision of Science Tools' BigSur System as a successor to the existing EOSDIS Core System (ECS). It should be understood that BigSur was designed specifically with this job in mind and that it has been in production service at a NASA DAAC since September, 1997 with a perfect service record. This isn't to say that installation is taking advantage of the full capabilities of BigSur - it isn't , precisely because the existing ECS is the paradigm presently embraced by NASA HQ.
We posit that what is needed is a robust system to interconnect researchers at a deep level, utilizing the information management capabilities of database systems to good effect. This system's core responsibility is to manage collection and dissemination of the meta-data regarding the enabling infrastructure necessary to permit research systems to be connected using the latest technologies available, to facilitate collaborative interaction and to provide for high-quality access by the lay public.
One of the greatest challenges of assembling a most useful means of connecting established researchers and research centers is overcoming the cacophony of voices of individualism which inherently emanate from each participant. The information systems these participants create are tailored to specific problems and are in and of themselves not designed to be adapted to solve the problems of others, much less the more general problem of unifying a community, leaving as unthinkable and hardly imaginable a system which can unify NASA's Earth System Science Research as a whole. Yet it is becoming clear that this is precisely the need.
To address this challenge and create an information system which binds disparate researchers in a cohesive, unifying environment, the system must embrace fundamental diversity as an asset. The answers to the questions of unification are found in meta-data, without which the entire enterprise is not possible. Yet, the existing ECS cannot answer these questions because V0 and related implementations were not designed to unify a community and too many details are left as assumptions and presumptions. We posit that what is needed is an Adaptive-Orientation which converts diversity into an asset to be exploited by learning descriptions and features of existing strategies and paradigms. The system must not chase every new advancement in data access methods, such as new HDF formats, but rather "learn" (or absorb) those favored by researchers as they are introduced. The system must not embed technology to manipulate or interpret data formats, rather, it should record what data types exist, which objects are of those types, what tools exist to handle them, and possibly automate some conversion routines so that new formats are easily absorbed.
Site-independence of data-objects from meta-data provides a natural ability to cooperate in a distributed environment, and provide for sharing of workload between systems. BigSur's Distributed Processing System permits Scientific functions to be performed on any system in the network, as desired, with both process-level and system-level controls for staff to manage workload. Site independence also means that individual installations can have broad knowledge in their meta-data about resources existing elsewhere.
What we are proposing is at least one core installation of BigSur devoted to unifying purposes, working in conjunction with other BigSur installations and other information systems, whenever appropriate. The goal is not to replace existing systems, but to harness them as tools, incorporating them as components.
Since meta-data is key, it bears mentioning that there are many ways in which the system may be populated with data and meta-data starting with the automation of processing from within the system itself, and including technologies such as XML parsers of web-sites, email notification systems, FTP repositories, and so forth. Indeed, BigSur has been populated in precisely these ways in existing implementations. We can also load a very significant amount of meta-data from the existing GCMD repository. Some simple tools may be made available so researchers may describe their work themselves, eliminating a human bottle-neck and reducing error. As the number and size of installations grow, the distributed features may be used to divide workload and reduce resource loading such as network bandwidth. Over time, specific facilities or research groups may find the system interesting enough to have their own installation into which they put various resources such as conversion programs, leading to their use of The BigSur System for their own research. Or we may propose in the EMD RFP response to setup several such systems from the outset.
One of the greatest visions for science is a computational unification in which every researcher can interact with all other researchers through use of their own research system. The system we propose has all the right elements to do that: The capacity to describe the work of every researcher in their own terms, the ability to manage every type of data object and the functional processes that operate against them, the ability to automate this processing, and the ability to "publish" data to aid collaboration with others. With some type-conversions to perform transformations of data from one form to another - automated by the system - the system can join together the work of researchers from disparate disciplines. The system has full information about the associative relationships between scientific elements and so it "knows" the paradigms of each researcher. It never forgets, so as people change, knowledge of how things are done is not lost. These features not only enable research collaboration on a scale never previously envisaged, they also enable sharing and dissemination of scientific knowledge to the public at large with a sophistication unparalleled in history. It is easy to predict that eventually this new system will become the lingua Franca of science.
Contacts: Richard Troy: RTroy@ScienceTools.com, Olga Kingrey: Olga@ScienceTools.com
website contact: Webmistress