The NOAO Data Lab aims to provide infrastructure to maximize community use of the high-value survey datasets now being collected with NOAO telescopes and instruments. As a science exploration framework, the Data Lab allow users to access and search databases containing large (i.e. terabyte-scale) catalogs, visualize, analyze, and store the results of these searches, combine search results with data from other archives or facilities, and share these results with collaborators using a shared workspace and/or data publication service. In the process of implementing the needed tools and services, specific science cases are used to guide development of the system framework and tools. The result is a Year-1 capability demonstration that (fully or partially) implements each of the major architecture components in the context of a real-world science use-case. In this paper, we discuss how this model of science-driven development helped us to build a fully functional system capable of executing the chosen science case, and how we plan to scale this system to support general use in the next phase of the project.
KEYWORDS: Data storage, Clouds, Distributed computing, Astronomy, Databases, Data storage, Clouds, Data conversion, Observatories, Data archive systems, Data storage servers, Java
Collaborative research/computing environments are essential for working with the next generations of large astronomical data sets. A key component of them is a distributed storage system to enable data hosting, sharing, and publication. VOSpace1 is a lightweight interface providing network access to arbitrary backend storage solutions and endorsed by the International Virtual Observatory Alliance (IVOA). Although similar APIs exist, such as Amazon S3, WebDav, and Dropbox, VOSpace is designed to be protocol agnostic, focusing on data control operations, and supports asynchronous and third-party data transfers, thereby minimizing unnecessary data transfers. It also allows arbitrary computations to be triggered as a result of a transfer operation: for example, a file can be automatically ingested into a database when put into an active directory or a data reduction task, such as Sextractor, can be run on it. In this paper, we shall describe the VOSpace implementations that we have developed for the NOAO Data Lab. These offer both dedicated remote storage, accessible as a local file system via FUSE, and a local VOSpace service to easily enable data synchronization.
The time domain has been identied as one of the most important areas of astronomical research for the next
decade. The Virtual Observatory is in the vanguard with dedicated tools and services that enable and facilitate
the discovery, dissemination and analysis of time domain data. These range in scope from rapid notications of
time-critical astronomical transients to annotating long-term variables with the latest modelling results. In this
paper, we will review the prior art in these areas and focus on the capabilities that the VAO is bringing to bear
in support of time domain science. In particular, we will focus on the issues involved with the heterogeneous
collections of (ancilllary) data associated with astronomical transients, and the time series characterization and
classication tools required by the next generation of sky surveys, such as LSST and SKA.
We present the VOEventNet infrastructure for large-scale rapid follow-up of astronomical events, including selection,
annotation, machine intelligence, and coordination of observations. The VOEvent standard is central to this vision, with
distributed and replicated services rather than centralized facilities. We also describe some of the event brokers, services,
and software that are connected to the network. These technologies will become more important in the coming years,
with new event streams from Gaia, LOFAR, LIGO, LSST, and many others.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.