Difference between revisions of "CyberInfrastructure Proposal For EarthCube Community"

Revision as of 22:22, 31 July 2012

This proposal presents a cyberInfrastructure for sharing and discovery of long tail of science data.

Objective

The proposed system will facilitate scientists to upload their data and annotate them using community developed vocabularies such as GCMD and AGI. It will also enable refining or adapting automatically suggested annotations. Once published, the data can be discovered using keywords, topic terms and attribute value pairs in a faceted search. The system will provide tools for flexible search and for harmonizing structural, content and semantic heterogeneity.

We have identified three different workflows to cater to the different requirements for publishing long tail of science data. To accommodate variations in the nature of the data, and the expertise of a user, the three workflows present different trade-offs between convenience of data sharing and breadth of data analysis to be performed.

Publishing Data in Original Form/Legacy Data

This workflow will allow long tail of science data to be published in its original form which can be legacy data, or unstructured text, images and tables as found in technical papers, or available separately. We will provide tools to read the data in its original form, annotate the data and index it to make it searchable. Initially, the focus will be on processing and indexing only captions for images and tables.

Publishing Data in Digital Format

This workflow will serve the long tail of science data that is in a digital format such as EXCEL, CSV and relational data files. We will also provide additional capabilities to the user by suggesting additional annotations for the structured data and encouraging data publishers annotating columns and cell values in their data to provide more insights on their data.

Publishing Data in Linked Data

Linked Data initiative http://linkeddata.org/ emerged from Semantic Web technologies in the recent past making its own way in the web by providing a publishing and querying paradigm for structured raw data. Arrival of LOD to semantic web changed the way we share data in the web; primarily on how to interconnect data sets together. Currently it consists of 295 data sets with 31 billion RDF triples and it covers a broad range of domains such as Life Sciences, Geography, Government, Media, Education, Publication and so on.

At this stage long tail scientists have the capability to convert their data in to the RDF format and publish their data in Linked Open Data. This will be able to standardize the data itself and furthermore this will allow scientists to interlink their data with other data sets exist in linked open data. This makes their data available for more advanced intelligent applications such as federated querying. Even though there are existing tools to convert data in to RDF and publish data, still this requires scientists to get some help from computer scientists. Our system will provide the relevant sophisticated tools for them to use.

Architecture

The following image illustrates the architecture of the proposed system.

• Data Registry

Data publishers will register their data through the data registry and provenance information such as author, location and etc will also be collected. Sample of the form of data can be registered is given at a later section.

• Annotator

Registered data will be annotated using standard vocabularies such as (GCMD and AGI index) which is stored in a vocabulary registry. Annotation tools will suggest the possible matches for the user and user will have the ability to further refine the suggestions given by the system. Annotations will be stored in the Meta Data Store.

Kino http://wiki.knoesis.org/index.php/Kino is an integrated suite of tools that enables scientists to annotate Web documents and we plan extend this to facilitate annotation for this proposal.

• Indexer

Collected data and its associated meta data will be indexed to facilitate Searching.

• Simple Search

Simple Search facilitates key word based queries where user can specify some key words and system will provide a ranked list of results.

• Faceted Search

In addition to the Simple Search functionality system will provide the Faceted Search where users can provide the attribute value pairs to search/discover data. Users have the ability to incoperate provenance information for search as well.

• Mapping to RDF

As defined in the third work flow given data can be transformed to RDF using existing tools and this allows data publishers to convert the data in to a standard form.

• Data Publisher

This component will upload the RDF converted data into Linked Open Data and it will be accessed and queried from any where in the world.

• Semantic Browsing

Semantic Browsing will allow us to navigate through the RDF data sets which is based on the triples. iExplore http://knoesis.wright.edu/iExplore/ is a tool we developed for Semantic Browsing.

Form of Data

Please click on the images to enlarge.

Table

Image

Unstructured Data