Virtuoso questions

On april 25 2014 we had a knowledge session on Virtuoso Details virtuoso webinar on 25 april. This page contains the questions and answers.

See also: LinkedIn item: Vragen over Virtuoso voor kennisoverdracht LOD2

Attendants

• Martin Kaltenböck (Semantic Web Company, LOD2 PUBLINK)
• Hugh Williams (OpenLink Software)
• Patrick van Kleef (OpenLink Software)
• Gerard Persoon (PiLOD Platform,GerardPersoonBV)
• Richard Nagelmaeker (PiLOD Platform, Ordina)
• Pieter van Everdingen (PiLOD Wiki, OpenInc)
• Rein van 't Veer (Digital Heritage)

Questions

How to show RDF resources when accessed by their own URI

• VIa the HTML proxy description service (/about/html) or the Virtuoso Faceted Browser service (/describe). http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtFacetBrowserInstallConfig
• OpenLink public Sponger service: http://uriburner.com
• OpenLink public Faceted Browser: http://uriburner.com/fct
• OpenLink LOD Cloud CacheServer (60+ billion triples): http://lod.openlinksw.com
• RDF HTML description of sponged PiLOD Web Page: http://bit.ly/1jIanPE
• Faceted Browser results page of sponged PiLOD Web Page: http://bit.ly/1tI28Lw
• Virtuoso Linked Data Deployment Guide for Generic URI naming scheme creation http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtDeployingLinkedDataGuide

What are the differences between the open source and the closed source version of Virtuoso?

• The main differences are the following which are closed source only:
  • Clustering & High Availability
  • Virtual Database
  • Replication
  • ACL control over large numbers of graphs
• http://virtuoso.openlinksw.com/features-comparison-matrix

Can you share your experiences with other projects that are working with the combination of Hadoop, Virtuoso and Elasticsearch, in environments with a lot of data?

• Most implementation we come across in the Enterprise are using Virtuoso as the RDF Quad Store for indexing of Meta Data in conjunction with a combination of Hadoop, Elasticsearch, Lucense/Solr , Key Value (No-SQL) database and traditional relational databases.
• Example Elsevier http://www.slideshare.net/ayagoda/elsevier-smart-content-ldr-sem-tech-2012-v4 Smart Content Linked Data Repository

Is the new LOD2 ETL tool Unified Views an open source tool that can be used within the PiLOD platform to get Hadoop data into an Virtuoso environment?

• Unified Views is open source and will be part for the final LOD2 Stack release https://grips.semantic-web.at/display/UDDOC/Introduction
• There was a LOD2 Webinar on Unified Views earlier this month
• Don’t see anything specifically about Hadoop support but I would imagine hadoop jobs can be defined in its processing pipeline for invocation in processing data (large), which once processed and transformed to RDF can be store in Virtuoso as a support triple store in the LOD2 Stack

What are the typical use cases for Virtuoso Sponger? And what are the differences, pros and cons compared to other RDF middleware?

• The Virtuoso Sponger Middleware is used for the transformation of structured and unstructured web accessible resources to RDF Linked Data, using a combination of extractor (transformers)and(enrichment) cartridges. http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtSponger
• Cartridges can be written with its programmable API http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtSpongerCartridgeProgrammersGuide
• Pros:
  • Automatically integrated with Virtuoso Quad Store
  • 200+ Cartridges already exist and are provided with Virtuoso
  • List of available Sponger Cartridges http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSources
  • Aperture cartridge for PDF and Powerpoint http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/RDFMapper
• Cons:
  • Cannot be used with other triple stores
  • Currently working in Fusepool P3 project on decoupling the Virtuoso sponger from the Quad Store and also to make it a more flexible RDF ETL processing pipeline

What are best practices to convert RDBMS data to RDF data?

• Virtuoso RDF Linked Data Views http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VOSSQL2RDF
• Virtuoso R2RML support http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtR2RML
• D2R Server which is part of the LOD2 Stack http://lod2.eu/Project/D2R-Server.html
• Hand crafted scripts/programs for mapping Relational Data to RDF Datasets
• DB2Triples already being used by one of the PiLOD platform users https://github.com/antidot/db2triples/

What are the possibilities with Geodata in Virtuoso 7.1?

• The following GeoSpatial data types are supported along with associated geometries
  • Point
  • MultiPoint
  • LineString
  • MultiLineString
  • Box
  • Polygon
  • MultiPolygon
  • GeometryCollection
• Update to Virtuoso RDF Geometry documentation being prepared http://docs.openlinksw.com/virtuoso/rdfsparqlgeospat.html
• Virtuoso SPARQL Extensions for Geo Spatials Geography Tutorial http://virtuoso.openlinksw.com/tutorials/sparql/SPARQL_Tutorials_Part_10/SPARQL_Tutorials_Part_10.html#(1)

• Virtuoso SPARQL Geo Spatials and Geography PivotViewer Examples http://virtuoso.openlinksw.com/tutorials/sparql/SPARQL_Tutorials_Part_11/SPARQL_Tutorials_Part_11.html#(1)

What are the known issues with Geo data support in Virtuoso 7.1 and what is the schedule and priority to resolve those issues?

• Working on full OGC compliance http://www.opengeospatial.org/
• Working on full GeoSPARQL compliance http://www.geosparql.org/
• Both as part of our involvement in the GeoKnow project, see GeoKnow D2.3.1 deliverable http://svn.aksw.org/projects/GeoKnow/Public/D2.3.1_Prototype_of_Built-in_Geospatial_Capabilities.pdf

• GeoKnow ends in 1.5 years time so would expect to have full compliance by then, but hope to be well before that http://geoknow.eu

Which parties are also interested in Geo Data?

• We are also working with other Geo Data specialist who are interested in Linked Geo Data support to ensure the implementation is right. http://www.w3.org/2014/03/lgd/
• See list of attendees in Linked Geo Data workshop 2014 in London for interested parties http://www.w3.org/2014/03/lgd/
  • Ordinance Survey http://data.ordnancesurvey.co.uk/UK
  • Digital Heritage Netherlands http://www.ldbc.eu:8090/download/attachments/5538064/Presentation+LDBC+Heritage+%26+Location.pptx?version=1modificationDate=1396604660000
  • Geodan NL http://www.ldbc.eu:8090/download/attachments/5538064/Presentatie+LDBC+CWI+2014-04-03.pptx?version=2modificationDate=1396604384000
  • many more … http://www.w3.org/2014/03/lgd/attendees
• Interesting paper on PiLOD Web site entitled, From Geo-Data to Linked Data: Automated Transformation from GML to RDF Boek/BrinkEtAl-GML2RDF
• GeoKnow Generator is a tool for transforming Geo Data to RDF Linked Data http://generator.geoknow.eu
  • Based on LOD2 Stack which GeoKnow project will take over the maintenance of when LOD2 project ends this year.

How to store time related data (temporal) ?

• No built in support for temporal data per say
• time series data can be encoded as triple in the graph
  • Using a semantic approach for monitoring applications semantic approach for monitoring applications http://www.ldbc.eu:8090/download/attachments/5538064/Virtuoso+in+PMS+Coentunnel+-+Bastiaan+Bijl.pdf?version=1modificationDate=1396597958000i
  • Need to dig out examples on performed comparison operations on queries with datetime elements ...

What would be your reading recommendation for the Virtuoso documentation?

• Presentations http://www.slideshare.net/rumito/virtuoso-universal-server-overview-presentation
• Tutorials http://demo.openlinksw.com/tutorial/
• Tips and Tricks http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtTipsAndTricksGuide
• Mailing list http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VOSMailingLists
• Support Forums http://boards.openlinksw.com/support

What are best practices to run the server?

• how to monitor availability
  • sys_stat function http://docs.openlinksw.com/virtuoso/fn_sys_stat.html
  • can be hooked into Network resource monitoring tools like Zabbix or protocols like SNMP
• how to make backups
  • Full support for online and incremental backup and restore
  • http://docs.openlinksw.com/virtuoso/backup.html
• how to optimize the server / queries
  • Database Performance Tuning Guide http://docs.openlinksw.com/virtuoso/ptune.html
  • RDF Performance Tuning Guide http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFPerformanceTuning
  • Key requirement is to have database workset in memory, as accessing disk will always reduce performance
• how to secure a graph for a specific user or usergroup
  • RDF Graph Groups http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtRDFGraphsSecurity
  • Access Control Lists (ACLs) http://docs.openlinksw.com/virtuoso/wsacl.html
• how to upgrade to newer versions without losing data
  • ensure Full backup in place
  • generally any v7.x DB will work with 7.x binary
  • ensure database checkpointed before shutting down
  • v6 database can be automatically upgrade to v7
    • Would be in row store mode though
    • Probably should dump data and reload to get column store RDF tables which consumes a third of the space/memory which is key benefit ...

Virtuoso (Universal server)

Semantic Web Company (SWC) is a technology provider headquartered in Vienna (Austria). SWC supports organizations from all industrial sectors worldwide to improve their information management. Our core product PoolParty has outstanding capabilities to extract meaning from big data by making use of linked data technologies.

Freelance Enterprise & Informatie Architectuur Services
- Semantiek, Metadata, Linked Data & Knowledge Graphs -

HyperText Markup Language (HTML).

Weergave van een feit, begrip of aanwijzing, geschikt voor overdracht, interpretatie of verwerking door een persoon of apparaat

Structured Query Language (SQL)

Extraction, Transformation and Load (ETL)

The LOD2 Stack has been renamed to the Linked Data Stack in 2014. The LOD2 Stack comprises a number of tools for managing the life-cycle of Linked Data. The life-cycle comprises in particular the stages: 1) Extraction of RDF from text, XML and SQL,2) Querying and Exploration using SPARQL, 3) Authoring of Linked Data using a Semantic Wiki, 4) Semi-automatic link discovery between Linked Data sources, 5) Knowledge-base Enrichment and Repair

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.The project includes these modules: 1) Hadoop Common: The common utilities that support the other Hadoop modules. 2) Hadoop Distributed File System (HDFS™): A distributed file system that provides high-throughput access to application data. 3) Hadoop YARN: A framework for job scheduling and cluster resource management. 4) Hadoop MapReduce: A YARN-based system for parallel processing of large data sets.

Een usecase in software engineering en systems engineering is een beschrijving van een gedrag van een systeem, dat reageert op een verzoek dat stamt van buiten het systeem. Met andere woorden, de use case beschrijft "wie" met het betreffende systeem "wat" kan doen (bron: Wikipedia).

Linked Data is een internettechnologie waarmee je gestructureerde gegevens op een uniforme manier kunt beschrijven, modelleren, opslaan, publiceren en onderling kunt koppelen via het internet met behulp van RDF-gebaseerde open standaarden en URI's (bijv. W3C-aanbevelingen en vocabulaires).

R2RML is a language for expressing customized mappings from relational databases to RDF datasets. Such mappings provide the ability to view existing relational data in the RDF data model, expressed in a structure and target vocabulary of the mapping author's choice. R2RML mappings are themselves RDF graphs and written down in Turtle syntax. R2RML enables different types of mapping implementations. Processors could, for example, offer a virtual SPARQL endpoint over the mapped relational data, or generate RDF dumps, or offer a Linked Data interface.

GeoSPARQL is zowel een beknopte ontologie voor geo-informatie als een uitbreiding op SPARQL met functionaliteit voor ruimtelijke vragen (in de buurt van, overlapt met, etc). GeoSPARQL is gestandaardiseerd bij het Open Geospatial Consortium.

Weergave van een feit, begrip of aanwijzing, geschikt voor overdracht, interpretatie of verwerking door een persoon of apparaat