Experiments with Uniprot RDF and RDF Gateway

[Author: Geoff Chappell Last Update: 2005-06-16]

The goal of this effort is to demonstrate the ability of RDF Gateway to efficiently store and query massive amounts of RDF data in its native RDF repository. The Uniprot RDF project provides all UniProt protein sequence and annotation data in RDF format and is an excellent large source of data. Thanks to Eric Jain for making it available to all (and to Eric Miller for pointing me at it.)

New - Check out the experimental SPARQL query page

Loading the data

The uniprot data is delivered in a number of rdf/xml files ranging in size from 4kB to 11.4GB. In total, the files contain nearly 262 million triples - sufficent to test the new bulk loading capabilities of RDF Gateway (to be released in upcoming version). The script below was used to load the data.

The load took several days to complete, so there is admittedly room to improve the loading performance. The good news is that the process was never I/O bound (processor was at 100% the whole time) so there is good reason to believe that optimization should be possible (especially since the actual insertion time into all indices was about 24 hours, the rest of the time was preprocessing to pre-sort the data). The files were loaded into a triple-only table and were fully indexed (three indices with complete index statistics plus full-text index). The file ended up being 38.2 GB on disk, of which 27.2 GB were in use (excess was because of 50% table growth size used when loading).

Queries

Below are some sample queries I developed against the Uniprot data. All of the queries assume the data has been imported into a table named uniprot. Anyone with more knowledge of the data, please feel free to suggest more interesting examples (I'll also be adding a SPARQL interface soon so you can write your own.)

Example 1 - Find all predicates used in the data [Try It ]

select ?p using uniprot where {?p ?s ?o} order by ?p;

Example 2 - Find up to 100 things that have a property that contains specific search terms [Try It ]

select top 100 ?p ?s ?o using uniprot where {?p ?s ?o} and ?o like 'rat' and ?o like 'virus';

Example 3 - Find protein sequences within a range of masses [Try It ]

select top 100 ?s ?prot ?n ?o using uniprot where {[urn:lsid:uniprot.org:ontology:mass] ?s ?o} and between(?o, 19400, 19410) and {[urn:lsid:uniprot.org:ontology:sequence] ?prot ?s} and {[urn:lsid:uniprot.org:ontology:name] ?prot ?n} order by ?o, ?n

Example 4 - Find out who is citing publications about dogs [Try It ]

select top 10 ?n ?o using uniprot where {?p ?s ?o} and ?o like 'dog' and {[rdf:type] ?s [urn:lsid:uniprot.org:ontology:Journal_Citation]} and {[urn:lsid:uniprot.org:ontology:author] ?s ?n}

Example 5 - Find all proteins associated with a specific organism and the species of the organism [Try It ]

rulebase x{ //create a rule expressing the transitivity of subclass infer {[rdfs:subClassOf] ?a ?c} from {[rdfs:subClassOf] ?a ?b} and {[rdfs:subClassOf] ?b ?c}; } select ?s ?cn ?n ?x using uniprot rulebase x where {[uni:commonName] ?s ?cn} and ?cn like 'goose' and {[rdfs:subClassOf] ?s ?sup} and {[uni:scientificName] ?sup ?n} and {[uni:rank] ?sup [uni:Species]} and {[uni:organism] ?x ?s};

Example 6 - Names of proteins with a gene with a specific name or synonym [Try It ]

select ?protein, ?name using uniprot where {[uni:name] ?gene "CRB"^^xsd:string } and {[uni:encodedBy] ?protein ?gene} and {[rdf:type] ?protein [uni:Protein]} and {[uni:name] ?protein ?name};