Samuel's Project Blog

I was a bit worried over the performance of the RDF facilities in Bioclipse, as a SPARQL query for doing NMR Spectrum similarity search, including a numerical comparison (against datasets which are attached) were quite unsatisfactory, being some 2 orders of magnitude worse than some Prolog code I wrote for doing the same task (But of course, this might not be the way Pellet is optimized to work).

Anyway, when testing the same SPARQL query with Jena, I got some quite different results, as seen in the graph to the right; Jena in fact outperformed the prolog code! Interesting..

Test results

Pellet/SPARQL vs SWI-Prolog

So, while Prolog far outperformed Pellet for the task (at least when Pellet was to do it using SPARQL) ...

Jena/SPARQL vs SWI-Prolog

... at the same time Jena outperforms Prolog, running the same SPARQL query above. Interesting!

In Tabular form

Code

SPARQL Query

PREFIX owl: <http://www.w3.org/2002/07/owl#> 
PREFIX afn: <http://jena.hpl.hp.com/ARQ/function#> 
PREFIX fn: <http://www.w3.org/2005/xpath-functions#> 
PREFIX nmr: <http://www.nmrshiftdb.org/onto#>  
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 
SELECT ?s ?s1 ?s2 ?s3
WHERE {
  ?s nmr:hasPeak [ nmr:hasShift ?s1 ] , 
                 [ nmr:hasShift ?s2 ] , 
                 [ nmr:hasShift ?s3 ] . 
FILTER ( fn:abs(?s1 - 17.6) < 0.3 && 
         fn:abs(?s2 - 18.3) < 0.3 && 
         fn:abs(?s3 - 22.6) < 0.3 ) 
} LIMIT 1"

Prolog code

findMolWithPeakValsNear( SearchShiftVals, Mols ) :-    
  % Pick the Mols in 'Mol', that match the pattern:    
  %   listPeakShiftsOfMol( Mol, MolShiftVals ),
  %   containsListElemsNear( SearchShiftVals, MolShiftVals )    
  % and collect them in 'Mols'.
  setof( Mol,     
         ( listPeakShiftsOfMol( Mol, MolShiftVals ),         % A Mol's shift values are collected  
   containsListElemsNear( SearchShiftVals, MolShiftVals ) ), % ...and compared against the given SearchShiftVals
   [Mols|MolTail] ).                                         % In 'Mols', all 'Mol's, for which their shift     
                                                             % values match the SearchShiftVals, are collected. 
% Given a 'Mol', give it's shiftvalues in list form, in 'ListOfPeaks'
listPeakShiftsOfMol( Mol, ListOfPeaks ) :-
  hasSpectrum( Mol, Spectrum ),
  findall( ShiftVal,     
           ( hasPeak( Spectrum, Peak ),
             hasShiftVal( Peak, ShiftVal ) ),     
             ListOfPeaks ).    
 
% Compare two lists to see if list2 has near-matches for each of the values in list1
containsListElemsNear( [ElemHead|ElemTail], List ) :-    
  memberCloseTo( ElemHead, List ),
  ( containsListElemsNear( ElemTail, List );    
    ElemTail == [] ).  
 
%%%%%%%%%%%%%%%%%%%%%%%%
% Recursive construct: %
%%%%%%%%%%%%%%%%%%%%%%%%
% Test first the end criterion:
memberCloseTo( X, [ Y | Tail ] ) :-    
  closeTo( X, Y ).
% but if the above doesn't validate, then recursively continue with the tail of List2:
memberCloseTo( X, [ Y | Tail ] ) :-    
  memberCloseTo( X, Tail ).    
 
% Numerical near-match    
closeTo( Val1, Val2 ) :-
  abs(Val1 - Val2) =< 0.3.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Convenience accessory methods %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
hasShiftVal( Peak, ShiftVal ) :-  
rdf_db:rdf( Peak, 'http://www.nmrshiftdb.org/onto#hasShift',
literal(type('http://www.w3.org/2001/XMLSchema#decimal',
ShiftValLiteral))),
  atom_number_create( ShiftValLiteral, ShiftVal ).        
hasSpectrum( Subject, Predicate ) :-    
  rdf_db:rdf( Subject, 'http://www.nmrshiftdb.org/onto#hasSpectrum', Predicate). 
hasPeak( Subject, Predicate ) :-
  rdf_db:rdf( Subject, 'http://www.nmrshiftdb.org/onto#hasPeak', Predicate).

% Wrapper method for the atom_number/2 method which converts atoms (string constants) to number.    
% The wrapper methods avoids exceptions on empty atoms, instead converting into a zero.    
atom_number_create( Atom, Number ) :-     
  atom_length( Atom, AtomLength ), AtomLength > 0 -> % IF atom is not empty 
  atom_number( Atom, Number );                       % THEN Convert the atom to a numerical value     
  atom_number( '0', Number ).                        % ELSE Convert to a zero

Attached files

Attached are the datasets (nmrshiftdata10...100.rdf.xml) for the different number of spectra, and the Bioclipse JavaScript files for the tests (NMR.Pellet.js, NMR.Jena.js and NMR.Swipl.js, as well as NMR.Pellet.Init, which I used for preparing the RDF stores for both Jena and Pellet).

What do you think of that title? :) To me it sounds like one of the (many) natural next steps forward for Bioclipse sometime in future¹.

Explicit knowledge is too expensive

There are lots of things that can't be answered by a computer from data alone. Maybe the majority of what we humans perceive as knowledge is inferred from a combination of data (simple fact statements about reality) and rules that tell how facts can be combined together to allow making implicit knowledge (knowledge that is not persisted as facts anywhere, but has to be inferred from other facts and rules) become explicit.

One can easily imagine though, that storing every single piece of knowledge that could be stated, as an explicit fact, would require more storage than can probably ever be made available in this universe.

Simulations can make knowledge explicit, from first princples

It is not too hard to come up with some processes which are just too complex and involves too much variability² that it is unrealistic to try to capture every imaginable state of of that system or process in explicit facts. Instead we must seek the "first principles" that defines the process, and through simulations make explicit any knowledge we are looking for, at the time we need it (one can of course cache often accessed knowledge).

On something I realized a minute ago ...

Though being really different types of technologies, it might at first be tempting to compare a SPARQL query with a Prolog rule returning a list of results (Or at least it was to me until just a minute ago). In fact, SPARQL queries and prolog rules that return their results as lists, DO share some similarities. For example, in both you provide patterns of RDF statements with variables that are to be bound to each other or to RDF entities, in order to find all queried entities that match the pattern.

But then comes some important differences, in how SPARQL handles cases where one wants to evaluate looked-up entities with a function, like an arithmetic one. In SPARQL this has to be done (I think) with the FILTER construct, but that also means that backtracking is not done if nothing passes the filter (and that is the true meaning of a filter anyway, isn't it).

I needed a Bioclipse manager method that could take an arbitrary number of arguments, (for a general purpose prolog method mapper). Through a useful discussion with jonalv, we figured out that there exists at least one working way of doing this, while there are a number of ways that do not work across both the Rhino/JavaScript though they work in Java alone.

A strategy for how to work with the Bioclipse/JPL/Prolog/Blipkit combination I'm setting up, is becoming clear.

The main idea with Bioclipse, as well as with having a prolog engine available in it, is for flexible and "interactive" knitting together of knowledge. One of the main questions regarding how to use a Bioclipse/JPL/Prolog/Blipkit combination, has been where to put the bulk of knowledge integration/reasoning code? There would in principle be three options for that:

1. Bioclipse (Javascript environment)
2. The Blipkit-Prolog/Bioclipse integration plugin (Java code, a.k.a. "Manager methods")
3. The prolog engine (As a prolog file)