UCD resolver library
====================

The UCD resolver library offers functions to resolve descriptions to
Unified Content Descriptors (UCDs) based on (mainly) Vizier_ assigned
UCDs.  This is done using various quantitative models -- don't expect
miracles.

There is a web interface exposing the libraries functionality at
http://vo.uni-hd.de/ucds/ui/ui/form.  A SOAP-based web service can be
provided on request.

High Level Interface
--------------------

To resolve a description to a ucd, say

::

  import ucdresolve
  ucdresolve.resolve(description)

This takes a while on the first call, further calls should not take
significant time.

The function returns a list consisting of tuples of a score and a
ucd string for the ten "best" ucds.  The scores from repeated calls are
not comparable.

The function may raise a ucdresolve.NoEvidence exception if none of
the tokens in description has been seen before (or if the description
only contains stop words -- common words like if, when or maybe).


To explain a ucd, say

::

  ucdresolve.explain(ucd)

You'll get back a string synthesized from the `official UCD list`_
issued by IVOA using a couple of rules.

The function may raise a ucdresolve.InvalidUCD exception if the UCD
passed in contains UCD words not in the official list or if it is
otherwise malformed, e.g., by having secondary words in a primary
position.

The high level functions of the library are thread-safe (or
rather, it's a bug if they aren't).

Low Level Interface
-------------------

Most of the functionality of the library is based on corpora.  These are
collections of UCD objects.  Not all UCD objects can be used with all
types of corpora.

Some of the more useful corpus classes include:

* corpus.FeedableCorpus -- a basic corpus that you can iterate over and
  that can be constructed from a source
* corpus.CollocWeightingCorpus -- has a resolve(ucd) method that tries
  to assign UCDs by some sort of generalized Dunning's test
* corpus.BayesianCorpus -- has a resolve(ucd) method that classifies
  ucd using naive Bayesian classification
* corpus.VectorModelCorpus and corpus.CumulativeVectorModelCorpus --
  these only work with VectorModelUCDs and try to resolve UCDs with
  a fairly standard vector model.
* ivoawords.WordDirectory -- is a special corpus for ivoaucd.IvoaUCDs,
  i.e. they are a collection of atomic UCD words.  WordDirectories have
  a validate(splitUCD) method that does a syntactic check of a ucd.


To build a corpus, you will usually go through FeedableCorpus'
fromSource class method or some variant of it.  fromSource takes an
input as its first argument; all other arguments are passed to the
constructor of the corpus class itself.  Input has to iterate over
whatever data the UCD constructor takes (i.e., usually lines).

For example, when you have a dump of Vizier's ucd-description corpus in
vlines.txt, you could construct a plain corpus containing this data
using

::

  from ucdresolve import corpus, ucdmodel

  corpus = corpus.FeedableCorpus.fromSource(open("vlines.txt"),
    ucdFactory=ucdmodel.UCD.fromVizierLine)

To find all compound ucds mentioned there, you could say:

::

  compounds = set([u.ucd for u in corpus if ";" in u.ucd])

To get explanations for these, you can use the high level interface:

::

  import ucdresolve, pprint
  pprint.pprint([(u, ucdresolve.explain(u)) for u in compounds])

In reality, you might encounter bad UCDs, so you'll probably want to
go for the more verbose version

::

  for u in compounds:
    try:
      print u, ucdresolve.explain(u)
    except ucdresolve.InvalidUCD:
      print "Bad UCD: %s"%repr(u)

  


.. _official UCD list: http://www.ivoa.net/Documents/latest/UCD.html
.. _Vizier: http://webviz.u-strasbg.fr/viz-bin/VizieR