Note: Of course, in each version, many minor improvements and bug fixes
go in. To see the full history, use
svn log http://svn.ari.uni-heidelberg.de/svn/gavo/python/trunk/ | less
to see the gory details. If anything not mentioned here breaks something
for you, we'd consider it a severe bug, though.
Version 2.7.2 (2023-01-15)
* Support for content_qualifier in Datalink.
* The license-X procs now use SPDX URIs, and they now point to versions
4 of CC-BY and CC-BY-SA.
* Expunging the use of the (deprecated) cgi module. In consequence,
request.fields (used with uploads) contains somewhat different things.
If you've been using DaCHS' APIs, you shouldn't notice.
* Minor changes: compatility with numpy >1.24
Version 2.7 (2022-11-25)
* Adding simple, numpy-like vector operators to ADQL. Also, the FITS
binary table grammar now copes with vectors there.
* Adding a prototype for a mixin for the obscore visibility extension.
* ivo_epoch_prop and ivo_epoch_prop_pos UDFs in ADQL.
* Adding support of CASE in ADQL.
* epntap2 shape parameter is now called coverage, following a spec change.
* The BAND parameter for FITS/SODA cutouts now can deal with non-wavelength
spectral coordinates.
* Prototype support for local_semantics in Datalink.
* TAP columns now have more meaningful table headings, which prettifies
the output when using the ADQL browser form.
* Processor support for astrometry.net changed in several, incompatible
ways. If you have a calibration script you may need to re-run, you will
have to update it.
* Many minor fixes (e.g., protocol adaptation for product links so
browsers download images on dual-protocol services; DaCHS now copes
with Postgres 15; less misguided planning in the presence of CTEs;
VOTable serialisation of n-D arrays now actually works; ADQL UDFs
without support in postgres are no longer advertised; we use astropy
for sexagesimal formatting, which might change outputs in a few corner
cases; UCD from the 1.5 list are now ok by DaCHS; math in
ReStructuredText documents is rendered better).
Version 2.6 (2022-05-31)
* Adding a prototype for a LineTAP mixin and some support code (such
as a libinchi binding).
* You can now write vizierexprs over pairs of min/max columns (with
a bit of custom code, so far).
* There's now a spectralUnit displayHint quite like displayUnit,
except it can convert between wavelength, frequency, and energy, too.
* resdir="." now means "the resource directory is the directory the RD
sits in". This is the new recommended default, intended to help, e.g.,
when mirroring. This is complemented by a new RD macro \resdir
evaluating to the actual resdir.
* index elements can now have option children. This can be used
to create partial indexes, adapt the fill factor, or move indexes
to separate tablespaces.
* You can now hash your [web]adminPasswd (use dachs adm hashPassword); note
that DaCHS can longer control servers with hashed passwords.
* odbcGrammar can now have a makeQuery proc; this should make incremental
harvesting from remote databases a lot simpler.
* Now obtaining string frequencies on TEXT columns with a statistics
property set to enumerate in dachs limits. Also new in dachs limits: a -d
flag to dump the stats obtained.
* There is now //procs#negatableBoolean to support form queries against
booleans that actually constrain to False.
* Experimental support for SSO 2.1 basic authentication.
* We are now casting em_min/_max and a few more to reals (rather than
doubles) in obscore. This will make postgres use indexes on large
SSAP tables.
* Obscore registry records can now contain STC coverage. It now
pays to occasionally run dachs limits //obscore.
* DaCHS now includes referenceURLs to the originating services in
the VOTables it delivers.
* If you are using the "tree" root page, change $description to
$!description in the tmpl_resDetails script (description is now
rendered as HTML).
* As per SCS 1.03 Erratum 2, we now return modern VOTables even from
the SCS renderer.
* Many minor changes (e.g., no [web]adsMirror setting, makeIAUId
rowmaker function, MOCs now have a notion of their maximum order
and serialise that, RDs are pulled from obscore during re-indexing,
compatibility with twisted 23, utils.formatFloat to produce readable
floats, dachs limits -d dumps current stats; boosters can write
more sorts of arrays; DaCHS will no longer deluge you with mails
when things go persistently wrong but limit itself to 1 mail/hour/type;
we are materialising CTEs on postgres>=11 to maintain planning barriers
with WITH; less obscore locking during imports).
Version 2.5 (2021-11-17)
* Async overhaul: TAP is now based on core and renderer as usual, and
async operation in general is less special. This in particular means
some changes to useruws. If you're using it talk to Markus before
upgrading.
* UCD overhaul: DaCHS now validates UCDs (and warns against bad ones),
and it hides ugly SIAv1 ones from other protocols.
* DaCHS no longer accepts whitespace after a scale factor in unit strings.
It should never have done this, as VOUnit forbids it.
* Adding partial support for HDF5: At least astropy- and VAEX-generated
HDF5 should now be importable, with booster support for VAEX.
* Updating EPN-TAP support following Spec changes. Make sure your
EPN-TAP RDs are in good shape before upgrading, as DaCHS will need
them to update the in-DB metadata.
* DaCHS can now deal with materialized views.
* Many small additions (experimental DALIInterface in TAP; preparse
functions on inputKeys; better systemd integration; built-in schema
names are now unversioned; SIL fallbacks; COALESCE in ADQL; support
for writing float arrays in boosters; adopting MOC 1.1; products of
units can now be collapsed to powers).
* Oh: you may want to have a look at
http://docs.g-vo.org/DaCHS/tutorial.html#access-logs if you have to keep
access statistics.
Version 2.4 (2021-06-09)
* Massive blind discovery overhaul. dachs limits no longer touches
RDs but instead stores its results in the database. Advanced column
metadata (median, percentiles, fill factor) is now processed and
published to the registry using a prototype schema g-colstat.
Run dachs limits on your interesting RDs.
* New dachs gencol subcommand to quickly generate column elements in
RDs from VOTables, FITS bintables, or VizieR byte-by-byte files.
* New datalink descriptor generator //datalink#fromtable for making
datalinks for tables not containing products.
* New gavo_specconv ADQL user defined function to convert between
various spectral units.
* new //procs#declare-indexes-from STREAM for use in views; //ssap#views
uses it automatically.
* The queries that determine spatial coverage of SIAP and SSAP services
now require full MOC support in pgsphere.
* New meta superseded to point users to newer versions of dropped resources
(and removed previous superseded-url property).
* New footprint editor in the datalink/SODA XSLT with modeless operation;
the datalink HTML formatter now also presents links hierarchically
as per the datalink/core vocabulary.
* Slight overhaul of the default root page templates to no longer use
the dumb I and Q icons. The icons are still there, but I want to remove
them eventually. If you have a custom root-template, consider merging
my changes.
* A missing upload in form evaluation would so far have been (None, None),
and it's just None now. If you evaluate uploads in custom code, you
may have to adapt the code.
* As usual, many minor fixes and additions; in particular, DaCHS
now works with python 3.8.
Version 2.3 (2021-01-11)
* Passwords in the user table are now hashed. The upgrade will hash
them. You will no longer be able to recover them after the upgrade.
* DaCHS now instructs the database to return all bits of floating point
numbers; this may make a few of your displays quite a bit uglier
(complain and we'll consider a fix), but it's the right thing to
do in scientific software.
* The wildcards in the directory parts of sources patterns are now
expanded; this may in rare cases match additional sources when
you re-import data.
* TAP results now have column names in the case given in the RD; this may
change TAP results if you have mixed-case column names in your RD (which
is nothing but trouble, so don't do it).
* You can now list modules you want in a procedure application in its
setup/imports attribute.
* The examples endpoint now returns a 404 if no examples are defined
on a service; this fixes the stilts taplint E-EXA-EXDH-1 error you get
when you don't have TAP examples.
* simbadinterface's Sesame now uses dc.metastore to cache requests.
Previous saveNew, id, and debug parameters are no longer supported
(the base.caches.getSesame interface is unchanged).
* Updated javascript delivered with the tar to what comes with Debian
buster. Caution: This might break user javascript (see howDoI.rstx).
Also restored the WebSAMP button in the default response.
* table.query() or querier.query() is now seriously deprecated. See
howDoI for how to migrate.
* DaCHS now honours upgrade-insecure-requests that common web browsers
issue. So: Finally say no to forcibly "upgrade" plain http requests.
* DaCHS will now refuse to use x-unregistered as an authority when
publishing resources or creating publisher DIDs.
* For licensing issues, the Debian package no longer contains the CC logos.
If you want them back, drop cc0.png, ccby.png, and ccybysa.png into
/var/gavo/web/nv_static/img.
* As usual, many minor fixes and additions.
Version 2.2 (2020-10-13)
* Support for writing time series according to a Nebot et al
note. See:
http://docs.g-vo.org/DaCHS/tutorial.html#building-time-series-votables
* Support for the MOC constructor function in ADQL, plus new UDFs
gavo_mocunion, and gavo_mocintersect (need bleeding-edge pgsphere
to work).
* DaCHS now has basic support for dealing with IVOA vocabularies. One
direct effect is that dachs val will now warn you if your subject
keywords are not taken from the UAT. New ADQL UDF gavo_vocmatch
using this. The "plain" root page template now shows UDF term ids as
their human-readable labels.
* Support for frictionless data packages
(https://specs.frictionlessdata.io/data-package/; well, of course
you don't get to import these into full DaCHS resources.)
* dachs val ALL only validates published RDs now. You get the old
behaviour of traversing the file system when passing ALL_RECURSE.
* New [web]root config item if you want to have a different service
behind your root page.
* Attributes written as elements (foo) erroneously
accepted (but ignored) sub-attributes so far. DaCHS now rejects them.
This *might* lead to RDs rejected on upgrades; just delete the attributes
-- they didn't do anything before and were just typos.
* If you have custom flot plots in a template, you now need to use
style="Points" | "Lines" rather than usingIndex = 0 | 1.
* The sidebar no longer has links to a privacy policy and a disclaimer
(that few filled in). Look for _sidebarlocal in the tutorial to see
how to get it back (or put something else into the sidebar).
* The default logo now is a DaCHS one rather than GAVO's. I don't think
anyone will mind.
* You can now have scripts on tables; they are no longer allowed on RDs
(where they would only serve as sources for copying -- if you had
that, put them into an otherwise empty data/make element). There are
now also afterMeta scripts executed after table metadata is written.
Version 2.1 (2020-05-29)
* DaCHS now runs on python3 (and no longer on python2). Any code that
is within your RDs will have to be ported; look out for division (integer
division is now //), urllib (the quote function is now in urllib.parse),
bytes/str in comparisons and the like (in tests, you'll have to write
b"xy" in self.data or re.search(r".*", self.data); assertHasStrings will
work without modification as long as self.data is utf-8 encoded). See
also http://python3porting.com/problems.html
* Spatial indexing should now preferably go through //scs#pgs-pos-index
rather than //scs#q3cindex. The latter still is a good deal more
efficient, but on the long run we'd like to drop q3c as a dependency.
ADQL will now generate q3c conditions where it sees such an index,
pgsphere ones otherwise.
* In consequence, SCS now supports pgsphere-indexed tables, too
(though there's a maximum cone size of 90 degrees then). If you've
made a custom cone search as per howDoI (see there for details), you
will have to fix your phraseMaker's genQuery function and have it
accept an extra first argument.
* Now returning VOTable version 1.4 documents unless instructed otherwise.
* You can now set a semantics attribute on datalink meta makers, and
that is now the strongly recommended way to set semantics on datalinks.
* If you have SSA services that don't serve spectra, you can now set
a productType meta to generate a SimpleDALRegExt draft productType
declaration.
* dachs info now accepts a --sample-percent option to restrict
statistics gathering to a subset of a table (won't work for
views though).
* The SCS renderer now use DALI-style parameter syntax rather than PQL
before. Basically: intervals are now "min max" rather than "min/max"
(the former being what the VO converges to).
* The ADQL translator now uses nrows information on tables. It's
worth running dachs limits on RDs with larger tables. Also, nrows
information is now kept in the database, the RD is no longer
overwritten (you can still specify nrows manually).
* The ADQL querier now tries to enable parallel queries where they
look promising (cf. https://blog.g-vo.org/parallel-queries/).
* New gavo_normal_random ADQL user defined function for a (rather
naive) way to have a normal-distributed random number.
* Removed the docform renderer; there was nothing it did that the
form renderer couldn't. In case you're using it somewhere: just
write "form" instead of "docform" everywhere, republish, done.
* The forceUnique attribute on tableDef doesn't do anything any more. Also,
the dupePolicies are now implemented as triggers (rather than rules).
Function changes: with a check dupePolicy, NULLs are no longer considered
equal to anything but only equal to NULL. The exception raised by the
check policies are now InternalErrors.
* Services no longer recognise the sortKey property. Use the core's
defaultSortKey property instead.
* Built-in validation links on info pages (log in as admin to see them)
now point to VO Paris validation services.
* On sufficiently recent user agents, javascript-based preview loading
will now happen when the table row is shown.
* The SIAP mixin now also has an index on centerRA/centerDec, which
is handy whan obscore-publishing large image collections. If you
have one, run ``dachs imp -I my_large_siap/q``.
* Both copyright and _copyright meta are now mapped to "rights" (because
VOResource 1.1 lets you have arbitrary text there). If you have more
than one of these in an RD, this will change behaviour (they will appear
together).
* There's now index/@metaOnly just like there was
foreignKey/@metaOnly; use that to declare indices on views where
necessary, probably via ``.
* RD metadata is no longer managed in stateDir but in a database
table. The update should be automatic with dachs upgrade. After
the update, you can remove the updated* files in gavo/state.
* Date-like metadata is now managed differently and partly has different
names; there's essentially dataUpdated (date of last dachs imp) and
recTimestamp (date of last dachs pub).
* You can now ``dachs imp //obscore recover`` if automatically
repair things of your obscore declaration seriously broke.
* If you have custom OAI set name (the sets attribute in publish elements),
you now have to declare them in [ivoa]validOAISets.
* DaCHS now runs directory on twisted.web (rather than nevow as before).
This required fairly massive internal changes. If you have custom
renderers, these will require massive changes (see README in
gavo.formal). However, except for n:render="" with mapping data,
nevow templates should just continue to work.
* Renderers no longer receive SvcResults but plain data objects from
services.
* Api Docs are now built with sphinx-apidoc rather than epydoc.
* Now properly supporting external https termination. Set
[web]adaptProtocol to False to make that work.
* The soap renderer currently won't work on Debian systems because
Debian's python3 twisted doesn't support soap. If this does not
hurt anybody, I'll probably pull the plug on soap altogether.
* Dropped the ancient //scs#q3positions mixin. If there's anyone
who still used it: Migrate. It's been a mistake, as were the
[m]img.jpeg renders. I'm sure you don't use these.
* Plenty of internal reorganisation, dropping of cruft, etc. In all,
since 1.4, that's 262 commits with 17714 lines added and 15472 lines
removed (of ~95 klines of python and ~10 klines of RD in DaCHS these
days; that there's more added than removed is mostly due to more
tests and some glue code to ensure nevow backwards compatibilty).
Version 1.4 (2019-10-15)
* New XMLGrammar to parse from generic XML (e.g., VOEvent).
* New experimental ADQL features: TABLESAMPLE (might become standard in
2.2), and GENERATE_SERIES (who knows?)
* Working around a bug in twisted from Debian buster that prevents
TOPCAT uploads from working.
* Updating DDs no longer re-make their dependencies unless they create
tables. If dependents really need to be remade, add
remakeOnDataChange=True to the *dependent* DD. This should make
updates of obscore-published tables quite a bit faster.
* Datalink descriptorGenerators may now return None to indicate
a dataset does not exist.
* DaCHS now produces an informative Server header.
* Lots of minor improvements and fixes.
Version 1.3 (2019-05-28)
* New //ssap#view mixin that should be used for future SSAP services,
and that existing SSAP services should migrate to at some point.
See https://blog.g-vo.org/a-new-view-on-ssap-in-dachs/ for details.
* Columns can now be hidden from TAP/ADQL (and other interfaces) by
setting hidden="True".
* There is now [web]maxSyncUploadSize=500k as the default upload limit
on sync queries. In compensation, clients uploading too much now receive a
more useful error message.
* Adding support for https (certificate reading, certificate updating
with letsencrypt, registering alternate endpoints, no WebSAMP with https).
See https://blog.g-vo.org/https-in-dachs/ for details.
* New source_table and preview columns in obscore. If you're using
the various obscore mixins, this should be automatic. If you have
defined views manually, you will have to amend these (and have a
broken obscore until a dachs upgrade ran without error).
* No longer producing arraysize="1" for scalars (except char, for
compatibility with a legacy TOPCAT workaround).
* Support for draft TIMESYS in VOTable (with STC 2 annotation; ask for
details if you're interested).
* You can now add targetType and targetTitle properties to URL-valued
columns to help Aladin figure out what to do with URLs (see
http://docs.g-vo.org/DaCHS/ref.html#datalinks-as-product-urls).
* New gavo_transform, gavo_ipix, and gavo_urlescale ufuncs for ADQL,
fixed gavo_urlescape to have acceptable performance.
* Now generating CatalogResource records with auxiliary capabilities
in accordance with Oct 2018 VODataService WD.
* //soda#sdm_genDesc now matches accref rather than pubDID by default.
If you use Datalink with SSA and have a custom pubDID schema (or no index
on accref), add a useAccref="False" to your descriptorGenerator statement.
* There is now a --foreground option for dachs serve start (this is
mainly to play nice with systemd).
* Fixes for various bugs (most notable: '' in ADQL, WCS in SIAP cutout
products).
Version 1.2 (2018-07-17)
* New dachs start command to produce structured templates for certain
service types.
* Support for ADQL 2.1 draft, including almost all of the optional
parts (run dachs imp //adql for some nice new UDFs).
* New coverage element (with updaters) to build and declare the
space-time-spectral coverage of a resource.
* New odbcGrammar to pull rawdicts from remote databases.
* You can now declare associated datalink services for tables (the
_associatedDatalinkSvc meta item).
* We now force matplotlib to read its configuration from
configDir/matplotlibrc; to get a default, just run dachs init.
* Now supporting VOSI 1.1; in particular, DaCHS now understands the
detail hints and has per-table endpoints.
* The indices generated by the ssa mixins are now a bit more sensible
considering typical query modes. You probably want to dachs imp -I
your ssap data collections as convenient.
* ssapCore no longer wantonly adds preview columns. If you have
previews with spectra, you probably want to add
auto to your ssapCores.
* You can now add a statisticsTarget property to columns; you will
want this on largeish tables with non-uniformly distributed values
to aid the query planner.
* DaCHS's log now by default does not contain IP addresses, user agents,
and referrers any more, which should mostly keep you from processing
personal data. To get back the previous behaviour, set
[web]logFormat in /etc/gavo.rc to "combined".
* As usual, many minor bug fixes and improvements (e.g., memmapping
FITSes for cutout again, delimited table references in ADQL, new-style
tutorial resource records, correct obscore standardId)
Version 1.1 (2017-12-01)
* DaCHS now officially requires python 2.7.
* Now supporting TAP 1.1; in particular, TOP n doesn't trump MAXREC
any more, and it doesn't affect OVERFLOW indication. Also, TAP_SCHEMA
is updated (this happens as a side effect of dachs upgrade).
* Now serialising spoint, scircle, and friends to DALI 1.1 xtypes
(timestamp, point, polygon, circle). Fields explicitly marked with
adql:POINT or adql:REGION will still be serialised to STC-S. Do this
only if you have no choice (DaCHS has this for obscore and epntap s_region
right now).
* Sanitised output column selection. This may make for slight changes
in service responses, in particular in VOTable formats. See
http://docs.g-vo.org/DaCHS/ref.html#output-tables
* DaCHS no longer comes with an outdated version pyparsing and instead
uses what's installed on the system. The Debian package further
re-uses additional system resources if available (rjsmin, jquery).
* DaCHS now tries a bit harder to come up with sensible names for
SODA result files.
* map/@source is no longer limited to identifier-like strings; any key
that's in your source is fair game.
* For incremental imports with data that's updated now and then,
there's now ignoreSources/@fromdbUpdating.
* relative imports from custom code ("import foo" in a custom core,
for instance, getting res/foo.py) no longer work. See
http://docs.g-vo.org/DaCHS/ref.html#importing-modules for details.
* fixed a severe bug in the creation of obscore metadata from SSAP
tables. If you use //obscore#publishSSAPHCD or //obscore#publishSSAPMIXC
mixins, update the obscore definitions by running dachs imp -m ,
followed by dachs imp //obscore (only necessary once at the end).
* You can now define a footer.html template that's added at the foot
of the main page content.
Version 1.0 (2017-07-11)
* DaCHS' main entry point is now actually called dachs (i.e., call
dachs imp q and such in the future). gavo will work as an alias
for quite a while to come, though.
* UWS support is now for version 1.1 (i.e., there's creationDate,
filters in the joblist, and slow polling).
* Added "declarative" licenses. Please read the Licensing chapter in
the tutorial and slap licenses on your data.
* Now using astropy.wcs instead of pywcs, and astropy.io.fits instead of
pyfits. The respective APIs have, unfortunately, changed quite a bit.
If you used them (e.g., in processors), you'll have to change your
code; it's unlikely services are impacted at runtime.
(see also http://docs.g-vo.org/DaCHS/howDoI.html#update-my-code).
* Removed //epntap#table-2_0. Use //epntap2#table-2_0 instead (sorry).
* Removed sdmCore (use SODA instead)
* Removed SODA procs in //datalink (use the ones from //soda instead)
* Removed imp -u flag and the corresponding updateMode parse option.
If you used that or the uploadCore, just mark the dds involved
with updating="True".
* Massive sanitation of input parameter processing. If you've been
using inputTable, inputDD, or have been doing creative things with
inputKeys, please check the respective services carefully. See
also "Building Service Interface" in the reference documentation.
Most user-visible change: If you've been using repeated parameters
to fill array-valued inputs, that's no longer allowed; if you actually
must have this kind of thing, you'll need a custom core and must fill
the arrays by hand.
* In DaCHS' SQL interface, tuples now are matched to records and lists
to arrays (was the other way round before). If while importing you
manually created lists to fill to array-like columns, you'll have to
make tuples from these now.
* rsc.makeData or rsc.TableForDef no longer automatically make connections
when used on database tables. You must give them explicit
connection arguments now ("with base.getTableConn() as conn:").
* logo_tiny and logo_big are now ignored, all logos spit out by DaCHS are
based on logo_medium.png, including, if not overridden, the favicon
(that you will now get if you have not set it before).
* Removed (probably largely unused features) editCore, SDM2 support,
pkg_resource overrides, simpleView, computedCore.
* Removed argparse module shipped with DaCHS. This breaks compatibility
with python 2.6 (although you can still run DaCHS with a manually
installed argparse.py in 2.6).
* Hopefully more useful manpage -- have a peek!
Version 0.9.8 (2016-12-08)
* SODA/Datalink implementation is now considered stable. Go forth and
make fabulous links and cutout services. Check the overhauled datalink
section in the reference documentation if you have a couple of minutes:
http://docs.g-vo.org/DaCHS/ref.html#datalink-and-soda
* TAP_SCHEMA columns now has new column_index column. Its addition should
be automatic as long as your TAP-published RDs are in order.
* EPN-TAP 2.0 changed some column and parameter names, and the mixin
followed. If you've already used EPN-TAP 2.0, you will have to re-import
your data and slightly adapt your RD (e.g., c1_min -> c1min,
t_exp_min -> time_exp_min etc).
* DaCHS now emits warnings for columns whose names are SQL or ADQL
reserved names (this includes popular items like area, size, and distance).
Even if things have worked so far, please change these names. We
guarantee you'll regret it if you don't.
* Full support for VOResource 1.1 relationships
(http://docs.g-vo.org/vocab-test/relationship_type). Please try them
if you have appropriate data.
Version 0.9.7 (2016-08-18)
* Behaviour change: pythonCores no longer run in a file system sandbox
(can't do that thread-safely).
* Support for SIAP version 2 (see tutorial for details)
* The default VOTable version delivered is now VOTable 1.3 (actually,
it's been 1.3 for a while now, we just didn't declare it)
* support for Obscore 1.1 (update should be automatic)
* support for EPN-TAP 2.0 (manually change to the #table-2.0 mixin)
* DaCHS now supports CORS for web-based UIs
* DaCHS's ADQL cross matches should now have "good" query plans
regardless or argument order (but are half as fast as in previous.
"good" cases).
* SCS services publishing TAP-readable tables now get an auxiliary
TAP publication automatically (you'll need to re-gavo pub them,
though, if you want the Registry to notice quickly).
* Removed bbox-based SIAP because DaCHS has been requiring pgsphere
for quite a while now, and hence there's not need for a lousy fallback.
* gavo admin xsdVal and XSD-validating regression tests now work
even with package installations; so: you got your own schema mirror now!
* New gavo admin updateTAPSchema command for syncing TAP_SCHEMA with
the RDs.
* New built-in preview-managing scheme for large data collections.
* As usual, many small bug fixes and new features.
Version 0.9.6 (2016-03-16)
* Several new indices are now created by default; you probably want them,
too. Run "gavo imp -I //products". Similarly, gavo imp -I on all
RDs that have SSA tables with datalink services (if you don't know what
that is, you don't have it)
* New soda RD containing (almost) all to build a WD-compliant cube
cutout service (and a bit more). You can update any existing
services you have by changing "datalink" to "soda" in the referenced
procDefs.
* API change: base.makeMetaItem and makeMetaValue were a pain. They
have simply gone in the expectation that nobody has actually used them.
Protest if we're wrong. Also there is no type attribute on meta
any more. Also, no meta/@type any more in RDs. I hope you've not
really used *that*.
* API change: When CondDescs yielded more than one fragement,
the fragments were combined with AND so far. Now, it's OR by
default (you can restore the previoius behaviour using condDesc/@joiner)
* Some changes in the way the selection and evaluation of sort keys work
in particular in web interfaces. If you've been playing games with this,
check if what you did still works.
* Support for DALI 1.1 xtype="interval" parameters.
* New recommended way to specify author lists; see
http://docs.g-vo.org/DaCHS/tutorial.html#authors
* As usual, many small bug fixes and new features.
Version 0.9.5 (2015-10-07)
* **RD breakage**: The old DaCHS implemenation got the meaning of
spectralSI and fluxSI completely wrong. You will fairly certainly have
to change RDs doing SSA. Any occurrence of spectralSI will have to become
spectralUnit, and fluxSI fluxUnit (run gavo val ALL to catch RDs where
that's necessary). While you could add correct spectralSI and fluxSI
values (which would look like "10e-10 L" or so), the DaCHS authors
currently consider that a waste of time.
* Possible service breakage: We're now using VOTable tabledata parsers
to parse the values of PARAMs; the same code is used to parse HTTP
parameters. If you define custom InputKeys and got metadata slightly
wrong, these services might fail now. If you have InputKeys anywhere
in your RDs, make sure the corresponding services still behave as
expected.
* Column naming is now ADQL-compliant, which means that leading underscores
are no longer considered ok for regular identifiers (though they are
fine to postgres). You should run a ``gavo val ALL`` and fix any warnings
to the effect that something is not a "regular ADQL identifier" you may
get. Also read
http://docs.g-vo.org/DaCHS/commonproblems.html#column-tab-foo-is-not-a-regular-adql-identifier
Since there's no harm, but it may automatically fix things, just run
gavo imp -m //tap
after upgrading.
* SLAP support: If you have data on spectral lines, DaCHS now speaks
the protocol to distribute them.
* Generic UWS support: If you want to provide non-TAP UWS services
(like large server-side compute jobs and such), there's now the uws.xml
renderer (in combination with a pythonCore or customCore).
Details at http://docs.g-vo.org/DaCHS/ref.html#custom-uwses
* Sidebar style has changed; actually removed the old support code, so
the old behaviour cannot be restored by templates and javascript alone,
but I'd be happy to provide you with the necessary hooks.
* By popular demand: The "_r" column (in cone searches, distance between
the cone center and the actual object) is there. To get it, use
an scsCore rather than a dbCore in your scs services. It works with
the form renderer, too.
* In VO services (scs.xml, siap.xml, ssap.xml), even fields from an
explicit output table are now filtered by verbLevel. This means
that fields you deemed unimportant might no longer show up in default
answers. To get them back, either lower their verbLevel or add
a query parameter VERB=3.
Version 0.9.4 (2015-02-03)
* Uniform referencing to RDs and elements in them, allowing to skip
input-relative paths in all of gavo's subcommands.
* Some fixes for Debian jessie compatibility.
* fitsProdGrammar supports compressed FITSes transparently.
* New templating system for writing more beautiful FITS headers
in processing (see kapteyn/q in hdinputs for examples)
* New grammar for working with CDF files
* New api renderer for DAL-style machine-oriented services (see
sp_ace/q in hdinputs for an example)
* Transparent support for gzip compression on selected documents
* We now officially force SIAP services to have their bandpasses in m
(if this breaks your service, it's been broken before).
* Possibly bookmark-relevant change: _DBOPTIONS_LIMIT is now MAXREC
* Possibly scripting-relevant change: --ui option on gavo imp now
needs to be given before the imp.
* Possibly registry-relevant change: this upgrade will change your
TAP service's IVORN (to ivo:///tap; we admit that was
a dumb idea, but reverting now will only make things worse)
Version 0.9.3 (2014-09-04)
* Improved EPN-TAP support such that it actually works (this release is
mainly for visitors of EPSC); this includes a simple interface to
pyPDS.
* New "api" renderer that mimics DAL parameter style (including UPLOAD
and metadata VOTable) but doesn't assume particular data semantics.
* The registry entry for the built-in TAP service now uses the vanity
name; if you have published your TAP service to the Registry, run
gavo pub //tap to have a less ugly IVORN and access URL (nothing will
break if you don't).
Version 0.9.2 (2014-08-14)
* !Important!: Internal changes to the way mixins work cause some column
sequences to change. In particular, as a consequence you must re-import all
tables with non-trivial mixins. We are really sorry about that. Gavo
upgrade will warn about tables it finds that need re-importing.
* DaCHS can now produce JSON output (with certain not yet VO-standardised
conventions) from TAP and other services.
* TAP_SCHEMA type names are automatically updated to comply to
recently established conventions. If you have tricked weird type names
into TAP_SCHEMA (you'll know if you have), please ensure DaCHS hasn't
messed it up.
* DaCHS does not produce DataCollection registry records any more; they
are all CatalogServices with (TAP) services exposing the data
given as capabilities directly.
* The generation of examples (formerly TAP examples) is now done
completely differently. If you have material in __system/adqlexamples,
you'll have to move it to etc/userconfig.rd (see doc on the tap
renderer).
* DaCHS now has a hard dependency on python-lxml (apt-get it if on
Debian and not running from package, otherwise see http://lxml.de/)
Version 0.9.1 (2014-04-30)
* New regression testing framework derived from a local hack we've had
in the GAVO DC for a while now. Try it, it's your friend. See the
http://docs.g-vo.org/DaCHS/ref.html#regression-testing
* New framework for having operator-defined previews. Please have a look
at http://docs.g-vo.org/DaCHS/ref.html#product-previews for details.
An unfortunate side effect is that all your cached previews are now
invalid. Remove all files in (presumably) /var/gavo/web/previewCache.
* Column selection for non-HTML now always includes all columns explicitely
given in the output table (unless supressed by a displayHint). If
you lie about types in your output table (e.g., declare string types
for things that are really ints), you may now get errors on VOTable
generation. Fix the types or have a noxml=true displayHint in these
cases.
* Removed the table attribute on ForeignKeys (that's been deprecated
for more than a year now). Use the inTable attribute with an
actual table reference instead.
* Datalink support bumped to WD-2014-02-28
Version 0.9 (2013-12-13)
* API CHANGE: If you have custom logic (i.e., phraseMakers) on
string inputKeys with values, you'll now get back lists of strings
by default. This will break your phraseMakers in that situation.
Solution: Add multiplicity="single" to the inputKeys.
* Change to latest VOUnit version. Many unit strings that previously
appeared ok no longer are; in particular, exponentiation is now
** exclusively and no longer ^. You'll get warnings with gavo val.
* Added support for Datalink for both cutout-like jobs and as a
replacement for the tentative getData proposal. See the reference
documentation for how to migrate from getData to Datalink.
* No longer using sesame's SOAP interface (that's been shut down by now)
* gavo mkboost now takes the id of the directGrammar rather than a
pair of rd id and and table id. It can now also parse from fits
binary tables directly.
* Datalink prototype changed during development: the parameter is now
called ID in conformance with the WD; what was pubdid in the datalink
functions is now called pubDID; several other changes that only
concern you if you're running datalink (chances are you're not)
* Any resource using \standardPubDID or similar needs to be re-imported.
As that stuff hasn't been there in the last release, this probably
doesn't concern you either.
Version 0.8.9 (2013-07-13)
* We now officially require postgres >9.0. If this hurts you, tell
us, this is mainly a convenience matter.
* DaCHS now supports the new BINARY2 VOTable encoding
* Internal rework of web-facing parameter processing. If you use
custom condDescs, test those services throughly (though we've
tried to maintain backwards compatibility)
* TAP examples are now DALI compliant
* There's now a facility to add custom columns to the ObsCore table.
* SIAP processing rework. If you have SIAP services, you should probably
re-import them, since the old SIAP processing sometimes was a bit
too lenient.
* SIAP and SCS now count as "PQL" renderers, as SSAP before. This
means custom parameters created via buildFrom will allow range and list
syntaxes as for SSAP.
* Many minor updates, fixes, and compatibility improvements.
* New root-tree.html template that you can optionally use as the
root page; it provides some ajaxy alternative to the old root page.
Version 0.8.4 (2012-12-17)
* New gavo upgrade command -- run this after every svn update or other
upgrade (Debian packages to that automatically), and you should not
have to worry about schema changes.
* SIAP and SSAP dateObs columns are now MJD. Unless you have custom
web interfaces or feed the tables by some other means than the
setMeta proc applys, you should not notice (after a gavo upgrade).
If your SIAP/SSAP tables are part of any views except ivoa.obscore,
drop those views before running gavo upgrade, and check those services
after upgrading, anyway.
Version 0.8 (2012-08-15)
* The schema for the DaCHS-internal metadata tables changed. To adapt
your database, run
gavo imp --system //dc_tables
(to fill the tables again, say gavo pub -ma, but you'll do so in the
next step anyway)
* There's now a table tap_schema.groups. To create it, say:
gavo imp //tap createSchema
Do not give --system here unless you already have the tap_schema.groups
table; if you do give --system, the existing tables will be re-created and
you will have to say gavo pub -ma to re-fill them.
* The schema of the service list has changed again, and the changes
require a republication of all services. After the update, run
gavo imp --system //services
gavo pub -ma
* Some ADQL user-defined functions are now implemented as Postgres
stored procedures. Say::
gavo imp //adql
define those.
* The UWS code now writes to the UWS tables with less privileges.
To let it do that, you must update the privileges on all queue tables;
in standard DaCHS, that's only the TAP table, so simply say:
gavo imp -m //tap createJobTable
* The organization running the dc is now configurable. If you are
running the registry interface, you must now give authority.managingOrg,
organization.title, organization.description, and
organization.referenceURL in defaultmeta.txt.
authority.managingOrg usually is ivo:///org.
* The former readRoles and allRoles attribute of table and resource
elements are now called readProfiles and allProfiles and contain
profile names rather than DB roles. In the unlikely case you're
using this feature, you'd have to change the RDs.
* You can no longer map db profile "roles" to actual profile names in
the gavorc [profiles] section. If you actually need this, try
symbolic links to the profile files.
Version 0.7 (2012-01-12)
* Automatic dependency management for published resources (register and
adql). You must run
gavo imp --system //services upgrade_0.6.3_0.7
after the update.
* UWS job representation in the database changed. You must remove all
UWS jobs before upgrading. After upgrading, run
gavo imp //uws upgrade_0.6.3_0.7
gavo imp --system //tap createJobTable
You can ignore error messages during the first of those commands.
vi:et:sw=2:sta