Note: Of course, in each version, many minor improvements and bug fixes 
go in.  To see the full history, use  

svn log http://svn.ari.uni-heidelberg.de/svn/gavo/python/trunk/ | less

to see the gory details.  If anything not mentioned here breaks something
for you, we'd consider it a severe bug, though.

Version 1.4 (2019-10-15)

	* New XMLGrammar to parse from generic XML (e.g., VOEvent).

	* New experimental ADQL features: TABLESAMPLE (might become standard in
	  2.2), and GENERATE_SERIES (who knows?)

	* Working around a bug in twisted from Debian buster that prevents
	  TOPCAT uploads from working.

	* Updating DDs no longer re-make their dependencies unless they create
	  tables.  If dependents really need to be remade, add 
	  remakeOnDataChange=True to the *dependent* DD.  This should make
	  updates of obscore-published tables quite a bit faster.
	
	* Datalink descriptorGenerators may now return None to indicate
	  a dataset does not exist.
	
	* DaCHS now produces an informative Server header.
	
	* Lots of minor improvements and fixes.


Version 1.3 (2019-05-28)

	* New //ssap#view mixin that should be used for future SSAP services,
	  and that existing SSAP services should migrate to at some point.
	  See  https://blog.g-vo.org/a-new-view-on-ssap-in-dachs/ for details.

	* Columns can now be hidden from TAP/ADQL (and other interfaces) by
	  setting hidden="True".

	* There is now [web]maxSyncUploadSize=500k as the default upload limit
	  on sync queries.  In compensation, clients uploading too much now receive a
	  more useful error message.

	* Adding support for https (certificate reading, certificate updating
	  with letsencrypt, registering alternate endpoints, no WebSAMP with https).
	  See https://blog.g-vo.org/https-in-dachs/ for details.

	* New source_table and preview columns in obscore.  If you're using
		the various obscore mixins, this should be automatic.  If you have
		defined views manually, you will have to amend these (and have a 
		broken obscore until a dachs upgrade ran without error).

	* No longer producing arraysize="1" for scalars (except char, for
	  compatibility with a legacy TOPCAT workaround).

  * Support for draft TIMESYS in VOTable (with STC 2 annotation; ask for 
    details if you're interested).

  * You can now add targetType and targetTitle properties to URL-valued
  	columns to help Aladin figure out what to do with URLs (see
  	http://docs.g-vo.org/DaCHS/ref.html#datalinks-as-product-urls).

  * New gavo_transform, gavo_ipix, and gavo_urlescale ufuncs for ADQL,
    fixed gavo_urlescape to have acceptable performance.

	* New generating CatalogResource records with auxiliary capabilities
	  in accordance with Oct 2018 VODataService WD.
	
	* //soda#sdm_genDesc now matches accref rather than pubDID by default.  
		If you use Datalink with SSA and have a custom pubDID schema (or no index
		on accref), add a useAccref="False" to your descriptorGenerator statement.
	
	* There is now a --foreground option for dachs serve start (this is
	  mainly to play nice with systemd).

	* Fixes for various bugs (most notable: '' in ADQL, WCS in SIAP cutout
	  products).


Version 1.2 (2018-07-17)

	* New dachs start command to produce structured templates for certain
	  service types.

	* Support for ADQL 2.1 draft, including almost all of the optional
	  parts (run dachs imp //adql for some nice new UDFs).

  * New coverage element (with updaters) to build and declare the 
  	space-time-spectral coverage of a resource.
 
  * New odbcGrammar to pull rawdicts from remote databases.

 	* You can now declare associated datalink services for tables (the
 	  _associatedDatalinkSvc meta item).

 	* We now force matplotlib to read its configuration from 
 		configDir/matplotlibrc; to get a default, just run dachs init.
  
  * Now supporting VOSI 1.1; in particular, DaCHS now understands the
    detail hints and has per-table endpoints.

	* The indices generated by the ssa mixins are now a bit more sensible
	  considering typical query modes.  You probably want to dachs imp -I
	  your ssap data collections as convenient.

	* ssapCore no longer wantonly adds preview columns.  If you have
	  previews with spectra, you probably want to add 
	  <property name="previews">auto</property> to your ssapCores.

	* You can now add a statisticsTarget property to columns; you will
	  want this on largeish tables with non-uniformly distributed values
	  to aid the query planner.

	* DaCHS's log now by default does not contain IP addresses, user agents,
	  and referrers any more, which should mostly keep you from processing
	  personal data.  To get back the previous behaviour, set 
	  [web]logFormat in /etc/gavo.rc to "combined".

	* As usual, many minor bug fixes and improvements (e.g., memmapping
	  FITSes for cutout again, delimited table references in ADQL, new-style
	  tutorial resource records, correct obscore standardId)


Version 1.1 (2017-12-01)

	* DaCHS now officially requires python 2.7.

	* Now supporting TAP 1.1; in particular, TOP n doesn't trump MAXREC
	  any more, and it doesn't affect OVERFLOW indication.  Also, TAP_SCHEMA
	  is updated (this happens as a side effect of dachs upgrade).

	* Now serialising spoint, scircle, and friends to DALI 1.1 xtypes 
	  (timestamp, point, polygon, circle).  Fields explicitly marked with
	  adql:POINT or adql:REGION will still be serialised to STC-S.  Do this
	  only if you have no choice (DaCHS has this for obscore and epntap s_region
	  right now).
	  
	* Sanitised output column selection.  This may make for slight changes
	  in service responses, in particular in VOTable formats.  See
	  http://docs.g-vo.org/DaCHS/ref.html#output-tables

	* DaCHS no longer comes with an outdated version pyparsing and instead 
	  uses what's installed on the system.  The Debian package further
	  re-uses additional system resources if available (rjsmin, jquery).

	* DaCHS now tries a bit harder to come up with sensible names for
	  SODA result files.

	* map/@source is no longer limited to identifier-like strings; any key
	  that's in your source is fair game.
	
	* For incremental imports with data that's updated now and then,
	  there's now ignoreSources/@fromdbUpdating.

	* relative imports from custom code ("import foo" in a custom core,
	  for instance, getting res/foo.py) no longer work.  See 
	  http://docs.g-vo.org/DaCHS/ref.html#importing-modules for details.

	* fixed a severe bug in the creation of obscore metadata from SSAP
	  tables.  If you use //obscore#publishSSAPHCD or //obscore#publishSSAPMIXC
	  mixins, update the obscore definitions by running dachs imp -m <rdid>,
	  followed by dachs imp //obscore (only necessary once at the end).

	* You can now define a footer.html template that's added at the foot
	  of the main page content.


Version 1.0 (2017-07-11)

	* DaCHS' main entry point is now actually called dachs (i.e., call 
		dachs imp q and such in the future).  gavo will work as an alias
		for quite a while to come, though.

	* UWS support is now for version 1.1 (i.e., there's creationDate,
	  filters in the joblist, and slow polling).
	
	* Added "declarative" licenses.  Please read the Licensing chapter in
		the tutorial and slap licenses on your data.

	* Now using astropy.wcs instead of pywcs, and astropy.io.fits instead of
	  pyfits.  The respective APIs have, unfortunately, changed quite a bit.
	  If you used them (e.g., in processors), you'll have to change your
	  code; it's unlikely services are impacted at runtime.
	  (see also http://docs.g-vo.org/DaCHS/howDoI.html#update-my-code).

	* Removed //epntap#table-2_0.  Use //epntap2#table-2_0 instead (sorry).

	* Removed sdmCore (use SODA instead)

	* Removed SODA procs in //datalink (use the ones from //soda instead)

	* Removed imp -u flag and the corresponding updateMode parse option.
	  If you used that or the uploadCore, just mark the dds involved
	  with updating="True".

	* Massive sanitation of input parameter processing.  If you've been
	  using inputTable, inputDD, or have been doing creative things with
	  inputKeys, please check the respective services carefully.  See
	  also "Building Service Interface" in the reference documentation.
	  Most user-visible change: If you've been using repeated parameters
	  to fill array-valued inputs, that's no longer allowed; if you actually
	  must have this kind of thing, you'll need a custom core and must fill
	  the arrays by hand.
	
	* In DaCHS' SQL interface, tuples now are matched to records and lists
	  to arrays (was the other way round before).  If while importing you
	  manually created lists to fill to array-like columns, you'll have to
	  make tuples from these now.
	
	* rsc.makeData or rsc.TableForDef no longer automatically make connections
	  when used on database tables.  You must give them explicit
	  connection arguments now ("with base.getTableConn() as conn:").

	* logo_tiny and logo_big are now ignored, all logos spit out by DaCHS are
	  based on logo_medium.png, including, if not overridden, the favicon
	  (that you will now get if you have not set it before).

	* Removed (probably largely unused features) editCore, SDM2 support,
		pkg_resource overrides, simpleView, computedCore.

	* Removed argparse module shipped with DaCHS.  This breaks compatibility
	  with python 2.6 (although you can still run DaCHS with a manually
	  installed argparse.py in 2.6).

	* Hopefully more useful manpage -- have a peek!


Version 0.9.8 (2016-12-08)

	* SODA/Datalink implementation is now considered stable.  Go forth and
	  make fabulous links and cutout services.  Check the overhauled datalink 
	  section in the reference documentation if you have a couple of minutes:
		http://docs.g-vo.org/DaCHS/ref.html#datalink-and-soda

	* TAP_SCHEMA columns now has new column_index column.  Its addition should
	  be automatic as long as your TAP-published RDs are in order.

	* EPN-TAP 2.0 changed some column and parameter names, and the mixin
	  followed.  If you've already used EPN-TAP 2.0, you will have to re-import
	  your data and slightly adapt your RD (e.g., c1_min -> c1min, 
	  t_exp_min -> time_exp_min etc).

	* DaCHS now emits warnings for columns whose names are SQL or ADQL 
		reserved names (this includes popular items like area, size, and distance).
		Even if things have worked so far, please change these names.  We 
		guarantee you'll regret it if you don't.

	* Full support for VOResource 1.1 relationships 
		(http://docs.g-vo.org/vocab-test/relationship_type).  Please try them
		if you have appropriate data.


Version 0.9.7 (2016-08-18)

	* Behaviour change: pythonCores no longer run in a file system sandbox 
		(can't do that thread-safely).

	* Support for SIAP version 2 (see tutorial for details)

	* The default VOTable version delivered is now VOTable 1.3 (actually,
	  it's been 1.3 for a while now, we just didn't declare it)

	* support for Obscore 1.1 (update should be automatic)

	* support for EPN-TAP 2.0 (manually change to the #table-2.0 mixin)

	* DaCHS now supports CORS for web-based UIs

	* DaCHS's ADQL cross matches should now have "good" query plans 
	  regardless or argument order (but are half as  fast as in previous.
	  "good" cases).

	* SCS services publishing TAP-readable tables now get an auxiliary
	  TAP publication automatically (you'll need to re-gavo pub them,
	  though, if you want the Registry to notice quickly).

	* Removed bbox-based SIAP because DaCHS has been requiring pgsphere 
	  for quite a while now, and hence there's not need for a lousy fallback.

	* gavo admin xsdVal and XSD-validating regression tests now work
	  even with package installations; so: you got your own schema mirror now!

	* New gavo admin updateTAPSchema command for syncing TAP_SCHEMA with
	  the RDs.

	* New built-in preview-managing scheme for large data collections.

	* As usual, many small bug fixes and new features.


Version 0.9.6 (2016-03-16)

  * Several new indices are now created by default; you probably want them,
    too.  Run "gavo imp -I //products". Similarly, gavo imp -I <rd-id> on all
    RDs that have SSA tables with datalink services (if you don't know what
    that is, you don't have it)

  * New soda RD containing (almost) all to build a WD-compliant cube
    cutout service (and a bit more).  You can update any existing
    services you have by changing "datalink" to "soda" in the referenced
    procDefs.

  * API change: base.makeMetaItem and makeMetaValue were a pain.  They
		have simply gone in the expectation that nobody has actually used them.
		Protest if we're wrong.  Also there is no type attribute on meta
		any more.  Also, no meta/@type any more in RDs.  I hope you've not
		really used *that*.

	* API change: When CondDescs yielded more than one fragement,
	  the fragments were combined with AND so far.  Now, it's OR by
	  default (you can restore the previoius behaviour using condDesc/@joiner)

	* Some changes in the way the selection and evaluation of sort keys work
	  in particular in web interfaces.  If you've been playing games with this,
	  check if what you did still works.

	* Support for DALI 1.1 xtype="interval" parameters.

	* New recommended way to specify author lists; see
	  http://docs.g-vo.org/DaCHS/tutorial.html#authors

	* As usual, many small bug fixes and new features.


Version 0.9.5 (2015-10-07)

	* **RD breakage**: The old DaCHS implemenation got the meaning of
		spectralSI and fluxSI completely wrong.  You will fairly certainly have
		to change RDs doing SSA.  Any occurrence of spectralSI will have to become
		spectralUnit, and fluxSI fluxUnit (run gavo val ALL to catch RDs where
		that's necessary).  While you could add correct spectralSI and fluxSI
		values (which would look like "10e-10 L" or so), the DaCHS authors
		currently consider that a waste of time.

	* Possible service breakage: We're now using VOTable tabledata parsers
	  to parse the values of PARAMs; the same code is used to parse HTTP
	  parameters.  If you define custom InputKeys and got metadata slightly
	  wrong, these services might fail now.  If you have InputKeys anywhere
	  in your RDs, make sure the corresponding services still behave as
	  expected.

	* Column naming is now ADQL-compliant, which means that leading underscores
	  are no longer considered ok for regular identifiers (though they are
	  fine to postgres).  You should run a ``gavo val ALL`` and fix any warnings
	  to the effect that something is not a "regular ADQL identifier" you may
	  get.  Also read 
	  http://docs.g-vo.org/DaCHS/commonproblems.html#column-tab-foo-is-not-a-regular-adql-identifier
	  Since there's no harm, but it may automatically fix things, just run

	  	gavo imp -m //tap
	 	
	 	after upgrading.

	* SLAP support: If you have data on spectral lines, DaCHS now speaks
		the protocol to distribute them.

	* Generic UWS support: If you want to provide non-TAP UWS services
		(like large server-side compute jobs and such), there's now the uws.xml
		renderer (in combination with a pythonCore or customCore).
		Details at http://docs.g-vo.org/DaCHS/ref.html#custom-uwses

	* Sidebar style has changed; actually removed the old support code, so
	  the old behaviour cannot be restored by templates and javascript alone,
	  but I'd be happy to provide you with the necessary hooks.

	* By popular demand: The "_r" column (in cone searches, distance between
		the cone center and the actual object) is there.  To get it, use
		an scsCore rather than a dbCore in your scs services.  It works with
		the form renderer, too.

	* In VO services (scs.xml, siap.xml, ssap.xml), even fields from an
	  explicit output table are now filtered by verbLevel.  This means
	  that fields you deemed unimportant might no longer show up in default
	  answers.  To get them back, either lower their verbLevel or add
	  a query parameter VERB=3.


Version 0.9.4 (2015-02-03)

  * Uniform referencing to RDs and elements in them, allowing to skip
    input-relative paths in all of gavo's subcommands.

  * Some fixes for Debian jessie compatibility.

  * fitsProdGrammar supports compressed FITSes transparently.

  * New templating system for writing more beautiful FITS headers
    in processing (see kapteyn/q in hdinputs for examples)

  * New grammar for working with CDF files

  * New api renderer for DAL-style machine-oriented services (see
    sp_ace/q in hdinputs for an example)

  * Transparent support for gzip compression on selected documents

  * We now officially force SIAP services to have their bandpasses in m
    (if this breaks your service, it's been broken before).

  * Possibly bookmark-relevant change: _DBOPTIONS_LIMIT is now MAXREC

  * Possibly scripting-relevant change: --ui option on gavo imp now
    needs to be given before the imp.

  * Possibly registry-relevant change: this upgrade will change your
    TAP service's IVORN (to ivo://<authority>/tap; we admit that was
    a dumb idea, but reverting now will only make things worse)


Version 0.9.3 (2014-09-04)

  * Improved EPN-TAP support such that it actually works (this release is
    mainly for visitors of EPSC); this includes a simple interface to
    pyPDS.

  * New "api" renderer that mimics DAL parameter style (including UPLOAD
    and metadata VOTable) but doesn't assume particular data semantics.
  
  * The registry entry for the built-in TAP service now uses the vanity
    name; if you have published your TAP service to the Registry, run 
    gavo pub //tap to have a less ugly IVORN and access URL (nothing will
    break if you don't).


Version 0.9.2 (2014-08-14)
  
  * !Important!: Internal changes to the way mixins work cause some column
    sequences to change.  In particular, as a consequence you must re-import all
    tables with non-trivial mixins.  We are really sorry about that.  Gavo
    upgrade will warn about tables it finds that need re-importing.

  * DaCHS can now produce JSON output (with certain not yet VO-standardised
    conventions) from TAP and other services.

  * TAP_SCHEMA type names are automatically updated to comply to
    recently established conventions.  If you have tricked weird type names
    into TAP_SCHEMA (you'll know if you have), please ensure DaCHS hasn't 
    messed it up.

  * DaCHS does not produce DataCollection registry records any more; they
    are all CatalogServices with (TAP) services exposing the data
    given as capabilities directly.
  
  * The generation of examples (formerly TAP examples) is now done
    completely differently.  If you have material in __system/adqlexamples,
    you'll have to move it to etc/userconfig.rd (see doc on the tap
    renderer).

  * DaCHS now has a hard dependency on python-lxml (apt-get it if on
    Debian and not running from package, otherwise see http://lxml.de/)

Version 0.9.1 (2014-04-30)

  * New regression testing framework derived from a local hack we've had
    in the GAVO DC for a while now.  Try it, it's your friend.  See the
    http://docs.g-vo.org/DaCHS/ref.html#regression-testing

  * New framework for having operator-defined previews.  Please have a look
    at http://docs.g-vo.org/DaCHS/ref.html#product-previews for details.
    An unfortunate side effect is that all your cached previews are now 
    invalid.  Remove all files in (presumably) /var/gavo/web/previewCache.

  * Column selection for non-HTML now always includes all columns explicitely
    given in the output table (unless supressed by a displayHint).  If
    you lie about types in your output table (e.g., declare string types
    for things that are really ints), you may now get errors on VOTable
    generation.  Fix the types or have a noxml=true displayHint in these
    cases.
  
  * Removed the table attribute on ForeignKeys (that's been deprecated
    for more than a year now).  Use the inTable attribute with an
    actual table reference instead.
  
  * Datalink support bumped to WD-2014-02-28


Version 0.9 (2013-12-13)

  * API CHANGE: If you have custom logic (i.e., phraseMakers) on
    string inputKeys with values, you'll now get back lists of strings
    by default.  This will break your phraseMakers in that situation.
    Solution: Add multiplicity="single" to the inputKeys.

  * Change to latest VOUnit version.  Many unit strings that previously
    appeared ok no longer are; in particular, exponentiation is now
    ** exclusively and no longer ^.  You'll get warnings with gavo val.

  * Added support for Datalink for both cutout-like jobs and as a 
    replacement for the tentative getData proposal.  See the reference
    documentation for how to migrate from getData to Datalink.

  * No longer using sesame's SOAP interface (that's been shut down by now)

  * gavo mkboost now takes the id of the directGrammar rather than a
    pair of rd id and and table id.  It can now also parse from fits
    binary tables directly.

  * Datalink prototype changed during development: the parameter is now
    called ID in conformance with the WD; what was pubdid in the datalink
    functions is now called pubDID; several other changes that only
    concern you if you're running datalink (chances are you're not)

  * Any resource using \standardPubDID or similar needs to be re-imported.
    As that stuff hasn't been there in the last release, this probably
    doesn't concern you either.

Version 0.8.9 (2013-07-13)

  * We now officially require postgres >9.0.  If this hurts you, tell
    us, this is mainly a convenience matter.

  * DaCHS now supports the new BINARY2 VOTable encoding

  * Internal rework of web-facing parameter processing.  If you use
    custom condDescs, test those services throughly (though we've
    tried to maintain backwards compatibility)

  * TAP examples are now DALI compliant

  * There's now a facility to add custom columns to the ObsCore table.

  * SIAP processing rework.  If you have SIAP services, you should probably
    re-import them, since the old SIAP processing sometimes was a bit
    too lenient.

  * SIAP and SCS now count as "PQL" renderers, as SSAP before.  This
    means custom parameters created via buildFrom will allow range and list
    syntaxes as for SSAP.

  * Many minor updates, fixes, and compatibility improvements.

  * New root-tree.html template that you can optionally use as the
    root page; it provides some ajaxy alternative to the old root page.

Version 0.8.4 (2012-12-17)

  * New gavo upgrade command -- run this after every svn update or other
    upgrade (Debian packages to that automatically), and you should not
    have to worry about schema changes.

  * SIAP and SSAP dateObs columns are now MJD.  Unless you have custom
    web interfaces or feed the tables by some other means than the
    setMeta proc applys, you should not notice (after a gavo upgrade).
    If your SIAP/SSAP tables are part of any views except ivoa.obscore,
    drop those views before running gavo upgrade, and check those services 
    after upgrading, anyway.


Version 0.8 (2012-08-15)

  * The schema for the DaCHS-internal metadata tables changed.  To adapt
    your database, run

      gavo imp --system //dc_tables

    (to fill the tables again, say gavo pub -ma, but you'll do so in the
    next step anyway)

  * There's now a table tap_schema.groups.  To create it, say:

      gavo imp //tap createSchema 

    Do not give --system here unless you already have the tap_schema.groups
    table; if you do give --system, the existing tables will be re-created and
    you will have to say gavo pub -ma to re-fill them.

  * The schema of the service list has changed again, and the changes
    require a republication of all services.  After the update, run

      gavo imp --system //services
      gavo pub -ma

   * Some ADQL user-defined functions are now implemented as Postgres
    stored procedures.  Say::

      gavo imp //adql

    define those.

  * The UWS code now writes to the UWS tables with less privileges.
    To let it do that, you must update the privileges on all queue tables;
    in standard DaCHS, that's only the TAP table, so simply say:
    
      gavo imp -m //tap createJobTable

  * The organization running the dc is now configurable.  If you are
    running the registry interface, you must now give authority.managingOrg,
    organization.title, organization.description, and 
    organization.referenceURL in defaultmeta.txt.
    authority.managingOrg usually is ivo://<your authority>/org.

  * The former readRoles and allRoles attribute of table and resource
    elements are now called readProfiles and allProfiles and contain
    profile names rather than DB roles.  In the unlikely case you're
    using this feature, you'd have to change the RDs.

  * You can no longer map db profile "roles" to actual profile names in
    the gavorc [profiles] section.  If you actually need this, try
    symbolic links to the profile files.


Version 0.7 (2012-01-12)

  * Automatic dependency management for published resources (register and
    adql).  You must run 
    
      gavo imp --system //services upgrade_0.6.3_0.7

   after the update.

  * UWS job representation in the database changed.  You must remove all
    UWS jobs before upgrading.  After upgrading, run

      gavo imp //uws upgrade_0.6.3_0.7
      gavo imp --system //tap createJobTable

    You can ignore error messages during the first of those commands.