This is basically a mirror of the sources table of http://surveys.roe.ac.uk/ssa/dboverview.html Here's what Nigel Hambly had to say when sending the stuff to us: Date: Wed, 6 Feb 2013 12:17:03 +0000 From: Nigel Hambly To: "[iso-8859-1] Siegfried Röser" Cc: Elena Schilbach , Bertrand Goldman Subject: SuperCOSMOS bulk data set Hi Siegfried Further to yesterday's Skype telecon, here are the details of the SuperCOS dataset if you'd like to do a bulk transfer. The main science table, "Source" (which is equivalent to SDSS photoObj, or Pan-STARRS Average) is available here in 12 chunks: http://www-wfau.roe.ac.uk/www-data/ssa/source/ The 12 main files you want are ssaSource000ra030.bin 07-Jun-2010 18:02 24G ssaSource030ra060.bin 07-Jun-2010 19:25 23G ssaSource060ra090.bin 07-Jun-2010 22:40 32G ssaSource090ra120.bin 07-Jun-2010 22:59 39G ssaSource120ra150.bin 07-Jun-2010 23:28 35G ssaSource150ra180.bin 08-Jun-2010 08:06 33G ssaSource180ra210.bin 08-Jun-2010 08:22 33G ssaSource210ra240.bin 08-Jun-2010 08:42 39G ssaSource240ra270.bin 08-Jun-2010 09:14 62G ssaSource270ra300.bin 08-Jun-2010 11:11 80G ssaSource300ra330.bin 08-Jun-2010 11:29 40G ssaSource330ra360.bin 08-Jun-2010 11:42 26G There is also ssaSourceDeafultRow.bin which is just one row of data containing the default row, this got missed out in the main outgest as its RA is -ve. You almost certainly don't need it but it's used by us for cross-ref tables. The file sizes are 25858136898 ssaSource000ra030.bin 24398582844 ssaSource030ra060.bin 34075019936 ssaSource060ra090.bin 41340144220 ssaSource090ra120.bin 37362328434 ssaSource120ra150.bin 35174913752 ssaSource150ra180.bin 35502157516 ssaSource180ra210.bin 41504494724 ssaSource210ra240.bin 66761070976 ssaSource240ra270.bin 85426614622 ssaSource270ra300.bin 42697631196 ssaSource300ra330.bin 27639960122 ssaSource330ra360.bin 262 ssaSourceDeafultRow.bin Total size is 497741055502 bytes, row length is 262 so thats 1899775021 rows. No of recs in source table from SQL server is 1899775021 so at least that part's right :) We've also outgested pssaSource.bin (90947 rows) This is the source table in the Personal SSA, a one field subset of the full survey, that is also queryable via http://surveys.roe.ac.uk/ssa/sql.html indeed it's the default to encourage users to hone their queries on a small dataset first. You might like to ingest this first so that any sanity checks can be confirmed with quick queries of our system. eg a histogram of B mags select cast(round(classmagB*2.0,0) as int)/2.0,count(*) from source group by cast(round(classmagB*2.0,0) as int)/2.0 order by cast(round(classmagB*2.0,0) as int)/2.0 The datatypes of the columns are as per http://surveys.roe.ac.uk/ssa/www/ssa_browser.html http://surveys.roe.ac.uk/ssa/ssadocs/SSA_TABLE_SourceSchema.html#Source The python code below is an example of how the data can be read. import os import struct rowsize=262 binaryFile = file('ssatest.bin', 'rb') while True: rowBinary = binaryFile.read(rowsize) if not rowBinary: break print struct.unpack('<6q f 7d 5f B 12f 5B 4f 8i 8f', rowBinary) binaryFile.close() The whole detection (= Pan-STARRS Measure) table is here in RA sections http://www-wfau.roe.ac.uk/www-data/ssa-detection/ The file format is again binary and the data types of the attributes in the rows are detailed here: http://surveys.roe.ac.uk/ssa/www/SSA_TABLE_DetectionSchema.html#Detection FYI the total (source + detection) catalogue size is ~1.2 TB I believe. I hope this is useful to you Cheers ============================================================= Nigel Hambly Tel: +44-131-668-8234 Institute for Astronomy Fax: +44-131-668-8416 School of Physics and Astronomy University of Edinburgh Email: nch@roe.ac.uk Royal Observatory