TOPCAT and how to use it for Gaia

TOPCAT and how to use it for Gaia Mark Taylor (University of Bristol) Gaia DR1 Workshop ESAC, Madrid 2 November 2016 $Id: tcgaia_esac.tex,v 1.1 2016/10/14 15:23:07 mbt Exp $ Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 1/35

Outline TOPCAT very short intro Accessing Gaia-DR1 Data from TOPCAT GACS web interface Cone Search CDS XMatch client TOPCAT TAP window TGAS FITS file download (others) Examples Pleiades distance determination Match with local catalogue TGAS-Hipparcos colour-magnitude diagram All-sky density maps Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 2/35

TOPCAT TOPCAT = Tool for OPerations on Catalogues And Tables Capabilities: Does stuff with tables Talks to the Virtual Observatory Help is available: Comprehensive HTML / PDF user manual Help for Window Email support: button on every window on list: topcat-user@bristol.ac.uk in person: m.b.taylor@bristol.ac.uk Acknowledgement: 2005ASPC..347...29T http:/www.starlink.ac.uk/topcat/ Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 3/35

Data Access: GACS GACS Web interface Details Suitability: Usage: http://gea.esac.esa.int/archive large or complex queries has all Gaia DR tables persistent uploads result table management precalculated neighbour tables large compute resources... Download result table to local disk and Load into TOPCAT Or use SAMP button to transmit directly to TOPCAT (requires GACS access using HTTP not HTTPS) Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 4/35

Data Access: Cone Search Use TOPCAT s Cone Search window Details: Suitability: Usage: Retrieves all Gaia/TGAS sources within a given radius of a given sky position If you want all Gaia sources in given sky region Only selects on position (but can restrict further in TOPCAT) Limit: 10 million rows VO Cone Search menu item Keywords: gaia Find Services Select service ARI-Gaia Fill in Object Name or RA/Dec and Radius, and hit OK Verbosity selector controls which columns are included Documentation: http://www.starlink.ac.uk/topcat/sun253/conesearchdialog.html Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 5/35

Data Access: CDS X-Match Use TOPCAT s CDS Upload X-Match window Details: Suitability: Usage: Fast, scalable service for crossmatching VizieR tables TOPCAT can use it for crossmatch of loaded table against Gaia Nearest/all nearby Gaia sources for each row of local table Scales to millions (or more) of rows Not all columns returned Only selects on position (but can restrict results further in TOPCAT) Load local table into TOPCAT (or get it from VizieR, or TAP, or...) VO CDS Upload X-Match menu item or toolbar button Select VizieR Table ID/Alias: GAIA DR1 Docs: http://www.starlink.ac.uk/topcat/sun253/cdsuploadmatchwindow.html Ack: This research made use of the cross-match service provided by CDS, Strasbourg. Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 6/35

Data Access: TOPCAT TAP Window Use TOPCAT s TAP (Table Access Protocol) client window Details: Suitability: Usage: Talks directly to GACS/ARI TAP services, and other non-gaia TAP servers Similar functionality, different GUI to GACS web interface Suitable for ADQL queries Better integrated into TOPCAT than GACS web interface results load directly into TOPCAT can upload tables from TOPCAT Table/column metadata browsing different ADQL editing features (multi-tab, validation, undo/redo,...) Limits: 10 5 rows (GACS), 10 7 rows (ARI) VO Table Access Protocol (TAP) Query menu item or toolbar button Select Service Keywords: gaia ARI-Gaia or [ESA] GAIA Docs: http://www.starlink.ac.uk/topcat/sun253/taptableloaddialog.html Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 7/35

Data Access: TGAS Download Use monolithic TGAS FITS file Details: Suitability: Download to local disk, e.g.: http://andromeda.star.bristol.ac.uk/data/tgas_source.fits Size: 2 057 050 rows 60 columns, 660 Mb Good size for TOPCAT (unlike gaia source, 325 Gb) If you want all TGAS data, this may be the easiest way to use it in TOPCAT Usage Load into TOPCAT using Load window or from command line (instantaneous) Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 8/35

Data Access: Others Other ways to get Gaia DR1 data into TOPCAT: ARI-Gaia web page (http://gaia.ari.uni-heidelberg.de/) nice GUI precalculated statistics, single source search + TAP, Cone Search, more to come CDS X-Match web page (http://cdsxmatch.u-strasbg.fr/xmatch) allows VizieR/Gaia crossmatch as well as local/gaia crossmatch VizieR (http://vizier.u-strasbg.fr) Standard CDS interface + API (including access from TOPCAT) Gaia@AIP (https://gaia.aip.de/) And more... Scriptable multi-threaded MySQL queries (beta) Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 9/35

Example A: Pleiades distance Determine parallax of Pleiades (following Gaia-DR1 paper) Cone search TGAS within 5 of Pleiades Plot pmra vs pmdec Identify comoving sources, create subset graphically Plot parallax histogram of comoving subset Restrict subset further to exclude parallax outliers Use Statistics window to determine cluster µ ϖ, σ ϖ Visualise cluster and non-cluster sources: in 3d space, showing proper motions Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 10/35

A.1: TGAS Cone Search Want to query TGAS sources within 5 of Pleiades Use TOPCAT Cone Search window: VO Cone Search menu item Keywords: tgas Object Name: pleiades + Resolve Radius: 5... there are other ways to do it Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 11/35

A.2: Proper motion plot Plot sources in proper motion space: Graphics Plane Plot menu item or toolbar button X: pmra Y: pmdec Note overdensity far from (0,0) Graphically select this comoving cluster as new Subset button, drag mouse, again New Subset Name: comoving + Add Subset Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 12/35

A.3: Parallax histogram Plot parallax histogram of comoving subset Graphics Histogram plot menu item or toolbar button X: parallax Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 13/35

A.4: Exclude proper motion outliers Restrict comoving subset further to exclude parallax outliers Views Row Subsets menu item or toolbar button toolbar button to create new algebraic subset Subset Name: cluster Expression: comoving && parallax>6 && parallax<9 Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 14/35

A.5: Cluster proper motion statistics Views Column Statistics menu item or toolbar button Subset for Calculations: cluster See Mean and Stdev columns pleiades parallax 7.5 ± 0.4 mas careful with priors if converting to distance... but parallax error/parallax is quite high for all cluster members Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 15/35

A.6: 3d cluster positions Graphics Sphere Plot menu item or toolbar button Subsets tab: select cluster subset only Lon: ra Lat: dec Radius: 1./parallax Cluster positions are visible in 3d space You can turn on All/comoving subsets too Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 16/35

A.7: Visualise proper motions Graphics Sky Plot menu item or toolbar button Add new SkyVector form in Form tab Delta Longitude: pmra Delta Latitude: pmdec See the little arrows showing proper motion Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 17/35

Example B: Crossmatch with local catalogue Match with Gouliermis et al. 2006 (NGC 346 HST/ACS photometry) Find and download J/ApJS/166/549/table2 from VizieR load window... or any other way to load a positional catalogue in topcat Use CDS XMatch window to match with GAIA DR1 (Find mode: All) Sky plot of Gouliermis and Gouliermis/Gaia matched pairs (many counterparts) Plot histogram of angdist (spike near 0.3) Plot _RAJ2000-ra vs. _DEJ2000-dec Identify modal ( α, δ) Trace new subset to identify probable matches Now you have: NGC 346 proper motion estimate G magnitudes for Gouliermis sub-sample ( 1 800/99 000 sources) Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 18/35

B.1: Acquire NGC 346 catalogue Load catalogue from VizieR: VO VizieR Catalogue Service menu item All Rows check box Maximum Row Count: 100 000 Locate and load Gouliermis et al. 2006 (J/ApJS/166/549) Or grab it from CDS VizieR web page Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 19/35

B.2: Crossmatch with Gaia VO CDS Upload X-Match menu item or toolbar button Vizier Table ID: GAIA DR1 Radius: 1 arcsec Find mode: All Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 20/35

B.3: Plot crossmatch results Graphics Sky Plot menu item or toolbar button Plot NGC 364 points Plot Gaia Gouliermis associations ([Goul] RAJ2000, DEJ2000, [Gaia] ra, dec)... too many Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 21/35

B.4: Plot crossmatch offsets Graphics Histogram plot menu item or toolbar button X: angdist (Gaia Gouliermis association distance) (some) true associations near 0.35 arcsec Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 22/35

B.5: Identify true matches Plot matches in xmatch offset space: Graphics Plane Plot menu item or toolbar button X: RAJ2000 - ra Y: DEJ2000 - dec Obvious overdensity corresponds to true offset Select new subset true match graphically Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 23/35

B.6: Visualise true matches Return to sky plot Subsets tab: select true match only Common association vector, = ( α, δ) in true match subset, is displacement between Gouliermis & Gaia observations (1992? 2015.0): cos δ α +210 ± 20 mas δ 284 ± 15 mas so proper motion: µ α +9.1 mas.yr 1 µ δ 12 mas.yr 1 Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 24/35

B.7: Combine HST and Gaia photometry Joined table now has Gaia G-band photometry alongside HST V/I-band photometry Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 25/35

Example C: TGAS-Hipparcos CMD Use ARI Example query to generate fig 3 of Gaia DR1 paper Point TOPCAT TAP client at ARI-Gaia service Use Examples Service-Provided Gaia DR1 Color and Magnitude menu Plot Hipparcos B V vs absolute Gaia G Adjust TAP query to get more columns Colour-code points in CMD by parallax? galactic latitude? Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 26/35

C.1: Locate Gaia TAP service VO Table Access Protocol (TAP) Query menu item or toolbar button Select Service tab, Keywords: gaia Select ARI-Gaia Hit Use Service button/tab Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 27/35

C.2: Execute Example CMD query Use Examples menu near bottom of window Select Service-provided Gaia DR1 - Color and magnitude See ADQL is entered in window Hit Run Query to execute it Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 28/35

C.3: Plot CMD Graphics Plane Plot menu item or toolbar button X: b v Y: g mag abs gaia Axes control, use Y Flip checkbox... it s almost too easy. Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 29/35

C.4: Customise ADQL and re-execute Back to TAP window, customise the query Request also gaia.parallax, gaia.l, gaia.b, more? Query again Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 30/35

C.5: Use customised query results Adjust the plot Use Weighted or Aux mode to colour points Find anything interesting? (maybe not) Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 31/35

Example D: All-sky density Statistical/density maps of all TGAS/gaia sources Load tgas source.fits Sky marker plot: tweak projection, view sky-system etc Source densities using shading modes: flat, auto, density Statistical plots of parallax_error, phot_g_n_obs, hypot(pmra_error,pmde_error),... using shading modes aux, weighted Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 32/35

D.1: Display weighted TGAS all-sky plots Weighted phot g n obs in ecliptic coords Weighted parallax error in galactic coords Plot interactive 2Mrow density map in a few seconds Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 33/35

STILTS Most of TOPCAT s capabilities can be scripted STILTS: from command line (e.g. un*x shell) JyStilts: from Jython (python interface, but not CPython) Details Not covered in this talk! But some examples available: See http://www.star.bristol.ac.uk/~mbt/gaia/tutorial.html Full documentation and examples in http://www.starlink.ac.uk/stilts/ Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 34/35

Summary Lots of ways to get Gaia data into TOPCAT Different ones most suitable for different situations Lots of things you can do with it once it s there Play around with plots Use documentation Support on mailing list, email me,... Scriptable access/manipulation available using STILTS or JyStilts Materials: http://www.star.bristol.ac.uk/~mbt/gaia/tutorial.html Mark Taylor, TOPCAT and how to use it for Gaia, Gaia DR1 Workshop, ESAC, Madrid, 2 November 2016 35/35