Re: CRS best practices: Google Geocoding API [SEC=UNCLASSIFIED]

Hi Jon,

Thank you for your comments on this. They are most helpful.

If anyone knows of any studies that can help us to better quantify the amount of spatial data available on the web, together with SRS info, I think that it would be enlighting for the document's audience.

I do not equate published on the web with only that data published by Google and its peers.

I am very aware of the many petabytes of spatial data available via National Met Services and various other government organisations that operate within Spatial Data Infrastructures.

It would be good if this document was equally as relevant for this 'traditional' source of spatial data and also for SDIs to evolve over time to support a more semantic approach.

Bruce
________________________________
From: Jon Blower <j.d.blower@reading.ac.uk>
Sent: Monday, 27 March 2017 11:20:55 PM
To: Ed Parsons; Simon.Cox@csiro.au; Bruce Bannerman; jeremy.tandy@gmail.com; bcochrane@linz.govt.nz; public-sdw-wg@w3.org
Subject: Re: CRS best practices: Google Geocoding API [SEC=UNCLASSIFIED]

Ed,


Ø  our aim is to make aware and assist a primary audience who are not  Geo experts and have not added any explicit geospatial information to their data until now.

That’s certainly one aim of the group. But it’s clear from the constant re-emergence of this debate (and others like it) that others in the group have other expectations, connected with making the Web a better place for geo professionals too. So we need to either accommodate those expectations or rule them out of scope.

In case it wasn’t clear from my mail, I have no problem with talking about, and even recommending, WGS84 (sic) as long as the context is clear. But our outputs will have a varied audience (who will dip in and out of bits of our documents) and where possible we must avoid sweeping statements that cause concern in important sections of that audience. That’s all I’m saying, and we’re probably in violent agreement about most of it.


Ø  I would argue that much of the Geo expert community data published in CRS other than WGS84 is largely invisible on the web not accessible behind opaque service interfaces

That’s true in some cases, but there’s a mass of spatial data freely available as simple HTTP downloads. (I think what we’re exposing here is that we don’t have a good definition of what it means for data to be “published on the web”.)


Ø  so the claim that the vast majority of spatial data on the web is WGS84 holds true..

Whether this statement is true or not (I don’t think it is, but let’s disagree on that), there’s simply no need to make that strong a statement in our documents. It’s probably more accurate (and helpful) to say that publishing in WGS84 will help people to integrate data with mass-market web mapping technologies – why not stick with a statement along those lines?

(By the way, WGS84 might be the de-facto “web standard” for point data, but for raster data the “web standard” would be Web Mercator – so that’s another reason not to be too strong in statements about WGS84.)


Ø  In summary I would humbly suggest that this discussion is once again an example of the geospatial industry looking in on itself rather than out...

That is an unhelpful assertion, and not true in my opinion. It only serves to irritate people who are trying to make our outputs better. There are plenty of people in the “geospatial industry” (whatever that is) looking out (isn’t membership of this group evidence of that?) and their concerns are valid too.

Jon


Jon Blower | CTO, Institute for Environmental Analytics

Follow the IEA on Twitter @env_analytics <https://twitter.com/env_analytics>  and on LinkedIn The Institute for Environmental Analytics (IEA)<https://www.linkedin.com/company/the-institute-for-environmental-analytics?trk=biz-companies-cymhttps://www.linkedin.com/company/the-institute-for-environmental-analytics?trk=biz-companies-cym>

Philip Lyle Building, University of Reading, Whiteknights Campus, Reading RG6 6BX
T: +44 (0)118 378 5213 M: +44 (0)7919 112687
E: j.blower@the-iea.org<mailto:j.blower@the-iea.org> W: www.the-iea.org<http://www.the-iea.org/>

From: Ed Parsons <eparsons@google.com>
Date: Monday, 27 March 2017 11:54
To: "Simon.Cox@csiro.au" <Simon.Cox@csiro.au>, Jon Blower <sgs02jdb@reading.ac.uk>, "bruce.bannerman@bom.gov.au" <bruce.bannerman@bom.gov.au>, Jeremy Tandy <jeremy.tandy@gmail.com>, "bcochrane@linz.govt.nz" <bcochrane@linz.govt.nz>, "public-sdw-wg@w3.org" <public-sdw-wg@w3.org>
Subject: Re: CRS best practices: Google Geocoding API [SEC=UNCLASSIFIED]

No surprise I feel strongly about this, I restate once again that our aim is to make aware and assist a primary audience who are not  Geo experts and have not added any explicit geospatial information to their data until now. Under what circumstances would a CRS other than WGS84 (sic) be the best choice ?

We have of course another audience who are the Geo experts perhaps frustrated that the data they have shared so far has not had the impact in terms of usage that they might have hoped, to this community we offer alternatives that may make their content more accessible to the mainstream web - In this case publishing using alternative CRS such as WGS84 would amongst other things be beneficial.

I would argue that much of the Geo expert community data published in CRS other than WGS84 is largely invisible on the web not accessible behind opaque service interfaces, so the claim that the vast majority of spatial data on the web is WGS84 holds true..

In summary I would humbly suggest that this discussion is once again an example of the geospatial industry looking in on itself rather than out...

Ed




On Mon, 27 Mar 2017 at 11:10 <Simon.Cox@csiro.au> wrote:
What Jon says.
+1

From: Jon Blower [mailto:j.d.blower@reading.ac.uk<mailto:j.d.blower@reading.ac.uk>]
Sent: Monday, 27 March, 2017 20:03
To: Ed Parsons <eparsons@google.com<mailto:eparsons@google.com>>; Bruce Bannerman <bruce.bannerman@bom.gov.au<mailto:bruce.bannerman@bom.gov.au>>; Jeremy Tandy <jeremy.tandy@gmail.com<mailto:jeremy.tandy@gmail.com>>; Byron Cochrane <bcochrane@linz.govt.nz<mailto:bcochrane@linz.govt.nz>>; SDW WG Public List <public-sdw-wg@w3.org<mailto:public-sdw-wg@w3.org>>

Subject: Re: CRS best practices: Google Geocoding API [SEC=UNCLASSIFIED]

Hi all,

I think this debate shows we need to be careful about we phrase our key recommendations. I would agree with Bruce that the statement:

“(WGS 84) Coordinate Reference System is used in almost all cases where spatial data is published on the Web”

is overly strong and invites controversy unnecessarily. (How do we measure “almost all cases” anyway – by data storage? By data transferred? By number of “hits”? By number of datasets? They would all likely give different answers.) There are huge volumes of satellite imagery freely available on the web that are not published in WGS84, although equally they are not aimed at a typical “Google Maps” user. Then there are huge archives of weather, climate and astronomical data, all of which are spatial, and are not usually WGS84 either. Google Maps (and Bing, etc) have huge numbers of users/hits, but that doesn’t mean that they represent “almost all cases” of data publication.

I think we’ve had this debate before, but I’d be more comfortable with statements like:

“WGS84 is the most common CRS in mass-market spatial applications such as X, Y and Z. Publication of data in this CRS will make it easier to reach a wide audience of non-specialists, but publishers must be aware of the implications of doing so [for applications in which high accuracy is required].”

It would also be helpful to point to some good references where non-specialist can learn about the issues surrounding CRSs (Perhaps we are doing that already?)

We must be very careful to distinguish cases in which we are giving advice with a view to widening the audience for spatial data, and cases in which we are giving advice to spatial professionals who want to use the Web to share data with each other.

All the best,
Jon



From: Ed Parsons <eparsons@google.com<mailto:eparsons@google.com>>
Date: Monday, 27 March 2017 08:56
To: Bruce Bannerman <bruce.bannerman@bom.gov.au<mailto:bruce.bannerman@bom.gov.au>>, Jeremy Tandy <jeremy.tandy@gmail.com<mailto:jeremy.tandy@gmail.com>>, Byron Cochrane <bcochrane@linz.govt.nz<mailto:bcochrane@linz.govt.nz>>, SDW WG Public List <public-sdw-wg@w3.org<mailto:public-sdw-wg@w3.org>>
Subject: Re: CRS best practices: Google Geocoding API [SEC=UNCLASSIFIED]
Resent-From: <public-sdw-wg@w3.org<mailto:public-sdw-wg@w3.org>>
Resent-Date: Monday, 27 March 2017 08:57

Hi Bruce,

I think we are just going to disagree re the context of the document, to use your medical analogy we are aiming to produce the equivalent of your local pharmacy, not the university teaching hospital.

My point re the Google products was not a flippant one, you could of course include BIng, OpenStreetMap, Facebook, Twitter, Flickr, Whatsap, Apple Maps, iMessage -the users of which all produce and consume spatial data represented using WGS84.  We are now clearly in the age where the majority of spatial data on the web is not produced by experts alone.

Of course there are datasets and applications that use local projected CRS of the web, but compared to these consumer services I would imagine they represent a very small proportion... Of course I may be completely wrong and I'd be interested in your counter argument of this point ?

Ed



On Sun, 26 Mar 2017 at 23:08 Bruce Bannerman <bruce.bannerman@bom.gov.au<mailto:bruce.bannerman@bom.gov.au>> wrote:
Hi Ed,

Thanks for your comments.

I do understand the context of this document.

There are many sites with similar goals of making complex domains *appear* simple, e.g. self help medical diagnosis, and do it yourself building renovation.

Neither approach will help the user if they mis-diagnose a terminal illness, or remove a load bearing wall in ignorance.

There is a time and a place for professional advice and we should not trivialise it.

By all means, make the document as easy to use as possible, but do not trivialise these types of spatial issues and add the appropriate warning to seek professional advice.


Regarding your second point regarding Google Earth/Maps:

  *   These numbers are truely impressive. Well done to Google.
  *   However, there is a vast difference between the number of users of a product, and the amount of spatial data on the web. I stand by my comments.

Kind regards,

Bruce


From: Ed Parsons <eparsons@google.com<mailto:eparsons@google.com>>
Date: Friday, 24 March 2017 at 19:08
To: Bruce Bannerman <bruce.bannerman@bom.gov.au<mailto:bruce.bannerman@bom.gov.au>>, Jeremy Tandy <jeremy.tandy@gmail.com<mailto:jeremy.tandy@gmail.com>>, Byron Cochrane <bcochrane@linz.govt.nz<mailto:bcochrane@linz.govt.nz>>, SDW WG Public List <public-sdw-wg@w3.org<mailto:public-sdw-wg@w3.org>>
Subject: Re: CRS best practices: Google Geocoding API [SEC=UNCLASSIFIED]

Hello Bruce,

Thanks for your comments, I will leave it to Jeremy to directly respond to your points, however I would restate the context of this document... It is aimed at non-specialist users who are looking to simply publish data about locations on the web, we want to avoid the exact situation you suggest that expert advice is needed each time to you need to  publish spatial data, sounds like we need to further simplify and clarify...

One billion users of Google Maps and another billion users of Google Earth would I suggest indicate that  WGS84 is the most used CRS ?

Ed


On Fri, 24 Mar 2017 at 06:10 Bruce Bannerman <bruce.bannerman@bom.gov.au<mailto:bruce.bannerman@bom.gov.au>> wrote:
Hi Jeremy,

I was going to let this pass by to the keeper, having raised it many times before (and not wanting to join your BBQ  ;-)  ).

I am slowly working through the latest Best Practice as I get time. The BP has come a long way and there is certainly some good work it it.


However, I’m very uncomfortable with aspects of the current approach in the Best Practice regarding CRS and SRS in particular.

Some comments:


  *   First para: "you can express a location to within a few metres”. This statement ignores issues of precision and accuracy [1]. You can be as precise as you like, but that does not make your data accurate. In relation to the quoted text from the first paragraph, a precise coordinate that purports to be within a few meters due to it’s coordinate precision may in fact be inaccurate and perhaps kilometres away from that location.

  *   Second para: There is more to a SRS that a way of defining a set of coordinates. We need to cover the mathematical representation of the area covered by the SRS (the ellipsoid), the choice of projection to represent ellipsoidal data within a two dimensional space, together with the compromises that this brings to the data in representation of area, shape, distance, bearing, scale, direction, etc.

  *   I note the statement “(WGS 84) Coordinate Reference System is used in almost all cases where spatial data is published on the Web”. I find this statement dangerous and very difficult to believe!  Can the author of the statement please cite the relevant studies that support this claim?

  *   There may be serious implications for either using an incorrect SRS, or assuming an incorrect SRS.   Spatial data is typically used to underpin a wide range of decision making activities. Some of these decisions may have significant consequences to people, property and life. As an example of some of the potential impact of incorrectly combining data after ignoring SRS issues, please see some example diagrams referred to below:

     *   [2] image to illustrate differences in Australian SRS (ellipsoids and datums) - GDA94 and AGD66
     *   [3] image to illustrate differences in SRS (projections) - Lambert Conformal Conic and UTM MGA Zone 50
     *   [4] image to illustrate differences in SRS (projections) - Lambert Conformal Conic and VicGrid94
     *   [5] image to illustrate differences in SRS (ellipsoids, datum and projections) - Lambert Conformal Conic and World Mercator

  *   As you will see from the diagrams, if someone is combining data from different SRS, without accounting for the differences between those SRS, then the results of an analysis in support of a decision making process, together with that decision may well be flawed with potential consequences.

  *   Spatial professionals undertake substantial tertiary study and on the job work experience in order to understand their profession. We cannot expect that any lay person will be able to just pick it up e.g. By .reading this BP.

  *   I suggest that a valuable piece of advice for the BP is to advise readers to understand that there may be a limit to their understanding of specific issues and to seek professional advice where appropriate. SRS may be one such issue.

  *   I may have more comments on CRS and the BP, provided that I can get suitable time to continue my review.

Keep up the good work on this.

Kind regards,

Bruce


[1] http://2.bp.blogspot.com/-s6ypcjPzxF0/UyaU9mOg6-I/AAAAAAAAAxs/MEU1FcozbZ0/s1600/accuracy+vs+precision.jpg

[2] https://wiswiki.wmo.int/tiki-download_file.php?fileId=3465

[3] https://wiswiki.wmo.int/tiki-download_file.php?fileId=3467

[4] https://wiswiki.wmo.int/tiki-download_file.php?fileId=3468

[5] https://wiswiki.wmo.int/tiki-download_file.php?fileId=3466




From: Jeremy Tandy <jeremy.tandy@gmail.com<mailto:jeremy.tandy@gmail.com>>
Date: Wednesday, 8 March 2017 at 04:40
To: Byron Cochrane <bcochrane@linz.govt.nz<mailto:bcochrane@linz.govt.nz>>, SDW WG Public List <public-sdw-wg@w3.org<mailto:public-sdw-wg@w3.org>>
Subject: Re: CRS best practices: Google Geocoding API
Resent-From: <public-sdw-wg@w3.org<mailto:public-sdw-wg@w3.org>>
Resent-Date: Wednesday, 8 March 2017 at 04:41

Hi Byron. Thanks for this feedback. I completely agree :)

Hopefully BP17 [1] is clear on this now - see the note at the top of the "Possible Approach" section.

Jeremy

[1]: http://w3c.github.io/sdw/bp/#bp-crs
On Tue, 7 Mar 2017 at 00:31 Byron Cochrane <bcochrane@linz.govt.nz<mailto:bcochrane@linz.govt.nz>> wrote:
A BBQ! I got to get in on this . . .

First off, I feel a bit uncomfortable to say “then [it's safe to] assume” .  Because it often isn’t as I think is Bart’s point.  Something more along the line of “best guess is” is better.  Then emphasize that best practice is that CRS with lat/long (or coordinate) order and units of measurement needs to be clearly stated.

I want to second everything Josh stated.  I would hope that the point of Best Practices is to raise the bar (within reasonable  limits) and not stamp current common practice as best practice.  The fact that most current web developers only care about lat/long does not mean this is right and best.  It is in IMHO the main issue, in this BP, that needs to be addressed.  Part of the problem is that most of the general public, including developers, have popularly come to view the 2D Mercator view of the world as true when in fact it is a hugely distorted one.  Some basic knowledge about CRS is advantageous to data providers and web developers in order to encourage better use of spatial data.

This is not at all to say that only geo professionals are qualified to use spatial data.  Basic knowledge is within reach of anyone.  I hope we provide some in the SDWBP.  If we also provide pointers as to where to learn more, all the better.  This leads to more capable developers being able to create more useful and user friendly products.  There are always tradeoffs when selecting a CRS.  That is the basic message I wish to convey to the more general developers, even if we only do that in a cursory way.

Cheers,
Byron


From: Jeremy Tandy [mailto:jeremy.tandy@gmail.com<mailto:jeremy.tandy@gmail.com>]
Sent: Saturday, 4 March 2017 6:19 a.m.
To: Bart van Leeuwen
Cc: Ed Parsons; SDW WG Public List

Subject: Re: CRS best practices: Google Geocoding API

To be fair, the Google Geocoding API (which was where I started) uses latitude and longitude - so at least it's obvious that the coordinate position is some form angular measurement for anywhere on the Earth.


Jeremy
On Fri, 3 Mar 2017 at 17:14, Bart van Leeuwen <bart_van_leeuwen@netage.nl<mailto:bart_van_leeuwen@netage.nl>> wrote:
The problem with this definition is that the term "world view" is rather ambiguous.
I know a lot of Dutch public servants in geospatial related fields who's world view is no bigger then the 300x300km dutch CRS.
They assume the RD CRS as their "world view" making all wgs84 ( especially the negative numbers ) utterly confusing.

As much as I understand that the 'world view' of web developers is WGS84 assuming it for our audience might actually turn up the heat :)

My 2 cents.

Met Vriendelijke Groet / With Kind Regards
Bart van Leeuwen

Error! Filename not specified.
twitter: @semanticfire
tel. +31(0)6-53182997<tel:+31%206%2053182997>
Netage B.V.
http://netage.nl<http://netage.nl/>
Esdoornstraat 3
3461ER Linschoten
The Netherlands
Error! Filename not specified.



From:        Jeremy Tandy <jeremy.tandy@gmail.com<mailto:jeremy.tandy@gmail.com>>
To:        Ed Parsons <eparsons@google.com<mailto:eparsons@google.com>>, SDW WG Public List <public-sdw-wg@w3.org<mailto:public-sdw-wg@w3.org>>
Date:        03-03-2017 17:22
Subject:        Re: CRS best practices: Google Geocoding API
________________________________



Fair enough.

I suppose that if we write stuff in the BP document like this, we're documenting what is actually happening.

There's a risk that we end up encouraging people to be lazy and not bother to think about CRS. But then, if they're in the <rest of the world view "that I just need to use Lat & Long - Period :-)">TM then they will probably not even have considered that this is an issue in the first place. At least this advice is consistent with geospatial data collected from the vast majority of [consumer] devices on the planet - because they're using GPS.

Jeremy

On Fri, 3 Mar 2017 at 16:16 Ed Parsons <eparsons@google.com<mailto:eparsons@google.com>> wrote:
I think the first part is OK, the vertical datum part is less common and as a result it's more difficult to make a similar assumption.

Ed


On Fri, 3 Mar 2017 at 16:11 Jeremy Tandy <jeremy.tandy@gmail.com<mailto:jeremy.tandy@gmail.com>> wrote:
Hmmm.

schema.org<http://schema.org/> documents go to the trouble of saying "WGS 84" (although they don't describe the units either).

So (as much as most of the Geo-establishment will flame me for it) should we be saying:

"If neither your data nor the specification to which your data conforms to defines the coordinate reference system used, then [it's safe to] assume that the data with coordinate pairs uses longitude and latitude, defined in decimal degrees, and data with coordinate positions that have three values is longitude, latitude and elevation, defined in decimal degrees, decimal degrees and meters above sea-level. In both cases, the WGS 84 [geodetic] datum is assumed."

Let the barbecue begin.

Jeremy


On Fri, 3 Mar 2017 at 16:02 Ed Parsons <eparsons@google.com<mailto:eparsons@google.com>> wrote:
I think you are experiencing the rest of the world view "that I just need to use Lat & Long - Period :-)"

The use of WGS84 is documented here https://developers.google.com/maps/documentation/javascript/maptypes if you go looking for it, must I would argue that most mainstream web developers don't need to know..

btw this is also quite a nice explanation of tile based spatial indices ;-)

Ed



On Fri, 3 Mar 2017 at 15:14 Jeremy Tandy <jeremy.tandy@gmail.com<mailto:jeremy.tandy@gmail.com>> wrote:
Hi Ed- in the introductory material you wrote about CRS you make a reference to the Google Geocoding API [1], in that its responses explicitly state Lat and Long rather than a coordinate pair of ambiguous order.

Lat and Long are, by definition, angular measurements. OK - got that.

But parsing through the API documentation, I can't see any reference to the units or datum which is used.

Being a human, I'm prepared to guess that these are decimal degrees (because they look like floating point numbers). Easy for machines to figure that out too.

As a human, I'm also prepared to guess that the API uses the WGS84. But that is a tricky leap for machines to work out.

Does the API documentation say "WGS84" anywhere? If so, can you point me to it so I can refer to this explicitly? And if not, can you either justify why it doesn't matter, or get your colleagues to update the documentation (and then send me a link!).

(I think that we've all agreed that it's dangerous to _assume_ a CRS :-) )

Thanks, Jeremy

[1]: https://developers.google.com/maps/documentation/geocoding/intro
--

Ed Parsons FRGS
Geospatial Technologist, Google

Google Voice +44 (0)20 7881 4501<tel:%2B44%20%280%2920%207881%204501>
www.edparsons.com<http://www.edparsons.com/> @edparsons

--

Ed Parsons FRGS
Geospatial Technologist, Google

Google Voice +44 (0)20 7881 4501<tel:%2B44%20%280%2920%207881%204501>
www.edparsons.com<http://www.edparsons.com/> @edparsons

________________________________
This message contains information, which may be in confidence and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463<tel:0800%20665463> or info@linz.govt.nz<mailto:info@linz.govt.nz>) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.
--

Ed Parsons FRGS
Geospatial Technologist, Google

+44 7825 382263<tel:07825%20382263> @edparsons
www.edparsons.com<http://www.edparsons.com/>
--

Ed Parsons FRGS
Geospatial Technologist, Google

+44 7825 382263<tel:07825%20382263> @edparsons
www.edparsons.com<http://www.edparsons.com/>
--

Ed Parsons FRGS
Geospatial Technologist, Google

+44 7825 382263 @edparsons
www.edparsons.com<http://www.edparsons.com/>