- From: <Simon.Cox@csiro.au>
- Date: Mon, 5 Sep 2016 23:14:04 +0000
- To: <andrea.perego@jrc.ec.europa.eu>, <frans.knibbe@geodan.nl>, <jeremy.tandy@gmail.com>
- CC: <rob@metalinkage.com.au>, <jlieberman@tumblingwalls.com>, <chris.little@metoffice.gov.uk>, <public-sdw-wg@w3.org>
Thanks Andrea - But gosh - you have to dig deep to find it. And the sub-head 'dataset precision and accuracy' does not exactly shout 'units of measure' or 'scale' does it? It looks like a side-effect rather than a central concern. Am frankly amazed that this issue is not more central for quantitative data, which is a significant subset of general 'data on the web'. Simon -----Original Message----- From: Andrea Perego [mailto:andrea.perego@jrc.ec.europa.eu] Sent: Monday, 5 September 2016 7:15 PM To: Frans Knibbe <frans.knibbe@geodan.nl>; Jeremy Tandy <jeremy.tandy@gmail.com> Cc: Rob Atkinson <rob@metalinkage.com.au>; Joshua Lieberman <jlieberman@tumblingwalls.com>; Chris Little <chris.little@metoffice.gov.uk>; Cox, Simon (L&W, Clayton) <Simon.Cox@csiro.au>; public-sdw-wg@w3.org Subject: Re: Requirements for units of measure, accuracy and precision Just to note that UoMs are included in some of the examples of DQV, in particular in relation to spatial resolution - see: https://www.w3.org/TR/vocab-dqv/#ExpressDatasetAccuracyPrecision For more details, see my earlier messages on this topics [1,2]. Please note that the examples above are some of those added by the DWBP WG following our comments on the "representation of precision and accuracy" - see thread starting here: https://lists.w3.org/Archives/Public/public-sdw-comments/2016Mar/0001.html and related DWBP issue: https://www.w3.org/2013/dwbp/track/issues/243 Since DWBP deliverables include UoMs only for spatial resolution, I think it's up to us to address the other relevant use cases - possibly by building upon the DQV approach, if appropriate. Andrea ---- [1]https://lists.w3.org/Archives/Public/public-sdw-wg/2016Jul/0164.html [2]https://lists.w3.org/Archives/Public/public-sdw-wg/2016Jul/0253.html On 02/09/2016 17:26, Frans Knibbe wrote: > All, > thanks for replying, and a special thanks to Rob for fencing off > Jeremy's hijack attempt :-) > > An attempt to summarize and suggest steps to be taken: > > * One of us could submit a comment to the DWBPWG that we feel > indication of UoM and uncertainty in quantitative data deserves > attention in their document. Is there a volunteer? > (I would like to note that an earlier DWBP comment had me suggesting > they should encourage data providers using significant digits, which > can be seen a good way to meet the requirement for indicating > uncertainty in quantitative data. After some discussion, the DWBP > decided the subject was out of scope for data on the web (see this > message > <https://lists.w3.org/Archives/Public/public-dwbp-comments/2016Jun/0012.html>), > because it applies to the broader domain of quantitative data (that > can exist outside of the Web too). So I think next to the DWBP > already being fixed there is little chance of the comment being > acted upon.) > * There seems to be agreement that the UoM thing should be tied to CRS > definitions. Can we solve the issue by listing an indication of UoM > as one of the types of 'useful information' about a CRS in the CRS > Definition requirement > <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#CRSDefinition>? > Of course I would need to list some other typical properties of CRSs > for good measure, but that should not be a problem. > * We add a BP requirement to the UC&R that says "The use of precision > that matches uncertainty in coordinate data should be facilitated > and encouraged". Note that we already havea requirement for > describing spatial resolution in the metadata > <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#SpatialMetadata>. > > I hope we can agree on making concrete changes to the UCR document (or > not) soon. > > Greetings, > Frans > > > On 1 September 2016 at 13:41, Jeremy Tandy <jeremy.tandy@gmail.com > <mailto:jeremy.tandy@gmail.com>> wrote: > > @ Rob: > > > I think we need to step back and decide if this is about > requirements or BP > > Good point. I'll take the best practice question to a separate > thread ... > > On Wed, 31 Aug 2016 at 23:44 Rob Atkinson <rob@metalinkage.com.au > <mailto:rob@metalinkage.com.au>> wrote: > > > I think we need to step back and decide if this is about > requirements or BP, and target the elements of conversation > better to which deliverable is affected. > > IMHO there is a _requirement_ for a canonical mechanism to state > aspects of spatial properties. > I also think its a requirement for that mechanism to be common > with broader requirements for publishing metadata. Its not a > uniquely spatial concern - but spatial is typified by this being > relevant at dataset, service, feature and property levels of > granularity and the need for users to be able to discover and > interpret this information regardless of where it is best placed. > > If there is no consensus on an identifiable Best Practice - IMHO > we should note this as an unmet requirement in the BP document , > and refer to the catch-all (reuse something!) BP > > Rob > > > > > On Thu, 1 Sep 2016 at 02:27 Joshua Lieberman > <jlieberman@tumblingwalls.com > <mailto:jlieberman@tumblingwalls.com>> wrote: > > Jeremy, > > These may not be the terms we want to make a best practice > of. Precision (closeness of repeated measurements) and > accuracy (closeness to truth) are really concepts that are > impossible to compute directly, since one can’t either make > all possible repeated measurements, nor actually make true > measurements. What one can estimate and report is > uncertainty (the range within which the true value has some > likelihood of occurring if we accounted for all sources of > error) and resolution (predicted precision or fineness of > measurement). > > The spatial question in this is whether one should report > multidimensional uncertainty and resolution, using such > things as error ellipses or fuzzy geometric boundaries. It > could be a real mess to make a map with fuzzy topographic > contours, but the variation in hypsometric uncertainty > between different published maps would probably shock people > if they were made aware of it. > > Josh > >> On Aug 31, 2016, at 11:13 AM, Jeremy Tandy >> <jeremy.tandy@gmail.com <mailto:jeremy.tandy@gmail.com>> >> wrote: >> >> Ignoring the UoM thing for now (although I can see @josh's >> point about it being needed to define CRS), can folks tell >> me whether BP 5 "Describe the positional accuracy of >> spatial data" [1] is on the right track to meet >> requirements about precision and accuracy? If not, please >> can one of you fine folks make some suggests content changes? >> >> Thanks in advance. Jeremy >> >> [1]: http://w3c.github.io/sdw/bp/#desc-accuracy >> <http://w3c.github.io/sdw/bp/#desc-accuracy> >> >> On Wed, 31 Aug 2016 at 12:32 Joshua Lieberman >> <jlieberman@tumblingwalls.com >> <mailto:jlieberman@tumblingwalls.com>> wrote: >> >> Given that there are specific spatial aspects to both >> precision and uom, for example a precision ellipsoid >> is a geometry and a unit of measure such as an >> altitude is relatively to a datum such as a geoid, it >> seems reasonable to me that these can be included SDW >> BP's without worrying overmuch whether DWBP will ever >> get to these sorts of multi-dimensional and affine >> space issues. >> >> Josh >> >> Joshua Lieberman, Ph.D. >> Principal, Tumbling Walls Consultancy >> Tel/Direct: +1 617-431-6431 <tel:%2B1%20617-431-6431> >> jlieberman@tumblingwalls.com >> <mailto:jlieberman@tumblingwalls.com> >> >> On Aug 31, 2016, at 07:19, Little, Chris >> <chris.little@metoffice.gov.uk >> <mailto:chris.little@metoffice.gov.uk>> wrote: >> >>> Simon, Rob,____ >>> >>> __ __ >>> >>> I support submitting a formal public comment to DWBP, >>> AND doing something specific to spatial.____ >>> >>> __ __ >>> >>> Chris____ >>> >>> __ __ >>> >>> *From:*Rob Atkinson [mailto:rob@metalinkage.com.au] >>> *Sent:* Friday, August 26, 2016 1:42 AM >>> *To:* Simon.Cox@csiro.au <mailto:Simon.Cox@csiro.au>; >>> frans.knibbe@geodan.nl >>> <mailto:frans.knibbe@geodan.nl>; public-sdw-wg@w3.org >>> <mailto:public-sdw-wg@w3.org> >>> *Subject:* Re: Requirements for units of measure, >>> accuracy and precision____ >>> >>> __ __ >>> >>> When this was raised and we asked Phil for guidance, >>> we were basically told DWBP is pretty much fixed, if >>> we care it behooves us to address it even ifs its a >>> more general issue. I think we just need to note its >>> a more general issue, and provide what we feel is a >>> BP otherwise there is really no useful path to >>> implementation.____ >>> >>> __ __ >>> >>> Given precision is a first class concern of spatial >>> data - as spatial resolution is going to be ever >>> critical in the Internet of Things especially, we >>> need to handle this. Precision may be common to a >>> dataset due to a common methodology, specific to a >>> sensor, or time-varying - such as GPS coordinates. >>> The BP needs to address how to attach this at this >>> different levels of granularity. We need to cover >>> both measured position and gridded coverages as well, >>> and preferably not with completely different >>> approaches. Precision is not generally an attribute >>> of a CRS, unless we are going to define CRS for every >>> possible grid - and that would be a fairly major >>> departure from existing practice i suspect.____ >>> >>> __ __ >>> >>> UoM is a _similar_ case, and not inherently spatial, >>> so i agree it can be localised to CRS , TRS, but I >>> see no harm in tying it in to the same pattern as >>> spatial precision - whats needed is a BP to provide >>> metadata about values, to meet the hard requirement >>> of spatial CRS and precision, but its useful to >>> suggest this would be applicable to UoM. Certainly >>> having a completely different set of patterns for >>> attribution of uom and CRS feels like sub-optimal >>> practice. ____ >>> >>> __ __ >>> >>> Rob____ >>> >>> __ __ >>> >>> __ __ >>> >>> On Fri, 26 Aug 2016 at 10:09 <Simon.Cox@csiro.au >>> <mailto:Simon.Cox@csiro.au>> wrote:____ >>> >>> Ø The lack of information on units of measure >>> and the apparent lack of concern for proper >>> indication of uncertainty in numbers are >>> widespread in spatial data and something should >>> definitely be done about that. However, I >>> maintain that both problems are more general than >>> spatial data and are therefore out of scope for >>> the UCR document. We have tried hard to limit the >>> UCR requirements to only spatial data on the >>> web.____ >>> >>> ____ >>> >>> Frans – I agree with this assessment in >>> principle. Should we try to pass it back up to >>> the DWBP group? ____ >>> >>> ____ >>> >>> Best Practice 7 >>> https://www.w3.org/TR/dwbp/#quality >>> <https://www.w3.org/TR/dwbp/#quality> refers off >>> to the Data Quality Vocabulary, which may mean >>> they think it is dealt with, but I’m not >>> convinced. It’s one more step away from the >>> readers, at least, and none of the examples show >>> units of measure, or suggest where they would be >>> found. ‘units’ or ‘uom’ does not appear in their >>> document (in the sense that we mean, anyway). >>> ____ >>> >>> ____ >>> >>> However, this document is still in ‘Working >>> Draft’ status, although it is certainly a pretty >>> mature document by now. We could start by making >>> a comment on their public list >>> https://lists.w3.org/Archives/Public/public-dwbp-comments/ >>> <https://lists.w3.org/Archives/Public/public-dwbp-comments/> >>> to the effect that ____ >>> >>> ____ >>> >>> ‘The requirement to know the units of measure for >>> quantitative data (or another kind of ‘reference >>> system’ for other kinds of data values) is not >>> mentioned in https://www.w3.org/TR/dwbp/. Units >>> are required in order to use quantitative data, >>> so the Data on the Web Best Practices should >>> include a recommendation on the topic. This >>> affects the following benefit categories: >>> Comprehension (C), Processability (P), Reuse (R)’ >>> ____ >>> >>> ____ >>> >>> But perhaps this is a place where some more >>> formal coordination between the groups is >>> warranted? ____ >>> >>> @phila – has this issue ever been discussed in >>> the DWBP group? A quick search of the email >>> archive doesn’t turn anything up. ____ >>> >>> ____ >>> >>> --____ >>> >>> As a side comment – it is kinda remarkable how >>> this gets overlooked. I raised it in the Open >>> Knowledge forum on their Data >>> Packaging/Frictionless Data initiative. As soon >>> as I mentioned it there was acceptance that it is >>> in scope, but somehow no-one had brought it up >>> until then. ____ >>> >>> ____ >>> >>> Simon ____ >>> >>> ____ >>> >>> *From:*Frans Knibbe >>> [mailto:frans.knibbe@geodan.nl >>> <mailto:frans.knibbe@geodan.nl>] >>> *Sent:* Thursday, 25 August 2016 10:50 PM >>> *To:* SDW WG Public List <public-sdw-wg@w3.org >>> <mailto:public-sdw-wg@w3.org>>____ >>> >>> >>> *Subject:* Requirements for units of measure, >>> accuracy and precision____ >>> >>> ____ >>> >>> Hello,____ >>> >>> ____ >>> >>> In messages >>> https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0172.html >>> <https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0172.html> >>> and ____ >>> >>> https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0109.html >>> <https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0109.html> >>> the possibility of adding new requirements to the >>> UC&R doc was brought forward. Those should be >>> requirements that____ >>> >>> ____ >>> >>> A) the units of measure (UoM) in spatial data >>> should be made clear____ >>> >>> B) the precision of spatial data should be made >>> clear____ >>> >>> ____ >>> >>> From the looks of it, those requirements would at >>> least be requirements for the BP >>> deliverable.____ >>> >>> ____ >>> >>> The topic was also discussed in the last SSN >>> teleconference >>> <https://www.w3.org/2016/08/23-sdwssn-minutes>. I >>> thought it would be good to create a separate >>> thread for these related issues.____ >>> >>> ____ >>> >>> I will first repeat my initial response: The >>> requirements are very good requirements. The lack >>> of information on units of measure and the >>> apparent lack of concern for proper indication of >>> uncertainty in numbers are widespread in spatial >>> data and something should definitely be done >>> about that. However, I maintain that both >>> problems are more general than spatial data and >>> are therefore out of scope for the UCR document. >>> We have tried hard to limit the UCR requirements >>> to only spatial data on the web. This constraint >>> is specifically mentioned in the section on >>> methodology >>> <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#Methodology>. >>> If we were to neglect this constraint, then the >>> amount of requirements could run out of hand >>> quickly, the decisions on which requirements to >>> include or not would become very arbitrary and >>> the deliverable teams that are tasked with >>> meeting requirements would inevitably find out >>> that they are not in a position to meet the >>> requirements because they are not in scope for >>> their work. Of course the deliverable teams will >>> work with additional requirements next to the >>> ones mentioned in the UCR document. Those >>> additional requirements will be based on general >>> best practices. I think the UoM and precision >>> requirements fall in that class.____ >>> >>> ____ >>> >>> That said, perhaps there is a way to shape the >>> requirements in such a way that they can be >>> included in the UCR document, without violating >>> the spatial scope constraint too harshly. After >>> all, it has been done for other requirements too, >>> to be honest.____ >>> >>> ____ >>> >>> Let's start with the UoM requirement (A). I >>> assume that is about the UoM of coordinate data >>> only. I think this is already implicitly covered >>> by the CRS requirements Linking geometry to CRS >>> <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#LinkingCRS>, >>> Determinable CRS >>> <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#DeterminableCRS> >>> and CRS definition >>> <http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#CRSDefinition>: >>> If those requirements are met, it should always >>> be possible to know the UoM of coordinates, >>> because the UoM will be part of the CRS >>> definition. Perhaps we should be explicit in >>> mentioning that a CRS definition should include >>> an indication of UoM?____ >>> >>> ____ >>> >>> As for the requirement B, if we change the >>> wording a bit the requirement could be made >>> applicable to spatial data only and therefore be >>> in scope:____ >>> >>> ____ >>> >>> B2) The use of precision that matches uncertainty >>> in coordinate data should be facilitated and >>> encouraged____ >>> >>> ____ >>> >>> With this kind of wording I think the BP editors >>> have a fighting chance of meeting the >>> requirement.____ >>> >>> ____ >>> >>> Please share your thoughts...____ >>> >>> ____ >>> >>> Regards,____ >>> >>> Frans____ >>> > > -- Andrea Perego, Ph.D. Scientific / Technical Project Officer European Commission DG JRC Directorate B - Growth and Innovation Unit B6 - Digital Economy Via E. Fermi, 2749 - TP 262 21027 Ispra VA, Italy https://ec.europa.eu/jrc/ ---- The views expressed are purely those of the writer and may not in any circumstances be regarded as stating an official position of the European Commission.
Received on Monday, 5 September 2016 23:14:57 UTC