W3C home > Mailing lists > Public > public-sdw-wg@w3.org > September 2016

Re: Requirements for units of measure, accuracy and precision

From: Jeremy Tandy <jeremy.tandy@gmail.com>
Date: Tue, 06 Sep 2016 06:08:01 +0000
Message-ID: <CADtUq_0xMwK=EmnZfZRgaMLb05YSHuRbFcgdUUgPpXF1ekZ9Fg@mail.gmail.com>
To: Byron Cochrane <bcochrane@linz.govt.nz>, "Simon.Cox@csiro.au" <Simon.Cox@csiro.au>, "andrea.perego@jrc.ec.europa.eu" <andrea.perego@jrc.ec.europa.eu>, "frans.knibbe@geodan.nl" <frans.knibbe@geodan.nl>
Cc: "rob@metalinkage.com.au" <rob@metalinkage.com.au>, "jlieberman@tumblingwalls.com" <jlieberman@tumblingwalls.com>, "chris.little@metoffice.gov.uk" <chris.little@metoffice.gov.uk>, "public-sdw-wg@w3.org" <public-sdw-wg@w3.org>
Hi. We already do provide a link with in the BP doc to this ... See :
http://w3c.github.io/sdw/bp/#desc-accuracy

That said, it's buried there too. Seems that UoM (and other record/property
level metadata) is deserving of its own BP?

Jeremy


On Tue, 6 Sep 2016 at 00:33, Byron Cochrane <bcochrane@linz.govt.nz> wrote:

> That was buried! I think it would be useful to provide a link in SDWBP to
> this (https://www.w3.org/TR/vocab-dqv/#ExpressDatasetAccuracyPrecision)
> if we hope for others to find this guidance.
>
> Cheers,
> Byron
>
> -----Original Message-----
> From: Simon.Cox@csiro.au [mailto:Simon.Cox@csiro.au]
> Sent: Tuesday, 6 September 2016 11:14 a.m.
> To: andrea.perego@jrc.ec.europa.eu; frans.knibbe@geodan.nl;
> jeremy.tandy@gmail.com
> Cc: rob@metalinkage.com.au; jlieberman@tumblingwalls.com;
> chris.little@metoffice.gov.uk; public-sdw-wg@w3.org
> Subject: RE: Requirements for units of measure, accuracy and precision
>
> Thanks Andrea -
>
> But gosh - you have to dig deep to find it. And the sub-head 'dataset
> precision and accuracy' does not exactly shout 'units of measure' or
> 'scale' does it? It looks like a side-effect rather than a central concern.
>
> Am frankly amazed that this issue is not more central for quantitative
> data, which is a significant subset of general 'data on the web'.
>
> Simon
>
> -----Original Message-----
> From: Andrea Perego [mailto:andrea.perego@jrc.ec.europa.eu]
> Sent: Monday, 5 September 2016 7:15 PM
> To: Frans Knibbe <frans.knibbe@geodan.nl>; Jeremy Tandy <
> jeremy.tandy@gmail.com>
> Cc: Rob Atkinson <rob@metalinkage.com.au>; Joshua Lieberman <
> jlieberman@tumblingwalls.com>; Chris Little <chris.little@metoffice.gov.uk>;
> Cox, Simon (L&W, Clayton) <Simon.Cox@csiro.au>; public-sdw-wg@w3.org
> Subject: Re: Requirements for units of measure, accuracy and precision
>
> Just to note that UoMs are included in some of the examples of DQV, in
> particular in relation to spatial resolution - see:
>
> https://www.w3.org/TR/vocab-dqv/#ExpressDatasetAccuracyPrecision
>
> For more details, see my earlier messages on this topics [1,2].
>
> Please note that the examples above are some of those added by the DWBP WG
> following our comments on the "representation of precision and accuracy" -
> see thread starting here:
>
> https://lists.w3.org/Archives/Public/public-sdw-comments/2016Mar/0001.html
>
> and related DWBP issue:
>
> https://www.w3.org/2013/dwbp/track/issues/243
>
>
> Since DWBP deliverables include UoMs only for spatial resolution, I think
> it's up to us to address the other relevant use cases - possibly by
> building upon the DQV approach, if appropriate.
>
> Andrea
>
> ----
> [1]https://lists.w3.org/Archives/Public/public-sdw-wg/2016Jul/0164.html
> [2]https://lists.w3.org/Archives/Public/public-sdw-wg/2016Jul/0253.html
>
>
> On 02/09/2016 17:26, Frans Knibbe wrote:
> > All,
> > thanks for replying, and a special thanks to Rob for fencing off
> > Jeremy's hijack attempt :-)
> >
> > An attempt to summarize and suggest steps to be taken:
> >
> >   * One of us could submit a comment to the DWBPWG that we feel
> >     indication of UoM and uncertainty in quantitative data deserves
> >     attention in their document. Is there a volunteer?
> >     (I would like to note that an earlier DWBP comment had me suggesting
> >     they should encourage data providers using significant digits, which
> >     can be seen a good way to meet the requirement for indicating
> >     uncertainty in quantitative data. After some discussion, the DWBP
> >     decided the subject was out of scope for data on the web (see this
> >     message
> >     <
> https://lists.w3.org/Archives/Public/public-dwbp-comments/2016Jun/0012.html
> >),
> >     because it applies to the broader domain of quantitative data (that
> >     can exist outside of the Web too). So I think next to the DWBP
> >     already being fixed there is little chance of the comment being
> >     acted upon.)
> >   * There seems to be agreement that the UoM thing should be tied to CRS
> >     definitions. Can we solve the issue by listing an indication of UoM
> >     as one of the types of 'useful information' about a CRS in the CRS
> >     Definition requirement
> >     <
> http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#CRSDefinition
> >?
> >     Of course I would need to list some other typical properties of CRSs
> >     for good measure, but that should not be a problem.
> >   * We add a BP requirement to the UC&R that says "The use of precision
> >     that matches uncertainty in coordinate data should be facilitated
> >     and encouraged". Note that we already havea requirement for
> >     describing spatial resolution in the metadata
> >     <
> http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#SpatialMetadata
> >.
> >
> > I hope we can agree on making concrete changes to the UCR document (or
> > not) soon.
> >
> > Greetings,
> > Frans
> >
> >
> > On 1 September 2016 at 13:41, Jeremy Tandy <jeremy.tandy@gmail.com
> > <mailto:jeremy.tandy@gmail.com>> wrote:
> >
> >     @ Rob:
> >
> >     > I think we need to step back and decide if this is about
> >     requirements or BP
> >
> >     Good point. I'll take the best practice question to a separate
> >     thread ...
> >
> >     On Wed, 31 Aug 2016 at 23:44 Rob Atkinson <rob@metalinkage.com.au
> >     <mailto:rob@metalinkage.com.au>> wrote:
> >
> >
> >         I think we need to step back and decide if this is about
> >         requirements or BP, and target the elements of conversation
> >         better to which deliverable is affected.
> >
> >         IMHO there is a _requirement_ for a canonical mechanism to state
> >         aspects of spatial properties.
> >         I also think its a requirement for that mechanism to be common
> >         with broader requirements for publishing metadata.  Its not a
> >         uniquely spatial concern - but spatial is typified by this being
> >         relevant at dataset, service, feature and property levels of
> >         granularity and the need for users to be able to discover and
> >         interpret this information regardless of where it is best placed.
> >
> >         If there is no consensus on an identifiable Best Practice - IMHO
> >         we should note this as an unmet requirement in the BP document ,
> >         and refer to the catch-all (reuse something!) BP
> >
> >         Rob
> >
> >
> >
> >
> >         On Thu, 1 Sep 2016 at 02:27 Joshua Lieberman
> >         <jlieberman@tumblingwalls.com
> >         <mailto:jlieberman@tumblingwalls.com>> wrote:
> >
> >             Jeremy,
> >
> >             These may not be the terms we want to make a best practice
> >             of. Precision (closeness of repeated measurements) and
> >             accuracy (closeness to truth) are really concepts that are
> >             impossible to compute directly, since one can’t either make
> >             all possible repeated measurements, nor actually make true
> >             measurements. What one can estimate and report is
> >             uncertainty (the range within which the true value has some
> >             likelihood of occurring if we accounted for all sources of
> >             error) and resolution (predicted precision or fineness of
> >             measurement).
> >
> >             The spatial question in this is whether one should report
> >             multidimensional uncertainty and resolution, using such
> >             things as error ellipses or fuzzy geometric boundaries. It
> >             could be a real mess to make a map with fuzzy topographic
> >             contours, but the variation in hypsometric uncertainty
> >             between different published maps would probably shock people
> >             if they were made aware of it.
> >
> >             Josh
> >
> >>             On Aug 31, 2016, at 11:13 AM, Jeremy Tandy
> >>             <jeremy.tandy@gmail.com <mailto:jeremy.tandy@gmail.com>>
> >>             wrote:
> >>
> >>             Ignoring the UoM thing for now (although I can see @josh's
> >>             point about it being needed to define CRS), can folks tell
> >>             me whether BP 5 "Describe the positional accuracy of
> >>             spatial data" [1] is on the right track to meet
> >>             requirements about precision and accuracy? If not, please
> >>             can one of you fine folks make some suggests content
> changes?
> >>
> >>             Thanks in advance. Jeremy
> >>
> >>             [1]: http://w3c.github.io/sdw/bp/#desc-accuracy
> >>             <http://w3c.github.io/sdw/bp/#desc-accuracy>
> >>
> >>             On Wed, 31 Aug 2016 at 12:32 Joshua Lieberman
> >>             <jlieberman@tumblingwalls.com
> >>             <mailto:jlieberman@tumblingwalls.com>> wrote:
> >>
> >>                 Given that there are specific spatial aspects to both
> >>                 precision and uom, for example a precision ellipsoid
> >>                 is a geometry and a unit of measure such as an
> >>                 altitude is relatively to a datum such as a geoid, it
> >>                 seems reasonable to me that these can be included SDW
> >>                 BP's without worrying overmuch whether DWBP will ever
> >>                 get to these sorts of multi-dimensional and affine
> >>                 space issues.
> >>
> >>                 Josh
> >>
> >>                 Joshua Lieberman, Ph.D.
> >>                 Principal, Tumbling Walls Consultancy
> >>                 Tel/Direct: +1 617-431-6431 <tel:%2B1%20617-431-6431>
> >>                 jlieberman@tumblingwalls.com
> >>                 <mailto:jlieberman@tumblingwalls.com>
> >>
> >>                 On Aug 31, 2016, at 07:19, Little, Chris
> >>                 <chris.little@metoffice.gov.uk
> >>                 <mailto:chris.little@metoffice.gov.uk>> wrote:
> >>
> >>>                 Simon, Rob,____
> >>>
> >>>                 __ __
> >>>
> >>>                 I support submitting a formal public comment to DWBP,
> >>>                 AND doing something specific to spatial.____
> >>>
> >>>                 __ __
> >>>
> >>>                 Chris____
> >>>
> >>>                 __ __
> >>>
> >>>                 *From:*Rob Atkinson [mailto:rob@metalinkage.com.au]
> >>>                 *Sent:* Friday, August 26, 2016 1:42 AM
> >>>                 *To:* Simon.Cox@csiro.au <mailto:Simon.Cox@csiro.au>;
> >>>                 frans.knibbe@geodan.nl
> >>>                 <mailto:frans.knibbe@geodan.nl>; public-sdw-wg@w3.org
> >>>                 <mailto:public-sdw-wg@w3.org>
> >>>                 *Subject:* Re: Requirements for units of measure,
> >>>                 accuracy and precision____
> >>>
> >>>                 __ __
> >>>
> >>>                 When this was raised and we asked Phil for guidance,
> >>>                 we were basically told DWBP is pretty much fixed, if
> >>>                 we care it behooves us to address it even ifs its a
> >>>                 more general issue. I think we just need to note its
> >>>                 a more general issue, and provide what we feel is a
> >>>                 BP otherwise there is really no useful path to
> >>>                 implementation.____
> >>>
> >>>                 __ __
> >>>
> >>>                 Given precision is a first class concern of spatial
> >>>                 data - as spatial resolution is going to be ever
> >>>                 critical in the Internet of Things especially, we
> >>>                 need to handle this.  Precision may be common to a
> >>>                 dataset due to a common methodology, specific to a
> >>>                 sensor, or time-varying - such as GPS coordinates.
> >>>                 The BP needs to address how to attach this at this
> >>>                 different levels of granularity. We need to cover
> >>>                 both measured position and gridded coverages as well,
> >>>                 and preferably not with completely different
> >>>                 approaches. Precision is not generally an attribute
> >>>                 of a CRS, unless we are going to define CRS for every
> >>>                 possible grid - and that would be a fairly major
> >>>                 departure from existing practice i suspect.____
> >>>
> >>>                 __ __
> >>>
> >>>                 UoM is a _similar_ case, and not inherently spatial,
> >>>                 so i agree it can be localised to CRS , TRS,  but I
> >>>                 see no harm in tying it in to the same pattern as
> >>>                 spatial precision - whats needed is a BP to provide
> >>>                 metadata about values, to meet the hard requirement
> >>>                 of spatial CRS and precision, but its useful to
> >>>                 suggest this would be applicable to UoM. Certainly
> >>>                 having a completely different set of patterns for
> >>>                 attribution of uom and CRS feels like sub-optimal
> >>>                 practice. ____
> >>>
> >>>                 __ __
> >>>
> >>>                 Rob____
> >>>
> >>>                 __ __
> >>>
> >>>                 __ __
> >>>
> >>>                 On Fri, 26 Aug 2016 at 10:09 <Simon.Cox@csiro.au
> >>>                 <mailto:Simon.Cox@csiro.au>> wrote:____
> >>>
> >>>                     Ø  The lack of information on units of measure
> >>>                     and the apparent lack of concern for proper
> >>>                     indication of uncertainty in numbers are
> >>>                     widespread in spatial data and something should
> >>>                     definitely be done about that. However, I
> >>>                     maintain that both problems are more general than
> >>>                     spatial data and are therefore out of scope for
> >>>                     the UCR document. We have tried hard to limit the
> >>>                     UCR requirements to only spatial data on the
> >>> web.____
> >>>
> >>>                      ____
> >>>
> >>>                     Frans – I agree with this assessment in
> >>>                     principle. Should we try to pass it back up to
> >>>                     the DWBP group? ____
> >>>
> >>>                      ____
> >>>
> >>>                     Best Practice 7
> >>>                     https://www.w3.org/TR/dwbp/#quality
> >>>                     <https://www.w3.org/TR/dwbp/#quality> refers off
> >>>                     to the Data Quality Vocabulary, which may mean
> >>>                     they think it is dealt with, but I’m not
> >>>                     convinced. It’s one more step away from the
> >>>                     readers, at least, and none of the examples show
> >>>                     units of measure, or suggest where they would be
> >>>                     found. ‘units’ or ‘uom’ does not appear in their
> >>>                     document (in the sense that we mean, anyway).
> >>> ____
> >>>
> >>>                      ____
> >>>
> >>>                     However, this document is still in ‘Working
> >>>                     Draft’ status, although it is certainly a pretty
> >>>                     mature document by now. We could start by making
> >>>                     a comment on their public list
> >>>
> https://lists.w3.org/Archives/Public/public-dwbp-comments/
> >>>                     <
> https://lists.w3.org/Archives/Public/public-dwbp-comments/>
> >>>                     to the effect that ____
> >>>
> >>>                      ____
> >>>
> >>>                     ‘The requirement to know the units of measure for
> >>>                     quantitative data (or another kind of ‘reference
> >>>                     system’ for other kinds of data values) is not
> >>>                     mentioned in https://www.w3.org/TR/dwbp/. Units
> >>>                     are required in order to use quantitative data,
> >>>                     so the Data on the Web Best Practices should
> >>>                     include a recommendation on the topic. This
> >>>                     affects the following benefit categories:
> >>>                     Comprehension (C), Processability (P), Reuse (R)’
> >>>                     ____
> >>>
> >>>                      ____
> >>>
> >>>                     But perhaps this is a place where some more
> >>>                     formal coordination between the groups is
> >>>                     warranted? ____
> >>>
> >>>                     @phila – has this issue ever been discussed in
> >>>                     the DWBP group? A quick search of the email
> >>>                     archive doesn’t turn anything up. ____
> >>>
> >>>                      ____
> >>>
> >>>                     --____
> >>>
> >>>                     As a side comment – it is kinda remarkable how
> >>>                     this gets overlooked. I raised it in the Open
> >>>                     Knowledge forum on their Data
> >>>                     Packaging/Frictionless Data initiative. As soon
> >>>                     as I mentioned it there was acceptance that it is
> >>>                     in scope, but somehow no-one had brought it up
> >>>                     until then. ____
> >>>
> >>>                      ____
> >>>
> >>>                     Simon ____
> >>>
> >>>                      ____
> >>>
> >>>                     *From:*Frans Knibbe
> >>>                     [mailto:frans.knibbe@geodan.nl
> >>>                     <mailto:frans.knibbe@geodan.nl>]
> >>>                     *Sent:* Thursday, 25 August 2016 10:50 PM
> >>>                     *To:* SDW WG Public List <public-sdw-wg@w3.org
> >>>                     <mailto:public-sdw-wg@w3.org>>____
> >>>
> >>>
> >>>                     *Subject:* Requirements for units of measure,
> >>>                     accuracy and precision____
> >>>
> >>>                      ____
> >>>
> >>>                     Hello,____
> >>>
> >>>                      ____
> >>>
> >>>                     In messages
> >>>
> https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0172.html
> >>>                     <
> https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0172.html>
> >>>                     and ____
> >>>
> >>>
> https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0109.html
> >>>                     <
> https://lists.w3.org/Archives/Public/public-sdw-wg/2016Aug/0109.html>
> >>>                     the possibility of adding new requirements to the
> >>>                     UC&R doc was brought forward. Those should be
> >>>                     requirements that____
> >>>
> >>>                      ____
> >>>
> >>>                     A) the units of measure (UoM) in spatial data
> >>>                     should be made clear____
> >>>
> >>>                     B) the precision of spatial data should be made
> >>>                     clear____
> >>>
> >>>                      ____
> >>>
> >>>                     From the looks of it, those requirements would at
> >>>                     least be requirements for the BP
> >>> deliverable.____
> >>>
> >>>                      ____
> >>>
> >>>                     The topic was also discussed in the last SSN
> >>>                     teleconference
> >>>                     <https://www.w3.org/2016/08/23-sdwssn-minutes>. I
> >>>                     thought it would be good to create a separate
> >>>                     thread for these related issues.____
> >>>
> >>>                      ____
> >>>
> >>>                     I will first repeat my initial response: The
> >>>                     requirements are very good requirements. The lack
> >>>                     of information on units of measure and the
> >>>                     apparent lack of concern for proper indication of
> >>>                     uncertainty in numbers are widespread in spatial
> >>>                     data and something should definitely be done
> >>>                     about that. However, I maintain that both
> >>>                     problems are more general than spatial data and
> >>>                     are therefore out of scope for the UCR document.
> >>>                     We have tried hard to limit the UCR requirements
> >>>                     to only spatial data on the web. This constraint
> >>>                     is specifically mentioned in the section on
> >>>                     methodology
> >>>                     <
> http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#Methodology
> >.
> >>>                     If we were to neglect this constraint, then the
> >>>                     amount of requirements could run out of hand
> >>>                     quickly, the decisions on which requirements to
> >>>                     include or not would become very arbitrary and
> >>>                     the deliverable teams that are tasked with
> >>>                     meeting requirements would inevitably find out
> >>>                     that they are not in a position to meet the
> >>>                     requirements because they are not in scope for
> >>>                     their work. Of course the deliverable teams will
> >>>                     work with additional requirements next to the
> >>>                     ones mentioned in the UCR document. Those
> >>>                     additional requirements will be based on general
> >>>                     best practices. I think the UoM and precision
> >>>                     requirements fall in that class.____
> >>>
> >>>                      ____
> >>>
> >>>                     That said, perhaps there is a way to shape the
> >>>                     requirements in such a way that they can be
> >>>                     included in the UCR document, without violating
> >>>                     the spatial scope constraint too harshly. After
> >>>                     all, it has been done for other requirements too,
> >>>                     to be honest.____
> >>>
> >>>                      ____
> >>>
> >>>                     Let's start with the UoM requirement (A). I
> >>>                     assume that is about the UoM of coordinate data
> >>>                     only. I think this is already implicitly covered
> >>>                     by the CRS requirements Linking geometry to CRS
> >>>                     <
> http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#LinkingCRS
> >,
> >>>                     Determinable CRS
> >>>                     <
> http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#DeterminableCRS
> >
> >>>                     and CRS definition
> >>>                     <
> http://w3c.github.io/sdw/UseCases/SDWUseCasesAndRequirements.html#CRSDefinition
> >:
> >>>                     If those requirements are met, it should always
> >>>                     be possible to know the UoM of coordinates,
> >>>                     because the UoM will be part of the CRS
> >>>                     definition. Perhaps we should be explicit in
> >>>                     mentioning that a CRS definition should include
> >>>                     an indication of UoM?____
> >>>
> >>>                      ____
> >>>
> >>>                     As for the requirement B, if we change the
> >>>                     wording a bit the requirement could be made
> >>>                     applicable to spatial data only and therefore be
> >>>                     in scope:____
> >>>
> >>>                      ____
> >>>
> >>>                     B2) The use of precision that matches uncertainty
> >>>                     in coordinate data should be facilitated and
> >>>                     encouraged____
> >>>
> >>>                      ____
> >>>
> >>>                     With this kind of wording I think the BP editors
> >>>                     have a fighting chance of meeting the
> >>>                     requirement.____
> >>>
> >>>                      ____
> >>>
> >>>                     Please share your thoughts...____
> >>>
> >>>                      ____
> >>>
> >>>                     Regards,____
> >>>
> >>>                     Frans____
> >>>
> >
> >
>
> --
> Andrea Perego, Ph.D.
> Scientific / Technical Project Officer
> European Commission DG JRC
> Directorate B - Growth and Innovation
> Unit B6 - Digital Economy
> Via E. Fermi, 2749 - TP 262
> 21027 Ispra VA, Italy
>
> https://ec.europa.eu/jrc/
>
> ----
> The views expressed are purely those of the writer and may not in any
> circumstances be regarded as stating an official position of the European
> Commission.
>
> This message contains information, which may be in confidence and may be
> subject to legal privilege. If you are not the intended recipient, you must
> not peruse, use, disseminate, distribute or copy this message. If you have
> received this message in error, please notify us immediately (Phone 0800
> 665 463 or info@linz.govt.nz) and destroy the original message. LINZ
> accepts no responsibility for changes to this email, or for any
> attachments, after its transmission from LINZ. Thank You.
>
Received on Tuesday, 6 September 2016 06:08:42 UTC

This archive was generated by hypermail 2.4.0 : Thursday, 24 March 2022 20:31:25 UTC