Re: QB Data Cube Dicing. Was: Coverage subgroup update

A first start is a classification and identification of these types - then
we can attempt to model their behaviours further. Identification is
sufficient to to trigger inbuilt knowledge in tools on how to handle
things...

have a straw-man in progress - wil be interesting to see if we can cover
off all these cases ;-)

Rob

On Wed, 27 Jul 2016 at 22:55 Little, Chris <chris.little@metoffice.gov.uk>
wrote:

> Jon,
>
>
>
> So, how would an angular/periodic dimension/measure fit in with the RDF QB
> Dimensions?
>
>
>
> Perhaps there are two approaches:
>
> 1.       It is some sort of limited/finitely enumerated dimension like
> (c) but continuous
>
> 2.       It is continuous like (a) but with a limited finite range.
>
> I am not sure that the valid complaint about implementations should affect
> any ontology. Implementers only have to agree or be explicit whether they
> are assuming, in mathematical terms, a left closed/right open interval {
> E.g. 0 <= x < 360 } or vice versa. Of course a fully open interval { 0 < x
> < 360 } or closed interval { 0 <= x <= 360 } are fraught.
>
> I am not sure in constructing ontologies whether (a) with restriction or
> (c) plus extension is a better approach.
>
> I would like the ontologists to chip in here.
>
> Chris
>
>
>
> *From:* Jon Blower [mailto:j.d.blower@reading.ac.uk]
> *Sent:* Wednesday, July 27, 2016 1:14 PM
> *To:* Little, Chris; Rob Atkinson; Peter Baumann; Simon.Cox@csiro.au;
> Joshua Lieberman; bill@swirrl.com; public-sdw-wg@w3.org; Hedley, Mark
>
>
> *Subject:* Re: QB Data Cube Dicing. Was: Coverage subgroup update
>
>
>
> Hi Chris,
>
>
>
> Looks helpful – also there are angular coordinates (chiefly longitude).
> These are a Real Pain in the GIS world, where tools are very inconsistent
> about how they handle wrap-arounds (if they handle them at all), and a
> constant source of irritation when trying to translate between NetCDF and
> GIS stuff.
>
>
>
> Cheers,
> Jon
>
>
>
> *From: *Chris Little <chris.little@metoffice.gov.uk>
> *Date: *Wednesday, 27 July 2016 11:48
> *To: *Rob Atkinson <rob@metalinkage.com.au>, Peter Baumann <
> p.baumann@jacobs-university.de>, Jon Blower <sgs02jdb@reading.ac.uk>, "
> Simon.Cox@csiro.au" <Simon.Cox@csiro.au>, Joshua Lieberman <
> jlieberman@tumblingwalls.com>, "bill@swirrl.com" <bill@swirrl.com>, "
> public-sdw-wg@w3.org" <public-sdw-wg@w3.org>, Mark Hedley <
> mark.hedley@metoffice.gov.uk>
> *Subject: *RE: QB Data Cube Dicing. Was: Coverage subgroup update
>
>
>
> Rob, Simon, Josh, Jon, Peter, and colleagues,
>
>
>
> Having thought a little over the long week end, it seems to me that there
> is a simple, but useful, analysis of Dimensions that is probably worth
> capturing in a simple ontology, though I recognise that this is old hat for
> geospatial diehards and mathematicians. Of course, it may already exist
> somewhere.
>
> Apologies for not knowing any standard ontology notation:
>
>
>
> Dimension:
>
> a)      Coordinate type. I.e. equivalent to the continuous real number
> line, completely (totally) ordered, and any value can be chosen between any
> two others;
>
> b)      Discrete coordinate type. I.e.  equivalent to the integer only
> number line, completely (totally) ordered, and a value cannot be chosen
> between two consecutive  others;
>
> c)      Enumerated ordered set type, with intrinsic ordering which is
> also complete or total (in that any two elements can always be ordered).
> This is just the finite case of (b);
>
> d)      An enumerated set, but without any intrinsic ordering, except an
> ordering dependent on other attributes that could be imposed. E.g. a list
> of countries, placed in alphabetical order in French, or in order of areal
> size, or population.
>
> e)      An enumerated set, but without any intrinsic ordering, except
> ‘artificial’ ordering, not related to any other attributes. E.g. numerical
> identifiers
>
> All could be diced or tiled along a dimension, but would require slightly
> different approaches or implementations for the five categories. I suspect
> (b) and (c) could be merged, as could (d) and (e).
>
>
>
> This seems to fit neatly with CRSs, including temporal, already registered
> in OGC/EPSG and the OWL-Time Ontology.
>
>
>
> Does this seem useful?
>
>
>
> Chris
>
> -------------------------------
>
> *From:* Rob Atkinson [mailto:rob@metalinkage.com.au
> <rob@metalinkage.com.au>]
> *Sent:* Thursday, July 21, 2016 10:58 PM
> *To:* Peter Baumann; Little, Chris; Jon Blower; Simon.Cox@csiro.au;
> Joshua Lieberman; bill@swirrl.com; public-sdw-wg@w3.org; Hedley, Mark
> *Subject:* Re: QB Data Cube Dicing. Was: Coverage subgroup update
>
>
>
> .. im still hearing the undercurrent that this needs a recommended
> practice and primer or a governed definition with known semantics the
> client simply needs to understand... just like geom datatypes.
>
>
>
> On Fri, 22 Jul 2016 12:02 am Peter Baumann <p.baumann@jacobs-university.de>
> wrote:
>
> Hi Chrs,
>
> yes, my pleasure, they come attached. And, as you are asking about an
> ontology, this is reference I had sent to the list earlier:
>
>    1. A. Andrejev, P. Baumann, D. Misev, and T. Risch: *Spatio-Temporal
>    Gridded Data Processing on the Semantic Web*. 2015 IEEE Intl. Conf. on
>    Data Science and Data Intensive Systems (DSDIS 2015), Sydney, Australia,
>    December 11-13, 2015
>
>
> cheers,
> Peter
>
>
>
> On 07/21/2016 02:49 PM, Little, Chris wrote:
>
> Hi Peter,
>
>
>
> Thank you for the references. Do you have a copy that I could read/borrow
> please? I do not have a subscription to the Journals, and can only see the
> abstracts.
>
>
>
> Do the papers discuss a minimal ontology or set of concepts that could be
> useful?
>
>
>
> Chris
>
>
>
> *From:* Peter Baumann [mailto:p.baumann@jacobs-university.de
> <p.baumann@jacobs-university.de>]
> *Sent:* Thursday, July 21, 2016 1:21 PM
> *To:* Little, Chris; Jon Blower; Simon.Cox@csiro.au; Joshua Lieberman;
> bill@swirrl.com; public-sdw-wg@w3.org
> *Cc:* m.riechert@reading.ac.uk; Roger Brackin; Christine Perey (
> cperey@perey.com)
> *Subject:* Re: QB Data Cube Dicing. Was: Coverage subgroup update
>
>
>
> re tiling, we find this for example:
>
>    1. P. Furtado, P. Baumann: *Storage of Multidimensional Arrays based
>    on Arbitrary Tiling.* ICDE'99, March 23-26, 1999, Sydney, Australia
>    2. N. Widmann, P. Baumann: *Performance Evaluation of Multidimensional
>    Array Storage Techniques in Databases*, Proceedings of the 1999
>    International Database Engineering and Applications Symposium, Montreal,
>    Canada, August 1999
>
>
> -Peter
>
> On 07/20/2016 07:31 PM, Little, Chris wrote:
>
> Rob, Jon, Simon, Josh, Bill and colleagues,
>
>
>
> Apologies for spinning off another thread, but this seems a good time and
> place. Kick me well into touch if you wish.
>
>
>
> I have been interested in sub-setting data cubes, as a potentially
> scalable, sustainable approach to supporting large numbers of users/clients
> on lightweight devices. Think generalisation of map tiles to:
>
> a)      Point clouds, vectors, 3D geometries;
>
> b)      N dimensional map tiles, including non-spatial and non-temporal
> dimensions;
>
> c)      Pokemon-Go-Cov;
>
> d)     The WindAR proof of concept from me, Mike Reynolds and Christine
> Perey a couple of years ago;
>
> e)      RDF QB model ‘diced’ as well as ‘sliced’
>
> f)       Etc.
>
>
>
> I thought that the QB model would have enough generality but was
> disappointed to find slices only (but pleased at the simplicity, rigour and
> generality). There was a move in W3C to have some more granularity, but In
> understand that that was driven by the statistical spreadsheet ISO people
> in the direction of pivot tables and temporal summaries, and quite rightly
> failed.
>
>
>
> I would like to increase the generality in the direction of dicing as I
> said. For example, having sliced an n-D cube across a dimension to obtain
> an (n-1)-D cube, it could be still too big, so tile it/pre-format/dice once
> at server side. Map tile sets are the traditional example.
>
>
>
> I think and hope we should be able to rattle of a reasonably good
> extension of QB as a general (non-spatial) concept, and then produce some
> convincing use cases or examples, including spatial and temporal, to make
> it worthwhile.
>
>
>
> Roger Brackin and I failed miserably to get much traction with an OGC SWG
> last year, but I now see many more implementations coercing map tiles, in
> both 2-D and 3-D, for rasters, point clouds, vectors, geometry and more, to
> disseminate or give access to big data. Of course, many Met Ocean use cases
> are for n-D gridded data, where n is 3,4,5,6, …, etc.
>
>
>
> So what do you think?
>
>
>
> Chris
>
>
>
> *From:* Jon Blower [mailto:j.d.blower@reading.ac.uk
> <j.d.blower@reading.ac.uk>]
> *Sent:* Wednesday, July 20, 2016 12:50 AM
> *To:* Simon.Cox@csiro.au; bill@swirrl.com; public-sdw-wg@w3.org
> *Cc:* m.riechert@reading.ac.uk
> *Subject:* Re: Coverage subgroup update
>
>
>
> Hi Simon,
>
>
>
> Ø  QB provides a data model that allows you to express sub-setting
> operations in SPARQL. That looks like a win to me. I.e. think of QB as an
> API, not a payload.
>
>
>
> I’m not an expert in QB by any means, but I understand that the subsetting
> in QB essentially means taking a Slice (in their terminology), which is a
> rather limited kind of subset. I didn’t see a way of taking arbitrary
> subsets (e.g. by geographic coordinates) in the way that WCS could. Can you
> expand on this, perhaps giving some examples of different subset types that
> can be expressed in SPARQL using QB?
>
>
>
> Cheers,
> Jon
>
>
>
> *From: *"Simon.Cox@csiro.au" <Simon.Cox@csiro.au>
> *Date: *Wednesday, 20 July 2016 00:02
> *To: *"bill@swirrl.com" <bill@swirrl.com>, "public-sdw-wg@w3.org" <
> public-sdw-wg@w3.org>
> *Cc: *Maik Riechert <m.riechert@reading.ac.uk>, Jon Blower <
> sgs02jdb@reading.ac.uk>
> *Subject: *RE: Coverage subgroup update
>
>
>
> Ø  The main potential drawback of the RDF Data Cube approach in this
> context is its verbosity for large coverages.
>
>
>
> For sure. You wouldn’t want to deliver large coverages serialized as RDF.
>
>
>
> **But** - QB provides a data model that allows you to express sub-setting
> operations in SPARQL. That looks like a win to me. I.e. think of QB as an
> API, not a payload.
>
>
>
> *From:* Bill Roberts [mailto:bill@swirrl.com <bill@swirrl.com>]
> *Sent:* Wednesday, 20 July 2016 6:42 AM
> *To:* public-sdw-wg@w3.org
> *Cc:* Maik Riechert <m.riechert@reading.ac.uk>; Jon Blower <
> j.d.blower@reading.ac.uk>
> *Subject:* Coverage subgroup update
>
>
>
> Hi all
>
>
>
> Sorry for being a bit quiet on this over the last month or so - it was as
> a result of a combination of holiday and other commitments.
>
>
>
> However, some work on the topic has been continuing.  Here is an update
> for discussion in the SDW plenary call tomorrow.
>
>
>
> In particular I had a meeting in Reading on 5 July with Jon Blower and
> fellow-editor Maik Riechert.
>
>
>
> During that we came up with a proposed approach that I would like to put
> to the group.  The essence of this is that we take the CoverageJSON
> specification of Maik and Jon and put it forward as a potential W3C/OGC
> recommendation.  See
> https://github.com/covjson/specification/blob/master/spec.md for the
> current status of the CoverageJSON specification.
>
>
>
> That spec is still work in progress and we identified a couple of areas
> where we know we'll want to add to it, notably around a URI convention for
> identifying an extract of a gridded coverage, including the ability to
> identify a single point within a coverage. (Some initial discussion of this
> issue at https://github.com/covjson/specification/issues/66).
>
>
>
> Maik and Jon understandably feel that it is for others to judge whether
> their work is an appropriate solution to the requirements of the SDW
> group.  My opinion from our discussions and initial review of our
> requirements is that it is indeed a good solution and I hope I can be
> reasonably objective about that.
>
>
>
> My intention is to work through the requirements from the UCR again and
> systematically test and cross-reference them to parts of the CovJSON spec.
> I've set up a wiki page for that:
> https://www.w3.org/2015/spatial/wiki/Cross_reference_of_UCR_to_CovJSON_spec
>  That should give us a focus for identifying and discussing issues around
> the details of the spec and provide evidence of the suitability of the
> approach (or not, as the case may be).
>
>
>
> There has also been substantial interest and work within the coverage
> sub-group on how to apply the RDF Data Cube vocabulary to coverage data,
> and some experiments on possible adaptations to it.  The main potential
> drawback of the RDF Data Cube approach in this context is its verbosity for
> large coverages.  My feeling is that the standard RDF Data Cube approach
> could be a good option in the subset of applications where the total data
> volume is not excessive - creating a qb:Observation and associated triples
> for each data point in a coverage.  I'd like to see us prepare a note of
> some sort to explain how that would work.  I also think it would be
> possible and desirable to document a transformation algorithm or process
> for converting CoverageJSON (with its 'abbreviated' approach to defining
> the domain of a coverage) to an RDF Data Cube representation.
>
>
>
> So the proposed outputs of the group would then be:
>
>
>
> 1) the specification of the CoverageJSON format, to become a W3
> Recommendation (and OGC equivalent)
>
> 2) a Primer document to help people understand how to get started with it.
>  (Noting that Maik has already prepared some learning material at
> https://covjson.gitbooks.io/cookbook/content/)
>
> 3) contributions to the SDW BP relating to coverage data, to explain how
> CovJSON would be applied in relevant applications
>
> 4) a note on how RDF Data Cube can be used for coverages and a process for
> converting CovJSON to RDF Data Cube
>
>
>
> Naturally I expect to discuss this proposal in plenary and coverage
> sub-group calls!
>
>
>
> Best regards
>
>
>
> Bill
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
> Dr. Peter Baumann
>
>  - Professor of Computer Science, Jacobs University Bremen
>
>    www.faculty.jacobs-university.de/pbaumann
>
>    mail: p.baumann@jacobs-university.de
>
>    tel: +49-421-200-3178, fax: +49-421-200-493178
>
>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>
>    www.rasdaman.com, mail: baumann@rasdaman.com
>
>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>
> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>
>
>
>
>
>
>
> --
>
> Dr. Peter Baumann
>
>  - Professor of Computer Science, Jacobs University Bremen
>
>    www.faculty.jacobs-university.de/pbaumann
>
>    mail: p.baumann@jacobs-university.de
>
>    tel: +49-421-200-3178, fax: +49-421-200-493178
>
>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
>
>    www.rasdaman.com, mail: baumann@rasdaman.com
>
>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
>
> "Si forte in alienas manus oberraverit hec peregrina epistola incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli destinata, nec preripiat quisquam non sibi parata." (mail disclaimer, AD 1083)
>
>
>
>
>
>