Re: FW: Coverage subgroup - document for discussion

Chris

On Thu, 19 May 2016 at 09:11 Rob Atkinson <rob@metalinkage.com.au> wrote:

>
> I would point out that this topic, and this thread, exhibit a wide range
> of overlapping concerns, that would lead a new user to the field to a
> difficult challenge in how to address whcihever of these concerns they
> face. That, in a nutshell, is why we need a BP IMHO.,
>
> Looking at the SDW charter we see:
> "The Recommendation will include provision for describing the subset of
> coverages that are simple timeseries datasets - where a time-varying
> property is measured at a fixed location. OGC's WaterML
> <http://www.opengeospatial.org/standards/waterml> 2 Part 1 - Timeseries
> will be used as an initial basis.
>
> Given that coverage data can often be extremely large in size, publication
> of the individual data points as Linked Data may not always be appropriate.
> The Recommendation will include provision for describing an entire coverage
> dataset and subsets thereof published in more compact formats using Linked
> Data. For example where a third party wishes to annotate a subset of a
> large coverage dataset or a data provider wishes to publish a large
> coverage dataset in smaller subsets to support convenient reuse."
>
> In particular I would not limit the concept of spatial to coordinate
> geometry. IMHO need BP that allow us to describe coordinates, tesselations
> and other forms of grids with hierarchies, and hierarchies of nested
> features.  I would look for a BP that allowed the class of dimension to be
> easily recognised when defining the specific dimension of  a specific
> datacube, and the domain and range of the dimension to be easily accessed -
> i,e, definitions and values. I would also be hoping that there was a model
> that sould allow description of transformations between these, but I'd be
> happy merely for the mechanism for doing this to be identified.  Such a BP
> would allow me to characterise a s set of resources in a useful way, and
> start to work on the next steps.
>
> The same applies to the time dimension, and the sensor.
>
> Each is complex, but some basic patterns for common cases would help, and
> provide the potential for future extension to describe how to process those
> dimensions.
>
> The characterisation of relationships between
> slices/dices/branes/queries/traversals/derivations etc is also very
> complex. My own predilection here would be to get the basics of QB
> dimension description done, and then provide some informative examples of
> how these may then be used in the context of formally describing a subset.
> A future BP effort could then be applied to sorting out all the patterns.
>
> SDMX is not the only possible scope, but at least we know there are a set
> of use cases it does handle. And as Kerry points out, there are things we
> need that it doesnt handle (yet). We know these extensions are complicated
> enough a BP is required.
>
> Each BP pattern would have a limited scope and an example of how it could
> applied in a specific circumstance, and a recomendation for further BP
> scope.
>
> We could prioritise these requirements in different ways:
> 1) identifying dependencies - e.g. you can't do slices on geography
> without having a pattern for the type of geography (slices on coordinates
> are different to slices on some tesselation are different to slices on
> feature geographies). You prioritise from the top of the tree down until
> you can offer something useful
> 2) vote (which probably means taking your favourite end-use case and
> waling back up the dependency tree if one is smart )
> 3) see who works on what and argue about the overlaps.
> 4) plans D, E, etc
>
> My vote would be:
> 1) characterise the main types of spatial dimension in coverages and state
> what is in scope and what is out of scope for BP
> 2) create abstract classes for the in-scope types (on ontology module) and
> accept or reject RDF-QB as a basis for this
> 3) ditto for time
> 4) ask SSN group to review patterns and propose scope for sensors (scope
> may be "no BP recommended" or "future work on BP recommended")
> 5) characterise the main types of spatial and temporal dimension
> subsetting and state what is in scope and what is out of scope for BP
> 6) create abstract classes for the in-scope subsetting types (as ontology
> module), based on the abstract dimensions and accept or reject RDF-QB as a
> basis for this
> 7) develop informative examples of how these ontologies may be used to
> create links and provide enhanced context for accessing services via more
> complex protocols
>
> The JSON and RDF data encoding work could then use the vocabularies
> defined in these ontologies to improve the consistency self-description.
>
> rob
>
>
>
>
>
> On Thu, 19 May 2016 at 02:48 Kerry Taylor <kerry.taylor@anu.edu.au> wrote:
>
>>
>>
>> *From:* Kerry Taylor
>> *Sent:* Thursday, 19 May 2016 12:50 AM
>> *To:* 'Little, Chris' <chris.little@metoffice.gov.uk>
>> *Subject:* RE: Coverage subgroup - document for discussion
>>
>>
>>
>> Chris,
>>
>> I think what you want *was* in the datacube model in an early draft ---
>> a “subslice” but did not make it through to the final version.  I used it
>> in a climate data project. However, relying on the beautiful W3C processes
>> – you might be able to understand the why from here
>> https://www.w3.org/2011/gld/track/issues/34. I think we need something
>> like this for the coverage deliverable if we do go ahead with a qb model.
>>
>>
>>
>> As I get it, the main reason for dropping it is that it is not defined in
>> SDMX (the stats agency standard) and also (Dave Reynolds)
>>
>> > The use case you mention is perfectly reasonable, but it can be
>> addressed by a property in an extension namespace, and can be easily
>> re-added in a future version.
>>
>>
>>
>> So maybe that is something we should do? It makes sense to me.  Surely it
>> can be used to define appropriately granular “extracts” even outside the
>> original data publication. Does this do what you want? I’m not well enough
>> aware of the relevant WCS2.0 capability.
>>
>>
>>
>> --Kerry
>>
>> *From:* Little, Chris [mailto:chris.little@metoffice.gov.uk
>> <chris.little@metoffice.gov.uk>]
>> *Sent:* Wednesday, 18 May 2016 11:49 PM
>> *To:* Rob Atkinson <rob@metalinkage.com.au>; Bill Roberts <
>> bill@swirrl.com>; Jon Blower <j.d.blower@reading.ac.uk>
>> *Cc:* public-sdw-wg@w3.org; Roger Brackin <roger.brackin@envitia.com>
>> *Subject:* RE: Coverage subgroup - document for discussion
>>
>>
>>
>> Hi Rob,
>>
>>
>>
>> This is a bit of a diversion and probably does not help finish this SDW
>> WG  topic, but is a direction I want to go:
>>
>>
>>
>> The QB model of dimensions and slices stops short of what is in OGC
>> WCS2.0 – where any slice can be trimmed (a form of sub-setting) to a
>> bounding box aligned with the dimension axes. So far, so what.
>>
>>
>>
>> I am interested in the wholesale tiling of a data cube, as a one-off
>> process, to enable a wider range of sub-setting and supporting scalability
>> and reuse (if each tile given a persistent enough id). This is not really
>> anything new, and some would argue is only an implementation detail. I am
>> still interested. The tiles may not contain just single values from a
>> simple scalar data cube, but may contain point clouds, vector geometry or
>> other stuff – whatever the contents of the original data cube were.
>>
>>
>>
>> There are a variety of applicable uses cases, such as archive granule
>> retrieval, data dissemination to a very large number of low powered
>> devices, boundary conditions for a large number of local weather prediction
>> models.
>>
>>
>>
>> Whether the tiles are treated as a single multi-dimensional coverage or a
>> collection of a large number of lower dimensional coverages, I do not mind,
>> but it seems to me that this a simple and straightforward addition to the
>> QB model.
>>
>>
>>
>> Is it?
>>
>>
>>
>> Chris
>>
>> *From:* Rob Atkinson [mailto:rob@metalinkage.com.au
>> <rob@metalinkage.com.au>]
>> *Sent:* Wednesday, May 18, 2016 1:50 PM
>> *To:* Bill Roberts; Jon Blower
>> *Cc:* public-sdw-wg@w3.org
>> *Subject:* Re: Coverage subgroup - document for discussion
>>
>>
>>
>> Hi,
>>
>> I've put some detail on the page
>> https://www.w3.org/2015/spatial/wiki/Data_cube_for_coverage to identity
>> different possible directions for this aspect.
>>
>>
>>
>> FYI My project with OGC is concerned with UC1 and UC2, which seems
>> complementary to the other activites supporting this thread.
>>
>>
>>
>> Cheers
>>
>> Rob Atkinson
>>
>>
>>
>>
>>
>> On Wed, 18 May 2016 at 20:45 Bill Roberts <bill@swirrl.com> wrote:
>>
>> Thanks Jon, that's a useful perspective.  Certainly we talk about making
>> discovery and retrieval of the data easier, working nicely with web-based
>> technology etc - so we need to be clear about 'easier for whom'.
>> Inevitably different people will want different things so we will have to
>> be explicit about our priorities.
>>
>>
>>
>> The existing use cases cover quite a few of the scenarios you have
>> sketched out, but they don't yet link those to these kind of user
>> personas.  That might be worth doing - it probably wouldn't take long.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On 18 May 2016 at 11:35, Jon Blower <j.d.blower@reading.ac.uk> wrote:
>>
>> Hi Bill, all,
>>
>>
>>
>> Just some initial thoughts in advance of our telecon. There is lots of
>> good stuff in here, and it’s all relevant to the general area of
>> “Coverages”. Some of these issues are of course very complex and I don’t
>> think we’ll solve them all – and in fact this group might not be the best
>> place to do so.
>>
>>
>>
>> I wonder if it would help to structure the document and our thinking
>> around the different audiences we might aim at. For example:
>>
>>
>>
>>  * A “web developer” might need some explanation of what a coverage is
>> (“dummies’ guide”). He/she would probably like a simple API to access them,
>> and some simple formats with which he/she is familiar. The applications are
>> likely to be reasonable simple and visualisation-oriented, rather than
>> “deep” analysis.
>>
>>
>>
>>  * A “spatial data publisher” might already be familiar with the
>> terminology, but might want to know how to make his/her data more
>> discoverable by mass-market search engines, or how best to make use of
>> Linked Data and semantic stuff. He/she is probably going to be keen to
>> describe coverage data very precisely (e.g. using the “right” CRS and
>> full-res geometries), but is also interested in the cost/benefit tradeoff.
>>
>>
>>
>>  * A “data analyst/scientist” might be interested in quality and
>> uncertainty, and how to bring coverage data into his/her tools (e.g. GIS,
>> Python scripts). This kind of person may just want to download the data
>> files in an unmodified form, although data-extraction services can be
>> useful in some circumstances (and hosted processing is increasingly
>> popular).
>>
>>
>>
>>  * An “environmental consultant” may have very limited time to perform
>> some kind of analysis to form part of a report. If a dataset is hard to
>> find, access or understand it will probably simply be omitted from the
>> analysis. Often interested in a very specific geographic area. Needs to
>> quickly establish that a dataset is trustworthy,
>>
>>
>>
>>  * An “IT provider” might be interested in scalable and maintainable web
>> services for high-volume data that can be made part of his/her
>> organisation’s operational procedures. He/she probably has a low tolerance
>> for high-complexity or “bleeding edge” technology.
>>
>>
>>
>> This is just off the top of my head, and there are certainly more, and
>> there will also be lots of overlap. And I’m sure there’s lots to argue
>> about there. But this helps me, at least, put some structure on the Big
>> List. For each of these kinds of user, what would be the most useful thing
>> that we could do to help them (maybe a new technology, or a recommendation
>> to use something existing, or an admission that the problem remains
>> unsolved), in the context of this group?
>>
>>
>>
>> (Am I just reinventing the Use Cases here, or is this still useful for
>> the Coverage requirements?)
>>
>>
>>
>> Cheers,
>>
>> Jon
>>
>>
>>
>>
>>
>>
>>
>> *From: *Bill Roberts <bill@swirrl.com>
>> *Date: *Tuesday, 17 May 2016 23:44
>> *To: *"public-sdw-wg@w3.org" <public-sdw-wg@w3.org>
>> *Subject: *Coverage subgroup - document for discussion
>> *Resent-From: *<public-sdw-wg@w3.org>
>> *Resent-Date: *Tuesday, 17 May 2016 23:44
>>
>>
>>
>> Hi all
>>
>>
>>
>> I've made some initial notes on requirements in this wiki page:
>>
>>
>>
>> https://www.w3.org/2015/spatial/wiki/Coverage_draft_requirements
>>
>>
>>
>> I'd like to go through this on the call tomorrow (we probably won't get
>> all the way through it as there is quite a lot there).  If you are joining
>> the call it would be great if you could look at it in advance.
>>
>>
>>
>> Comments also welcome via this mailing list.
>>
>>
>>
>> Cheers
>>
>>
>>
>> Bill
>>
>>
>>
>>
>>
>>
>>
>>