Re: VSS instantiation discussion

On Wed, 2020-07-08 at 09:54 +0000, Schildt Sebastian (CR/ADT1) wrote:
> Hello,
>  
> As usual (well most of the time J ) what Ulf says makes sense.
>  
> First two points I would agree, however for the last one
> “All metadata from a node in YAML MUST be preserved in the translations,
> except for instantiation metadata that MUST not be present. “
>  
> I am not so sure. Because if exporting to “dumb” formats like .csv it
> might not even be possible to do that in a sane way. 

As of today I would say that as of today all formats are equally dumb or
smart (basically they include exactly the same information) but maybe not
in some of the future scenarios that were discussed.
 
Please explain further what you mean that it is not possible to do in a
sane way.   We have made proposals already how to describe the complete
"flat" tree after insantiation has happened.  So those are not sane or your
mind is on a different scenario.

> Personally I doubt the usefulness of CSV export anyway, but since some
> people use it/need it, such things should not be forbidden.

I have the same view of CSV but almost for all other formats too, including
JSON, as we discussed yesterday.   Individual consumers may want to have
these formats, but the VSS itself should stay concentrated on the meaning
of the source format in YAML.  That said, I think it's also most likely
that VSS must define some things about how instantiation is transformed
into a concrete tree, instead of leaving that open for implementations. 
I'm mostly hoping for a single basic translation in all cases, not
different ones depending on the target format.

Whether the VSS-tools provides converters to different formats is a
separate issue.  We should expect even more of them to pop up:

Recently, a converter to Franca IDL was updated and functioning.  This
serves a purpose because there are other existing and proven tools that
ingest Franca IDL, and because it is a generic interface language used for
other reasons.  (And for that matter the Vehicle Service Catalog - VSC will
be functionally compatible with Franca as well).

The same should be true for generation of AUTOSAR-XML, in my personal
opinion.  Also there we have a situation with  a platform and toolset that
expects a certain input format.

While I'm on the topic -- more examples:  We have already made conversions
of VSS to GraphQL schema, and tooling should to go into vss-tools
eventually.

Conversion to DTDL (basically JSON-LD) which is used by Microsoft is to be
expected, and it remains to be seen if such a converter might also be
hosted in GENIVI/vss-tools or not.

So, providing those tools can be useful but we should try to keep the
discussions apart if we can.   (continued below)
I also think in a lot of those cases a de-facto implementation could be
enough, with not so much formal requirement specifications at least
initially.

As far as CSV being (not) useful, my opinion is:
Either implement a ".TXT" format that only lists the VSS node names with
full paths, and nothing else, or simply accept that the *first column* of
the CSV is what we have used to serve this purpose.   That's my view at
least on what the CSV format is being used for today.

> I think we should formulate as guidelines
>   “All metadata from a node in YAML SHALL be preserved in the
> translations”

Sounds reasonable in general but I'm not sure I agree.  Maybe it is  a
devil's advocate counter point, but I could envision some target technology
where not all metadata is used.  So why then force to include it in a
conversion?  Let's say a particular target has no possibility to make use
of the description line, for example.  Why would you then put it into that
generated file?

Perhaps by preserved you mean that it should be a reasonable conversion *if
it is included* so that a kind of semantic compatibility remains?  I would
lean more towards such a statement.

Basically I think it is already implied that anyone will include all the
metadata that will be useful, and there is no risk of forgetting to do
that.  So IMHO no rule required, if you meant ath *all* metadata must be
included.   Again, this might be because I'm leaning towards the view that
most target formats are the responsibility of the "consumer" that needs
that target format.  And that consumer will surely ensure that the metadata
that is useful will be there, and none other.  Seems enough for me.

> For instantiation it is really the “instances are a VSS feature” vs.
> “instances are just syntactic sugar” we started yesterday . In the second
> case (I feel we are leaning that way currently)

Possibly but I'm not yet leaning that way.  I think the original plain-tree 
thinking is still a powerful model and a good result format.  It is
limiting from one sense, but it is keeping simplicity from another.

I fear that it is premature to try to solve what could become a much larger
issue. I am concerned we solve one issue, and then run immediately into the
next.  If we could call this  "a discussion for VSS version 3" then I would
be more comfortable.   Once we put instantiation to bed (and array and enum
types maybe), VSS 2 is already a hugely useful standard, in my view, that
production projects could actually use (as was done already with v1), and
it should not be held back by a v3 discussion that could take a long time
to get right.

One thing to remember is that the VSC is being built up now, and that is
basically something that will have almost all the modeling capability of
Franca IDL!  So there is a lot there that might come in and impact things.
 
In the longer run, the most logical view of the future seems to me that
VSS+VSC eventually merges into some kind of combined model.  And for that
reason I want to warn against stop-gap solutions in VSS that might be
limiting for a larger rewrite later on.

I think we would benefit from pushing some of this towards a "v3"
timeframe, but maybe we allow just a little more analysis (with Daniel's
coming examples) to see.

> “Instantiation metadata SHALL be processed and expanded so the output
> represents a flattened VSS tree”

Seems like we are in agreement.  But this is the "syntactic sugar" approach
I guess?  Unless you have some other notion of flattened tree, that somehow
keeps the information about instances.
 
> Then I feel for “common” output formats we have today (like JSON, CSV),
> we should find some middle ground between saying “this IS VSS” and “this
> is just random code, please ignore”. 

Fully agree.  Could we simply say that this is VSS vs VSS-tools?   VSS
should remain a specification, which may include _some_ requirements on how
to convert in certain cases.  Then some parts of VSS-tools might follow
that specification, whereas some might not.  I mean some tools might ADD
things that do not contradict -- never contradict but go *beyond* the
requirements stated in the VSS specification.

There is another and I think even more important distinction, because some
parts of the industry are still conflating these things, and that is:

1. The industry-wide standard catalog of signals.
2. The shared format/tooling/methodology of VSS, which is applicable both
for standard signals and proprietary catalogs.

The 2) is it really the "data model" if we want to pick up that definition
game again?  A model, is just that, it is a model for how to model data? 
It is not the actual data items.  Or???  Naming this is important.

I'm leaning towards that these two things should have explicitly different
names, because right now the concept of "VSS" kind of implies both of them,
and it seems to confuse some people.   Back on-topic for W3C protocols, it
is also somewhat unclear I think - the VISS protocol seems to embrace both
1) and 2), and does not distinguish betwen them.  But we should recognize
also that VISS and Gen2 are also technologies that *could* apply to
proprietary data.

> We can say “When exporting to JSON/CSV, the canonical JSON/CSV output
> format should look like this”, because I think even following the 3 rules
> you laid out, there are probably many JSON formats that can adhere to
> them.
>  
> Generators we are still “playing” with we can just mark as experimental
>  
>  
> Mit freundlichen Grüßen / Best regards
> 
> Sebastian Schildt 
> 
> Communication and Network Technology (DS1) (CR/ADT1)
> Robert Bosch GmbH | Renningen | 70465 Stuttgart | GERMANY | www.bosch.com
> Tel. +49 711 811-15765 | Mobil +49 173 7124227 | 
> Sebastian.Schildt@de.bosch.com
> 
> Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart, HRB 14000;
> Aufsichtsratsvorsitzender: Franz Fehrenbach; Geschäftsführung: Dr.
> Volkmar Denner, 
> Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian
> Fischer, Dr. Stefan Hartung,
> Dr. Markus Heyn, Harald Kröger, Christoph Kübel, Rolf Najork, Uwe
> Raschke, Peter Tyroller 
> 
> Von: Ulf Bjorkengren <ulfbjorkengren@geotab.com> 
> Gesendet: Dienstag, 7. Juli 2020 21:20
> An: public-automotive <public-automotive@w3.org>
> Betreff: VSS instantiation discussion
>  
> Regarding the VSS instantiation discussion I think we need to establish a
> few ground rules, in order to not get lost. Below is my attempt on a few
> rules for such a list.
>  
> - YAML MUST unambiguously specify the path to a leaf node.
> - Translations from YAML to other formats MUST preserve the path
> definitions. 
> - All metadata from a node in YAML MUST be preserved in the translations,
> except for instantiation metadata that MUST not be present. 
>  
> Regarding Gen2 and JSON I would say that Gen2 is agnostic to what format
> the VSS tree uses. Gen2 uses JSON as payload format, but that has nothing
> to do with what format the VSS tree is represented in. In the Gen2 impl
> project the VSS tree is represented in the native C format, as an
> example. 
>  
> BR
> Ulf
> --
>  Ulf Bjorkengren
>  Geotab
>  Senior Connectivity Strategist | Ph. D. 
>  Mobile	 +45 53562142
>  Visit	 www.geotab.com
>

Received on Wednesday, 8 July 2020 11:12:10 UTC