Re: Web Charts

Semantic Web Interest Group,

A quick update: I recently broached Web Charts as a WHATWG GitHub proposal (https://github.com/whatwg/html/issues/9295).

In that discussion thread, we're talking about how end-users can search for, create, and style data visualizations using AI systems and also how AI systems can both answer users' questions about and engage in dialogues about the contents of data visualizations and about documents containing them, including in accessibility contexts.

The posts which broached accessibility innovations, in particular, are:

  1.  https://github.com/whatwg/html/issues/9295#issuecomment-1553783881

  2.  https://github.com/whatwg/html/issues/9295#issuecomment-1554341200


In these recent posts, I indicate uses of templates, e.g., Handlebars-like templates, to automatically generate natural-language text descriptions for parts of charts and for charts in their entireties such that these natural-language text descriptions could be subsequently processed by AI systems, enhancing their understandings of the contents of the charts to enable subsequent question-answering and dialogue.

Should any of these topics interest you, please do feel free to join the discussion. Thank you.


Best regards,
Adam Sobieski

________________________________
From: Adam Sobieski <adamsobieski@hotmail.com>
Sent: Saturday, May 13, 2023 4:34 PM
To: Hugh Glaser <hugh@glasers.org>
Cc: Alvaro Graves <alvaro.graves@gmail.com>; Martin Georg Skjæveland <martige@ifi.uio.no>; semantic-web@w3.org <semantic-web@w3.org>
Subject: Re: Web Charts


Hugh,


Thank you. I will take a closer look at the RDF Data Cube Vocabulary (https://www.w3.org/TR/vocab-data-cube/).

One of the points, or what is new, is the capability to use natural-language dialogue to engage with AI systems, e.g., LLMs, to search for, to create, and to style charts and infographics, and for AI systems to include charts and infographics in their multimodal responses. Hopefully, something like "Web Charts" can enhance these technologies while reducing computation- and transmission-related overhead costs for sending charts and infographics, alongside natural-language text, in AI systems' multimodal responses, at scale.

Due, in part, to extensibility, Semantic Web data formats are competitive with, if not advantageous to, other format possibilities. Presently, CSV and JSON are utilized by in the samples section of the LIDA project (https://microsoft.github.io/lida/gallery).

Extensible data for use in charts and infographics, processed and understood by the scripting logic which produces charts' interactive document structures (which would then be subsequently styled), could enhance interactive charts and infographics, for example highlighting or placing hover-over tooltips on particular data points in a chart or infographic.



Best regards,

Adam

________________________________
From: Hugh Glaser <hugh@glasers.org>
Sent: Saturday, May 13, 2023 9:40 AM
To: Adam Sobieski <adamsobieski@hotmail.com>
Cc: Alvaro Graves <alvaro.graves@gmail.com>; Martin Georg Skjæveland <martige@ifi.uio.no>; semantic-web@w3.org <semantic-web@w3.org>
Subject: Re: Web Charts

I may have missed the point, but I understand your “Web Charts” are about charting data?
(Rather than perhaps visualising the content of generic SPARQl stores, for example.)
If so, then the place we often start for representing and then querying the data is https://www.w3.org/TR/vocab-data-cube/ , which doesn’t seem to have been mentioned.
Good luck
Hugh

> On 12 May 2023, at 04:10, Adam Sobieski <adamsobieski@hotmail.com> wrote:
>
> Exciting news! I recently found that LIDA might soon be an open-source project: https://microsoft.github.io/lida/ . In the frequently asked questions' section "Source Code?", it says: "LIDA is an ongoing project and there is work underway to package and release the code. Stay tuned!"
>
> Research and development are underway with respect to creating and styling data visualizations, e.g., charts and infographics, using dialogue systems and chatbots and I am starting to also think about enhancing conversational search for retrieving existing such items. End-users would, then, be able to search for existing data visualizations and, if their desired data visualizations did not already exist, they would be able to continue in dialogue to create and to style new ones for their needs.
>
> Similarly, AI systems responding to users' questions would be able to search for existing multimedia resources before creating new ones as components of their multimodal responses.
>
> Back to "Web charts", for enhancing searching for existing charts, we might expand the model and the markup to include support for one or multiple metadata elements:
>
> <chart>   <metadata type="text/plain" lang="en">...</metadata>
>   <query endpoint="..." type="application/sparql-query">...</query>
>   <script type="text/javascript" src="..." />
>   <style type="text/css" src="..." />
> </chart>
>
> Also, on another note, image-based fallbacks could be specified for charts, e.g.,:
>
> <chart>
>   <metadata type="text/plain" lang="en">...</metadata>
>   <fallback type="image/png" src="..." />
>   <query endpoint="..." type="application/sparql-query">...</query>
>   <script type="text/javascript" src="..." />
>   <style type="text/css" src="..." />
> </chart>
>
> though we might want to use a source element instead:
>
> <chart>
>   <metadata type="text/plain" lang="en">...</metadata>
>   <fallback>
>     <source media="..." type="image/png" src="..." />
>     <source media="..." type="image/png" src="..." />
>   </fallback>
>   <query endpoint="..." type="application/sparql-query">...</query>
>   <script type="text/javascript" src="..." />
>   <style type="text/css" src="..." />
> </chart>
>
> Note that these models and markup are offered here for purposes of discussion and brainstorming and please do feel free to improve upon them as interesting to you.
>
>
> Best regards,
> Adam
>
> From: Adam Sobieski <adamsobieski@hotmail.com>
> Sent: Thursday, May 11, 2023 6:12 PM
> To: Alvaro Graves <alvaro.graves@gmail.com>; Martin Georg Skjæveland <martige@ifi.uio.no>
> Cc: semantic-web@w3.org <semantic-web@w3.org>
> Subject: Re: Web Charts
>  Thank you all for indicating and sharing D3.js, D3-SPARQL, SGVIZLER/SGVIZLER2, the Web Components repository: Component Party, SparqlExplorer, and the relevant publication: Content-preserving Graphics by Timothy Lebo, Alvaro Graves, and Deborah L. McGuinness.
>
> Also, observing that LLMs, e.g., ChatGPT, are being discussing in another thread in this mailing list, those interested in both "LLMs" and "charts and infographics" might find interesting the LIDA project, where, through natural-language dialogue, users can create and style charts and infographics.
>
> A demo video is available here: https://vimeo.com/820968433

> A publication is available here: https://arxiv.org/abs/2303.02927

>
> Perhaps a "Web charts" discussion and project could be useful for enhancing the dialogue-driven search for and creation of charts and infographics.
>
>
> Best regards,
> Adam
> From: Alvaro Graves <alvaro.graves@gmail.com>
> Sent: Thursday, May 11, 2023 10:08 AM
> To: Martin Georg Skjæveland <martige@ifi.uio.no>
> Cc: adamsobieski@hotmail.com <adamsobieski@hotmail.com>; semantic-web@w3.org <semantic-web@w3.org>
> Subject: Re: Web Charts
>
> This paper we wrote might be relevant, since it shares similar ideas
>
> https://ceur-ws.org/Vol-1034/LeboEtAl_COLD2013.pdf

>
>
> On Wed, May 10, 2023 at 11:16 PM Martin Georg Skjæveland <martige@ifi.uio.no> wrote:
> Hi,
>
> this reminds me of something I made a while back:
>
>   https://mgskjaeveland.github.io/sgvizler/

>
> which has been taken further by someone else to
>
>   https://bordercloud.github.io/sgvizler2/

>
> Not as sophisticated as your describe, I think, but along the same
> lines.
>
> Martin
>
> On Wed, 2023-05-10 at 21:20 +0000, Adam Sobieski wrote:
> > Semantic Web Interest Group,
> >
> > Hello. I’m not sure which other W3C mailing lists, if any, might also
> > be interested in these ideas but, as they involve Semantic Web
> > technologies, I'm writing here, today, to share some ideas pertaining
> > to charts-related markup which could be utilized from HTML.
> > Starting simply, we might consider that charts have: (1) data, (2)
> > scripting logic with which to produce document structure from that
> > data, and (3) a means of styling the produced document structure for
> > presentation and display.
> >
> > <chart version="1.0">
> >   <data type="application/rdf+xml" >...</data>
> >  <script type="text/javascript" src="..." />
> >  <style type="text/css" src="..." />
> > </chart>
> >
> > Next, we might consider that charts' data could be stored in remote
> > resources.
> >
> > <chart version="1.0">
> >   <data type="application/rdf+xml" src="..." />
> >   <script type="text/javascript" src="..." />
> >   <style type="text/css" src="..." />
> > </chart>
> >
> > Next, towards more advanced scenarios, we might consider that charts
> > might, instead of having static data provided, obtain dynamic data by
> > querying service endpoints, e.g., using SPARQL.
> >
> > <chart version="1.0">
> >   <query endpoint="..." type="application/sparql-query">...</query>
> >  <script type="text/javascript" src="..." />
> >   <style type="text/css" src="..." />
> > </chart>
> >
> > We might also consider specifying some attributes for these query-
> > related elements which pertain to the refreshing of data, e.g., for
> > indicating whether a chart was user-refreshable (per context menus),
> > whether it refreshed on a timer, and/or any other behaviors
> > pertaining to its input data.
> >
> > Any thoughts on these preliminary ideas pertaining to "Web charts"?
> > Are these initial ideas possible with existing technologies, e.g.,
> > Web Components?
> >
> >
> > Best regards,
> > Adam Sobieski
> > http://www.phoster.com

> >
> > P.S.: I thought of these ideas while updating theWikianswersproject
> > proposal: https://meta.wikimedia.org/wiki/Wikianswers .
> >
>
> --
> ----
> Alvaro Graves-Fuenzalida
> Web: http://graves.cl - Twitter: @alvarograves