Large Language Models, Visualizations, and Infographics

Civic Technology Community Group,

Hello. Earlier, I indicated that "users will soon be able to ask natural-language questions and engage in multimodal dialogues about large-scale, public-sector financial, accounting, and budgetary data, receiving responses comprised of language, mathematics, charts, diagrams, figures, graphs, infographics, and tables."

Should these topics interest you, here are some recent scientific advancements pertaining to large language models, visualizations, and infographics:


LIDA: A Demo Video
Victor Dibia
https://vimeo.com/820968433


LIDA: Automatic Generation of Grammar Agnostic Visualizations and Infographics with Large Language Models (ChatGPT, GPT4)
Victor Dibia
https://newsletter.victordibia.com/p/lida-automatic-generation-of-grammar

"This post provides a high-level description of the design of a tool (LIDA) that supports users in automated data exploration and visualization/infographic generation using LLMs and image generation models (IGM’s).

"TLDR; LIDA provides the following capabilities.

  *   Data Summarization: Create a compact but information dense natural language representation of datasets, useful as grounding context for data operations with LLMs.
  *   Automatic Data Exploration: Given some raw data, come up with data exploration goals that make sense for this data. EDA for free!
  *   Grammar Agnostic Visualization Generation: Generate visualizations in any language, any visualization grammar (e.g., matplotlib, ggplot, altair etc).
  *   Infographic Generation: Generate stylized but “data-faithful” infographics, directly from data. Extensive applications in interactive data storytelling.
  *   Visualization Ops: Enables a set of operations on generated visualizations including - natural language based visualization refinement (.e.g change the x axis to .. translate chart to … zoom in by 50% etc), visualization explanation (code explanation, accessibility descriptions), visualization code self-evaluation (evaluation on dimensions such as aesthetics, compliance, type, transformation etc). Many applications here for accessibility, education and learning."


LIDA: A Tool for Automatic Generation of Grammar-agnostic Visualizations and Infographics using Large Language Models
Victor Dibia
https://arxiv.org/abs/2303.02927

"Systems that support users in the automatic creation of visualizations must address several subtasks - understand the semantics of data, enumerate relevant visualization goals and generate visualization specifications. In this work, we pose visualization generation as a multi-stage generation problem and argue that well-orchestrated pipelines based on large language models (LLMs) and image generation models (IGMs) are suitable to addressing these tasks. We present LIDA, a novel tool for generating grammar-agnostic visualizations and infographics. LIDA comprises of 4 modules - A SUMMARIZER that converts data into a rich but compact natural language summary, a GOAL EXPLORER that enumerates visualization goals given the data, a VISGENERATOR that generates, refines, executes and filters visualization code and an INFOGRAPHER module that yields data-faithful stylized graphics using IGMs. LIDA provides a python API, and a hybrid USER INTERFACE (direct manipulation and natural language) for interactive chart, infographics and data story generation."



Best regards,
Adam Sobieski