Semantic 3D Content Accessibility and Educational Technology

Semantic 3D Content Accessibility Community Group,

Hello. Bridging semantics with visuospatial content will enable many exciting features for digital textbooks and other instructional materials involving interactive 3D-graphics diagrammatic widgets.

For discussion:


  1.
In addition to attaching semantics to 2D/3D points, one could use 2D/3D bounding boxes, silhouettes, and/or meshes.
  2.
In addition to graph-based metadata, metadata with extensive natural-language content can be of use for AI interoperability.
  3.
In addition to attaching static semantics to static 3D model contents, dynamic and stateful 3D models could vary their scene subgraphs or their "object models" as they underwent animations and/or transformations.

With respect to accessibility, AI systems could update widgets' "alt text" metadata as content varied, using semantic metadata and, perhaps, computer vision on (on-screen) renderings. With these capabilities, at any point in time, visually-impaired end-users or their tools could request the "alt text" of what was displayed in interactive 3D-graphics widgets.

It is interesting to consider how (dynamic) scene graphs and metadata could enable more general-purpose conversational user-interfaces for 3D scenes and objects, perhaps via MCP servers or AI agents available per widget. End-users could ask their AI assistants: "what is visible in the active diagram", "is the Golgi apparatus visible in the active diagram", or say: "zoom in on the mitochondria in the active diagram", or, "which part of the displayed engine is the carburetor", or "rotate the product to show the feature."

Thank you. I hope that these ideas are of some interest. I look forward to discussing these and any other semantic 3D content accessibility technical topics.


Best regards,
Adam Sobieski