Regarding user interfaces and smart clients

I'd really welcome any comments or thoughts to the following ideas that
I've been formulating. There's more generic linked data stuff in this post
than Hydra-specific discussion, but I'm including it here because the
entire crux of what I'm talking about revolves around the construction of a
modern web application using JSON-LD and Hydra.

First, my assumptions based on what I've learned (please correct me on
anything I'm wrong about):

1) To play nicely with the ideals of the semantic web, my client should be
able to consume external linked data, and external clients should be able
to consume my data. The same goes for my Hydra API, which is of course the
entire point of Hydra.

2) To be purely HATEOAS/REST-compliant, I should be able to point my client
at any endpoint which offers data in a format I can support (in my case, at
least JSON-LD and Hydra) and have it be able to offer some kind of user
experience using only what is returned from that endpoint. Any given
request should include links so that a client can advertise potential
further actions to the user. The client should not make any assumptions in
advance about what will be returned, because those assumptions will
inevitably be wrong if anything about the endpoint changes in the future.

## User Interface ##

My experience with application design is that the more generic you make a
client's capabilities, the harder it is to offer a really good user
experience. This of course is a problem when we're talking about linked
data and Hydra, as being generic is pretty much a necessity. So I've been
thinking about how we can "have our cake and eat it too". How can we offer
a great user experience, tailor our application to our website's needs, and
yet somehow remain HATEOAS/REST compliant so that we remain completely
interoperable with clients that know nothing about our application?

There are no two ways about this. User interface is really, really
important, and I can't help but notice a lack of focus on this fact by the
semantic web community. Unless you're talking about completely autonomous
agents, which I believe is Ruben's area of interest, you simply can't get
away from thinking about the user interface, and how you might offer a good
user experience. UI/UX experts get paid because they spend the time to
consider the data and service in advance and craft something that the user
is going to love, and which is going to do the best job possible in
allowing the user to interact with a service and its data. This usually
involves careful analysis of what the service provides, what is important,
and the various workflows that the user is expected to engage in.

How do we get our client to fill this role adequately? In all honestly, I
don't think we can, given what is on the table (of specifications) at this
point. I've seen discussions on the list where it is argued that attributes
such as labels and so forth have no place in the data because these are
presentational attributes and should not be intrinsically linked to the
data. I understand the argument, but then something still needs to be
offered whereby we can advertise metadata with which a decent user
interface can be constructed.

Questions which are precursors to an implementation model that I think
would work:

(a) If I return a JSON-LD document in response to an HTTP request, the
general idea of hypermedia and linked data is that I should include
hypermedia controls to related data. Hypermedia controls seem to me be
context-specific metadata and not always intrinsically-related to the
requested resource. Is this line of thinking correct?

(b) As above, is it generally understood that the document and its related
hypermedia controls have a dynamic relationship? For example, if I return
information about a user, then perhaps a link to the user's account history
will be included, or perhaps it won't; maybe the person making the request
doesn't have the appropriate credentials and shouldn't be offered certain
links.

(c) Building further on the above points; I know I also want to include a
link to a Hydra entry point, but is it appropriate to include a link to
that entry point from every document returned from my application? One
would assume that any route offered by my application may be called as a
first point of entry, and as such, the entry point would need to be
advertised so that the client is able to figure out what the API can do.

So my implementation model that has been materialising in my head looks
something like this:

1) All general requests to a server are subject to content negotiation with
respect to the data requested and the specified Accept header. So in most
cases, if the client requests JSON-LD, that's what it gets. If it requests
HTML though, we need to somehow decide how to render the [arbitrary] data
using HTML. That HTML would therefore need to include enough so that the
full client can bootstrap itself and then display the data that was
originally requested.

2) A client cannot truly be intelligent. If it receives an object of @type
schema:LocalBusiness, it isn't going to know that one attribute is more
important than another, or that certain attributes probably don't need to
be displayed initially, or that certain groups of attributes, if appearing
in tandem, would be better rendered using a specialised view, rather than
individually in a table of values. We need to teach our client how to
render a user interface in a way that both suits our application's
domain-specific requirements, and can also adapt to linked data that is
beyond the application's initial scope.

3) All linked data ultimately boils down to literals, so our client needs
to understand how to render literals, which means, at the absolute minimum,
it can take a set of triples, either directly or from a deconstructed
JSON-LD document, and render them out to a simple table of values.

4) We then "teach" our client to render common vocabulary entities in a
better way. Internally, this means overriding the default behaviour whereby
the client renders something instead of a general table of values, and for
additional data it can't account for, the interface/template/whatever that
is built to cater for that entity type, tries to make allowances for
additional data, such as rendering an optional table at the bottom,
offering a popup with the extra values, or whatever is best, given the
current context.

5) To make our client (paradoxically) both domain-specific and generic, we
accept that our API entry point is domain-specific by definition, so we
also teach our client how to render for our own needs. This means that the
method we use to teach our client rendering behaviours must include
functionality that overrides other behaviours defined earlier. This may
include overriding the rendering of specific data types, either always, or
only when certain conditions assert themselves. As a result of this
approach, our client starts with the baseline ability to render any kind of
linked data, but also becomes able to render a user interface purpose-built
for our application's domain-specific needs.

6) The world of linked data offers many, many vocabularies. It would be a
lot of work to try and teach our client to render every different entity
type in a user-friendly way. A possible solution is to come up with a
generic TBox vocabulary that gives us the tools to advertise a suggested
layout for a given entity type. Such a layout could include positioning
hints, sizing hints, etc. but would exclude anything to do with theming,
i.e. colours, fonts, etc. Furthermore, because a client could override a
given layout however it sees fit, it would mean we could publish our data
to the rest of the world and have them lay it out in an acceptable way. We
could still teach our client case-specific overrides when we have custom
needs. If there is already a vocabulary that caters for my idea, let me
know and I'll take a look.

7) Teaching a layout to the client could be done in either of two ways (and
these don't have to be mutually-exclusive.) First, you could simply
preconfigure the client to incorporate the layout vocabularies for, say,
schema.org, productontology.org, and your own domain-specific entities.
Second, it could be possible for linked data to advertise the location of a
layout vocabulary that supports it. Such metadata could easily be ignored
if not wanted.

This solution would be applicable to Hydra as it could be used to help us
advertise to a client a way to lay out the Hydra functionality so that it
can be used easily by other people without them wondering why we neglected
the user experience. At the very least, it would be useful to be able to
publish hints, e.g. "These two actions are primary calls to action." "These
three are data maintenance options and should be deprioritised." "This link
is important and should be loaded immediately." "This data should be
available but not displayed." You get the idea.

On a tangential note, one thing I'm still not clear on is how I should
treat the advertisement of hypermedia controls with respect to a given
resource and the "purity" of that data, given that cacheability
requirements may conflict with the dynamic nature of the hypermedia
controls on offer and the context in which they're offered.

Any comments, questions and/or criticisms are most welcome. Also, take a
look at a link Markus posted earlier regarding JSON-LD and web components,
as there are ideas there that may be relevant to a client such as what I
envisaged above.
http://updates.html5rocks.com/2015/03/creating-semantic-sites-with-web-components-and-jsonld

cheers,
Nathan Ridley


-- 
*Google*: axefrog@gmail.com
*Skype*: axefrog
*Twitter*: @NathanRidley
*Website*: http://axefrog.com / http://nathanridley.com
*Phone: *+61 (0) 475 072789