RE: [DPUB-ANNOTATION-UC] 2.1.3 general observation about language identification [I18N-ISSUE-458]

When I say “document structure”, I mean the document describing the annotation model (which may include or refer externally to the annotation as a non-textual values). I was very careful in my previous message to use the term “natural language text field” so that this didn’t become confused with other metadata usages (which might include things like the language preferences of the target audience, for example).

If a document format provides a natural language text field, then that field needs to provide direction and language metadata for that field. Ideally, a spanning element (such as HTML’s <span>) would allow markup within the field as well.

Non-textual or external annotations or data are also important. A sound or video file (a “resource”), for example, is not a natural language text field. In that case, language information about the resource might be interesting, but that’s not the same thing as document-defined natural language text. I wouldn’t argue against making such metadata available, of course. But I think that is a separate set of requirements than we’re discussing here.

The “annotation model” will take the form of a “document” with some sort of format. Where the model can contain natural language text, that text needs to have direction and language. This is different from “creation time” or other application specific values and may be different from (for example) target audience language metadata (similar to the old Content-Language metadata field), which DPUB might define separately from any specific field in the document.

That make sense?

Addison

From: Felix Sasaki [mailto:fsasaki@w3.org]
Sent: Wednesday, October 14, 2015 11:46 AM
To: Phillips, Addison
Cc: Robert Sanderson; public-digipub@w3.org; public-i18n-core@w3.org; Web Annotation
Subject: Re: [DPUB-ANNOTATION-UC] 2.1.3 general observation about language identification [I18N-ISSUE-458]

Hi Addison and all,

for apologies to Addison and my i18n WG colleagues for not following this earlier closely, and a question below.

Am 11.10.2015 um 19:46 schrieb Phillips, Addison <addison@lab126.com<mailto:addison@lab126.com>>:

Hi Rob,

Thanks for the reply.

Note that references to language tags should refer to BCP 47 rather than to (one of) the underlying RFCs such as 5646. BCP 47 is a stable reference.

I don’t agree that language and direction metadata should be lumped with other types of metadata. The best practice for natural language text fields is to provide for language and base direction metadata in the document structure


The annotation model is not necessarily to be applied to documents, see below an example where the target of the annotation is an image


{

  "@id": "http://example.org/anno1",

  "@type": "oa:Annotation",

  "body": {

    "@id": "http://example.org/body1",

    "@type": "dctypes:Sound",

    "format": "audio/mpeg",

    "language": "en"

  },

  "target": {

    "@id": "http://example.org/target1",

    "@type": "dctypes:Image",

    "format": "image/jpeg"

  }

}

So I am wondering what you refer to then saying „provide for language and base direction metadata in the *document* structure“. Happy to discuss this also tomorrow during the i18n WG call.

Best,

Felix



at the field level, which is why our comment calls it out. That’s because each field may be in a different language or have a different base direction. Note that this is consistent with e.g. HTML5 and with a number of ebook formats.

The language may also be a separate field at the document level and this comment isn’t about the kinds of metadata (including language) as you list in your reply.

(Note that the I18N WG is publishing a FPWD of “best practices for specification authors” this coming week that details some of the items above and may be helpful to your WG. See http://w3c.github.io/bp-i18n-specdev/#lang_resource)

Thanks,

Addison

From: Robert Sanderson [mailto:azaroth42@gmail.com]
Sent: Sunday, October 11, 2015 2:24 AM
To: Phillips, Addison
Cc: public-digipub@w3.org<mailto:public-digipub@w3.org>; public-i18n-core@w3.org<mailto:public-i18n-core@w3.org>; Web Annotation
Subject: Re: [DPUB-ANNOTATION-UC] 2.1.3 general observation about language identification [I18N-ISSUE-458]


Thanks again for the comments.

I agree that language metadata is important and that the set of use cases does not specifically include any metadata about the body or target resources.  That was somewhat intentional, so as to avoid trying to list out all of the possible descriptive features for resources, such as creator, creation time, language, file format, license or other rights statements, intended audience and so forth.  These are listed for the annotation itself, as the primary resource of interest.

In the Web Annotation data model, the language is explicitly included along with format and general class of the resource [1].  In the upcoming WD (next week [2]), we also add creator and creation time.  For language we refer to RFC 5646 as the value of the language property.  Is that sufficient to cover the requirements?

Many thanks,

Rob

[1] http://www.w3.org/TR/annotation-model/#body-and-target-metadata

[2] http://azaroth42.github.io/web-annotation/model/wd/index-renamed.html




On Sat, Oct 10, 2015 at 1:19 PM, Phillips, Addison <addison@lab126.com<mailto:addison@lab126.com>> wrote:
[1] 2.1.3 general observation about language identification
Description:
    http://www.w3.org/TR/dpub-annotation-uc/


    2.1.3 general observation about language identification

    Tags and annotations generally use natural language tokens (such as words). While Unicode allows text to be stored, passed, and processed without regard for the specific language, it is the case that strings can benefit from language metadata for character shaping, spell-checking, font selection and more. In additional, directionality information is usually desired.




--
Rob Sanderson
Information Standards Advocate
Digital Library Systems and Services
Stanford, CA 94305