[whatwg] Text APIs on <canvas>

On Tue, 06 May 2008 05:10:41 +0200, Ian Hickson <ian at hixie.ch> wrote:

> I have introduced the following APIs:
>
>    context.font
>    context.textAlign
>    context.textBaseline
>    context.fillText()
>    context.strokeText()

strokeText more or less requires a scalable font, which is not available  
in all fonts (and frequently not in mobile phones). I cannot see any  
comments on what should happen when the specified font is not capable of  
generating a stroke. I guess doing a fill instead is the only sensible  
thing to do (as the author has no way of knowing which fonts are available  
and/or scalable and it's bad if text disappears).

Stroking text is complicated as it is generally less common to have  
support for it in the font library. I would prefer to see stroking removed  
 from the spec as it will not work well with all fonts/devices.

//Tim

>    context.measureText()
>
> They are defined here:
>
>    http://www.whatwg.org/specs/web-apps/current-work/#text
>
> I haven't provided a way to render text to or along a path, nor a way to
> do vertical text, nor a way to measure anything but the nominal layout
> width of text (e.g. there's no way to measure bounding boxes or get
> baseline metrics). I also haven't provided a way to render document
> fragments straight to a canvas.
>
>
> Here is some of the feedback on canvas related to text. I have snipped  
> the
> feedback that wasn't constructive or didn't provide anything beyond
> requests for features without use cases and rationale, or that just
> bickered back and forth without making progress. If I snipped something
> that you think I should have responded to, please let me know. (I did  
> take
> everything into account, even the bits that I haven't included below.)
>
> I haven't made many comments below, because I didn't really have much to
> say. There were a lot of proposals and requests, but at the end of the  
> day
> there didn't seem to be strong arguments in favour of some things or  
> other
> things, so I haven't tried to defend the proposed API I put in the spec
> relative to the other APIs that were proposed. (I did take the various
> proposals that were made into account, and tried to adopt the best parts
> of each one.)
>
> If you have further feedback, please send comments to the list, as usual.
>
>
> On Thu, 18 May 2006, Lachlan Hunt wrote:
>>
>> [...] text can often be an important part of an image. Consider a graph
>> or chart generated from a table of data elsewhere in the page, it's
>> important to correctly label the axis and for such labels to remain in
>> the image when it's exported to PNG (or any other image format).
>
> Done.
>
>
> On Tue, 6 Jun 2006, Martin Atkins wrote:
>>
>> You can't really implement drawString as a single call, since there's no
>> guarantee that the font you requested will be available so you need to
>> be ready to deal with that eventuality.
>
> The API as designed can handle this if you pass it a maxWidth argument.
> The 'font' attribute can take multiple fonts, as in CSS.
>
>
>> A process which might work is:
>>
>> * Make a request to prepare some text with an ordered list of preferred
>> fonts, CSS-style.
>>
>> * Get back some kind of descriptor which allows you to discover various
>> characteristics of the dimensions of the text as it will be rendered.
>>
>> * At some later point, ask the canvas to "draw" the descriptor at a
>> given location.
>>
>> This allows for the browser to substitute other fonts while allowing the
>> canvas-based app to adapt the display to the characteristics of the
>> selected font.
>
> The API vaguely does something like this.
>
>
> On Thu, 3 Nov 2005, James Graham wrote:
>>
>> Allowing text in <canvas> has serious accessibility problems. Presumably
>> such text would not be resizable and encouraging authors to use
>> non-resizable text in their web-apps is not a good idea. I guess there
>> would also be (separate) issues with fonts; one assumes font
>> substitution on a bitmap drawing would be more problematic than font
>> substitution where the text is the main content.
>
> On Thu, 3 Nov 2005, Anne van Kesteren wrote:
>>
>> Another problem is that, since html:canvas does not have a DOM, the
>> actual content is not really accessible. However, I have seen people
>> having to use images with the alphabet inside html:canvas to get simple
>> text effects. Making that easier might be nice. (But addressing fonts,
>> line-height, etc. might be difficult.)
>
> On Tue, 8 Nov 2005, Vladimir Vukicevic wrote:
>>
>> There are accessibility problems for sure; however, accessibility is not
>> something that can be forced onto content authors.  They have to design
>> for accessibility, it won't happen for them.  If they don't, being able
>> to draw text into canvas is a relatively minor issue.
>>
>> If a DOM or if arbitrary high-resolution scaling of already drawn
>> content is desired, SVG's the best option.  However, there are a lot of
>> use cases where text in <canvas> is highly desirable.  <canvas> is
>> already going down a pseudo-CSS path for specifying colors and other
>> styles; I think it makes sense to extend text in a similar way, perhaps
>> by using a textStyle that refers to a CSS property (by ID? by class?
>> somehow), and then letting you render text strings into the canvas.
>
> On Tue, 17 Oct 2006, Charles Iliya Krempeaux wrote:
>>
>> I believe that some people's reason for not wanting to add it was
>> because of Accessibility concerns.
>>
>> Although normal text in a webpage... or even a text image (with the
>> "alt" attribute filled in properly) could be "read" by a person with
>> disabilities, text embedded in the canvas element could not.
>>
>> Perhaps people need to think about how to add Accessibility to the
>> canvas while allowing a "drawString" procedure.
>
> On Tue, 17 Oct 2006, Mathieu Henri wrote:
>>
>> But if a user agent do no support Canvas, it MUST fallback to the
>> descendant tags of the Canvas. There you have the accessibility mean.
>>
>> It would easy to use DOM methods to grab the texts to display in the
>> fallback content of the Canvas tag. Provided that the fallback markup is
>> exposed to JavaScript in the user agents that support Canvas.
>
> On Tue, 17 Oct 2006, James Graham wrote:
>>
>> So how would I, using my canvas-supporting browser, make the text larger
>> when the page author decided to make it illegibly (for me) small.
>>
>> The idea that accessibility features are for people who need special
>> browsers is a myth.
>>
>> (actually the whole text-on-a-drawing thing is a huge can of worms. What
>> if the browser has different fonts to the author? Font substitution can
>> lead to a font being used with the wrong width, leading to text
>> overlapping the drawing. (Browser implementations of) SVG have this
>> problem. The same issues can occur if the text can be rescaled but not
>> he graphic elements. The obvious solution to this - rescaling text and
>> graphic in lockstep - is an improvement but can lead to huge amounts of
>> unnecessary scrolling if the unscaled figure has a width close to the
>> screen width since there's no sane way to reflow a figure.)
>
> On Tue, 17 Oct 2006, Alfonso Baqueiro wrote:
>>
>> Well a drawString method in canvas can be used for drawing the axis
>> labels or values on a dinamic javascript graphic, but is non sense an
>> aural reader read it, theres no way (yet) to read an image to a blinded
>> people, there are cases where is imposible the accessibility, is
>> imposible for a blind to play video games, and accessibility dont stop
>> the video games creation. In the case of images or the canvas the alt
>> attribute could do the job.
>
> Since people are doing text already using bitmap fonts, not including  
> this
> API isn't helping accessibility. So this seems like a moot point. The
> authors who are responsible will provide accessible versions of the
> content regardless of whether the canvas contains text or not.
>
>
> On Wed, 17 May 2006, Gervase Markham wrote:
>>
>> This came up in Vlad's presentation at XTech today. My suggestion was to
>> implement drawDiv() or drawElement(), in addition to drawWindow(). That
>> way, you could fix the potentially hairy text rendering problem with one
>> simple API call, which could be made using the backup accessible content
>> which you had placed inside the <canvas> tag anyway. Everyone wins.
>>
>> <canvas id="foo">
>>   <table>
>>     <tr>
>>       <th id="ds1title">Data Set 1</th>
>>       <th id="ds2title">Data Set 2</th>
>>      ...
>>
>> foo.drawElement(document.getElementById("ds1title"), 45, 60);
>
> On Sat, 20 May 2006, Peter Hall wrote:
>>
>> I really like that idea. But I think you'd put the method on
>> CanvasRenderingContext2D instead of canvas itself?
>>
>> foo.getContext("2d").drawElement(document.getElementById("ds1title"),  
>> 45,
>> 60);
>>
>> In the latest Flash player, you can do something similar with
>> BitmapData.draw(), but there are additional arguments for matrix
>> transform, blending mode, clip rectangle and smoothing options. I'd
>> certainly want to at least be able to scale and clip what I'm drawing.
>
> On Wed, 18 Oct 2006, Gervase Markham wrote:
>>
>> I've suggested this in the past as a solution to this problem: why not
>> have a drawElement(elem) parameter?
>>
>> That way, you could build an accessible, readable version of the content
>> inside the <canvas> tag, as alternative content, and copy labels or
>> anything else into the <canvas> itself with drawElement(label). So the
>> same content serves both as the accessible version and the used version.
>>
>> This would give us great flexibility, because the text you do have is
>> controlled with all the power of the existing CSS and browser font
>> model, obviating the need for font controls or font objects on the
>> <canvas> API - which would inevitably be not as good as the CSS ones.
>> And if browsers acquire downloadable font support, so does canvas.
>>
>> I would speculate wildly that it might even be easy to implement too.
>> After all, I'm sure browsers have the ability to render the contents of
>> a <div> tag to a drawing buffer...
>
> On Wed, 18 Oct 2006, Mathieu Henri wrote:
>>
>> Indeed, adding a something like the toDataURL( [MIMEType] ) method on
>> the HTMLelement object would make our life so much easier and open a
>> whole new range of possibilities.
>
> On Wed, 18 Oct 2006, Stefan Haustein wrote:
>>
>> I think drawElement(elem) opens up a whole new can of worms:
>>
>> - how would an application determine the size of the text box?
>>
>> - where is the baseline position, needed for exact axis label
>> positioning?
>>
>> - there are probably issues with dynamically calculated text values
>>
>> - code with lots of cross references to elements will be difficult to
>> read
>>
>> - it needs to be specified whether css properties are inherited from the
>> parent element of "elem".
>>
>> - how much horizontal/vertical space is drawElement permitted to use for
>> rendering?
>>
>> - the implementation in actual browsers may be more complex than it
>> seems because of problems with internal data structures for rendering
>> hints and implicitly introducing the ability to render the same element
>> twice.
>>
>> - what happens with contained plugins, canvas elements,
>> self-references... all this stuff needs to be well-defined
>>
>> Moreover, drawElement() would not solve the drawText problem for
>> non-browser environments such as Rhino.
>
> On Wed, 18 Oct 2006, Gervase Markham wrote:
>>
>> The answer to all of these things is that the browser renders all the
>> elements in the page as it would if the <canvas> were not supported and
>> the alternate content were being used. It then basically screenshots the
>> area corresponding to the element (yes, I know this needs careful
>> definition) and draws that into the canvas.
>>
>> Like I said, we want to leverage the browser's deep and complex
>> knowledge of text rendering as much as possible, and just take the
>> resulting pixel output as it would be shown to the user.
>>
>> I know it's easy to state and there are edge cases. But we could put
>> limits on it like e.g. no plugins, no <object>, and still have something
>> very useful for rendering text.
>>
>> I mean, all you really need to support is stuff like:
>>
>> <style>
>>   .box {
>>     width: 300px;
>>     font-style: cursive;
>>     font-weight: bold;
>>     text-transform: uppercase;
>>    }
>> </style>
>>
>> <div class="box">Some text to play with</div>
>
> I haven't provided this because it is significantly more complex than  
> just
> drawing text, but it is something we should probably look at in a future
> version of the canvas API.
>
>
> On Mon, 23 Oct 2006, David Hyatt wrote:
>>
>> Rather than specifying stylistic information explicitly (via a font
>> object), I'd use a special parenthetical pseudo-element. thus allowing
>> the author to specify the style as for any other element on a page....
>> something like this...
>>
>> canvas::canvas-text(barchart)
>> {
>> 	font-size: 8px;
>> 	font-family: Monaco, monospace;
>> }
>>
>> and then the API would be something like:
>>
>> drawText(y-coord of baseline, "barchart", myText)
>>
>> and letter-spacing/word-spacing would work, small-caps would work,  
>> text-shadow
>> would work, etc. etc.
>
> It seems that at least the font-size would be something you'd want to
> control in the code rendering the text, though I could see an argument  
> for
> making the other aspects of fonts accessible from CSS... this seems a bit
> over-engineered though. I haven't provided this for now.
>
>
>> fitTextToPath might be an interesting API too.
>
> Indeed. I have noted something along these lines as a possible area to
> extend the API into in a future version.
>
>
> On Tue, 24 Oct 2006, Gervase Markham wrote:
>>
>> At the risk of overcomplicating, vertical text is a common requirement
>> for graphs and charts. If this is simpler to implement than the
>> "arbitrary text rotation" case, is it worth having a way of saying the
>> baseline is vertical rather than horizontal? Or would this be a job for
>> fitTextToPath()?
>
> I haven't done vertical text because CSS doesn't yet define how to render
> it. However, there is a drawVerticalText() method commented out in the
> spec that is intended to satisfy this which can be uncommented out when
> CSS defines how to do this.
>
>
> On Wed, 25 Oct 2006, Charles Iliya Krempeaux wrote:
>>
>> One thing that comes to mind is "Ruby" in the Japanese language.
>
> On Tue, 14 Nov 2006, Martin Atkins wrote:
>>
>> However, it'd also be useful to be able to render formatted blocks of
>> text, like this:
>>
>> void drawString(in float x1, in float y1, in float x2, in float y2,
>>                 in DOMString text)
>>
>> ...which wrap text and would make a textAlign of "justify" meaningful.
>> Suddenly a bunch more CSS properties become attractive (margins and
>> letter spacing, for example) but I think line-height alone would do for
>> now.
>>
>> Rendering formatted blocks of text seems to me to be such a common thing
>> that the canvas API should support it natively.
>>
>> This does, of course, require more functions on TextStyle to measure the
>> extent of a block of text. This could probably be confined to just the
>> following:
>>
>> float getBlockHeight(in DOMString s, float width)
>>
>> I think that a good use-case for this would be replacing regular
>> document elements with canvases, either using bare script or using XBL.
>> You point out in your document that drawElement(e) is tricky, but for
>> limited cases it would be nice to be able to obtain the computed style
>> of the canvas element to use for rendering, so that the text can match
>> the style of the surrounding document.
>>
>> This can be as simple as one method on the canvas context to fetch the
>> computed style, which would presumably return null in a non-browser
>> implementation of the API.
>>
>> CanvasTextStyle getInheritedStyle();
>>
>> For example, this could be used to create some kind of visual effect
>> (fancy headings?) or to render a graphical element from some XML
>> namespace that isn't supported in browsers yet.
>
> On Tue, 14 Nov 2006, Mathieu HENRI wrote:
>>
>> I still think by introducing the drawString() method into Canvas we are
>> opening the same can of worms that was open in SVG.
>>
>> If we go that way we will need a drawParagraph() method to draw multi
>> line strings or blocks of text with wrapping and a bounding width. We
>> also need to be able to stylize the text, i.e. changing the font-weight
>> / color / font-style ... of any word.
>>
>> The list goes on and on ... and HTML and CSS already cover it all.
>>
>> The HTMLElement.drawElement() method should be no problem to implement
>> since userAgents already do render HTMLElements.
>>
>> Having it return an ImageData object will make it insanely simple to
>> manipulate in Canvas. The text elements/contents can easily be in the
>> fall back content of the Canvas tag thus keeping it accessible. Getting
>> the bounding box of an HTMLElement is no problem either in JavaScript.
>> And applying gradients and patterns can be done using a fillRect() with
>> the appropriate globalCompositeOperation.
>>
>> Everything (almost) is there. Let's not re-invent a square wheel.
>
> Ruby, multiline layout, and other such complexities aren't handled by  
> this
> API. Full text layout like this would be better handled using an API
> similar to the drawElement() proposal. I've added a note to this effect  
> in
> the spec.
>
>
> On Wed, 18 Oct 2006, Stefan Haustein wrote:
>>
>> However, I think getAscent() is not sufficient, we should also add
>> getLeading() and getDescent():
>>
>> This would allow us to determine the total line height
>> (leading+ascent+descent) and the baseline position (leading+ascent):
>>
>> http://docs.rinet.ru/UJ11/art/17/17fig08n.jpg
>>
>> The total line height is important since it constitutes the minimum
>> vertical distance from baseline to baseline (Accents on uppercase
>> letters will be placed in the leading area, and they must not overlap
>> with the descent from the previous line).
>>
>> The baseline position is important for text alignment when using
>> different fonts/styles in a single line.
>>
>> Of course we, could add getBaselinePosition() and getHeight() instead of
>> some of the other methods, but including the three most basic values
>> seems to be a consistent approach (simple to remember) and all other
>> values can be calculated by summing them up somehow (no differences
>> needed).
>
> On Tue, 24 Oct 2006, David Hyatt wrote:
>>
>> I'm very reluctant to expose font metrics and information (yet).  I
>> think once you start getting into specifying fonts, you open up a can of
>> worms that would make this sort of API addition a lot harder.
>
> On Wed, 25 Oct 2006, Stefan Haustein wrote:
>>
>> I think it is very important to be able to determine the rendered size
>> of the text. Otherwise, there is no reliable way to make sure things do
>> not overlap. Also, some kinds of applications (flash-like effects,
>> labels expressing weight or distance, WYSIWYG text editors) may require
>> variable font sizes or styles.
>>
>> What do you think about
>>
>> context.textStyle = "barchart"; // style by reference
>>
>> context.textStyle = {  // set style directly
>>  "font-size": "8px",
>>  "font-family": "Monaco, monospace"
>> }
>>
>> context.drawText(x,y,string); context.getFontAscent();
>> context.getFontDescent();
>> context.getFontLeading();
>> context.getTextWidth(string);
>
> On Wed, 25 Oct 2006, Stefan Haustein wrote:
>>
>> However, I would still prefer an API design that keeps it simple to add
>> the metric query methods later. Perhaps they could be included and
>> simply return null if the requested information is not available (e.g.
>> for a dumb EPS render target?). Also, if we do not have access to font
>> metrics, I think we need an additional parameter for drawText() that
>> specifies the alignment of the text relative to the reference point, as
>> illustrated in the image below:
>>
>> http://www.developer.com/img/articles/2002/12/26/03fig12.jpg
>
> On Wed, 25 Oct 2006, Stefan Haustein wrote:
>>
>> for bulk text, I think Canvas is the wrong place. However, if we get the
>> metrics query methods, it would theoretically be possible to implement a
>> renderer for any kind of specific purpose.
>>
>> To display ruby annotations for labels (maps, some kind of educational
>> flash-like application, you name it...), one could draw the annotation
>> in a small font, aligned to the bottom and draw the base text aligned to
>> the top (given the propsed alignment parameter is added):
>>
>> canvas.textStyle = {"font-size": "5.5pt"};
>> canvas.drawString(100, 100, "ko ba ya shi", BOTTOM|HCENTER);
>> canvas.textStyle = {"font-size": "12pt"};
>> canvas.drawString(100, 100, "KO HAYASHI", TOP|HCENTER);
>
> On Wed, 25 Oct 2006, James Graham wrote:
>>
>> I still believe that any api that does not allow the author to query the
>> size of the text bounding-box[1], i.e. the overall width and height of
>> the box enclosing the text fragment, is useless for many of the use
>> cases presented so far e.g. figure labelling as, without this
>> information, it is impossible to ensure that adjacent text fragments do
>> not overlap.
>>
>> [1] I mean that in a non-technical sense
>
> On Wed, 25 Oct 2006, David Hyatt wrote:
>>
>> Yeah I see what you mean.  In addition to a drawText you probably want
>> something like a metricsForText API that would tell you the extent of
>> the string and the font metrics (line height, ascent, descent,
>> baseline).
>
> On Thu, 26 Oct 2006, Gervase Markham wrote:
>>
>> It's the responsibility of the page designer to make sure the text they
>> are measuring is styled the same as the text they want the metrics of.
>>
>> According to the current API suggestion, the styling of the text in a
>> canvas will not be affected by the style of parent elements, but will be
>> defined by the style of the pseudo element name passed to the drawText
>> function.
>
> The API has a measureText() method, which right now just returns the
> layout width of the text, but which could in future be extended to  
> support
> any other metrics we want to expose (e.g. the bounding box, the distance
> from the top of the em square to the top of the bounding box, the  
> relative
> positions of the baselines, etc).
>
>
> On Mon, 13 Nov 2006, Stefan Haustein wrote:
>>
>> I have tried to sum up the requirements for
>> CanvasRenderingContext2D.drawString() at
>> http://rhino-canvas.sf.net/www/drawstring.html
>>
>> The page contains an API proposal based on the Font/TextStyle object
>> approach that meets all those requirements, and also some motivation why
>> I have implemented this approach and not one of the alternatives
>> discussed here.
>
> Thanks, this was quite useful.
>
> Cheers,

Received on Wednesday, 7 May 2008 06:42:35 UTC