Introduction

I am a software engineer working at Outspring, a startup company developing an email client. I previously worked at Netscape on the Mail & News client, and later on the Mozilla CSS engine -- which might explain my fondness for internet standards. We regret we cannot be present at the workshop in Paris so I'll present our position in this paper under 3 chapters:

• Mission: what the WG is about
• Views: what we might want to see in future emails
• Needs: some of the problems we have to solve to get there

Some of the points exposed here are excerpts of the posts I sent to the mailing list, and the underlying theme is my opposition to any broad curtailment of HTML, CSS and other standards in emails.

I - Mission

Most importantly, we should define the mission of the "HTML in Email" workgroup. Where does it fit between the much larger HTML workgroup, the security Task Forces, and the industry's priorities and practices?

There is a temptation to address all kinds of problems we personally face when handling our email. The focus, however, should not be about privacy and security, nor spam and guaranteed delivery. There are other more advanced efforts that have been underway for years to address these questions and the "HTML in Email" workgroup is probably not the best venue to take them all over again. We need to keep these points in mind but it doesn't mean that we have to solve them.

Another temptation is to see a subset of HTML as a panacea for security and interoperability issues. By some aspects, of course, it is -- if all the mail clients were limited to plain-text and the web browsers to HTML 1.0, we wouldn't have any virus infections and rendering discrepancies -- but this is a very wrong way to approach the problem. Before their recent rebirth and convergence, web browsers had been plagued for years with security holes and viruses, they had a poor support of standards, they were not compatible with each other, and yet it did not prevent the public from benefiting enormously from the technological advances in the domain. These benefits have derived in a large part from applications that neither the authors of the standards nor the browser developers had imagined, but they were ultimately made possible because authors had enabled them. This reason alone is sufficient for me to say a big "NO!" to any effort to squash email into an HTML subset. [1]

In my opinion, the mission of the workgroup should not be to trim features in order to reach a lowest common denominator or a so-called "safe subset" (which would be illustrating yet again the human being's natural propensity to increase safety by reducing options and freedoms). We don't have to plan for those who spam, snoop or infect, they will be taken care of by much better equipped forces. On the opposite, we should work on what needs to be done to open HTML and enable a wider, fuller, support of internet standards in emails. What will come out of it? Probably many applications we haven't thought of yet... and that sounds much more exciting to me.

II - Views

A good place to start when trying to figure what we want to do might be to list the things we regret not being able to do - in a word, address the deficiencies.

The technology used for email is lagging years behind what has been deployed for the web. Most of the emails nowadays are still as appealing as a Telex, and the responsibility falls for the most part on tech-savvy people like us who for a long time resisted HTML-Mail that they perceived as a big waste of resources. Another reason might be that new technologies on the web increase the lucrative number of "eyeballs" while nicer, more human-friendly, emails don't bring in any money. The irony in this sorry state of affairs is that email is often listed as the main reason for companies and individuals to invest in computer equipment, and a lot of people spend more time online using their email program than any other program, but regrettably the most used program is also the one that has the least evolved in the past 20 years.

The consequences of this delay in adopting newer technologies are difficult to measure. It's impossible to tell what the web would have been like without Javascript, without CSS, without forms. Same goes for email. Many aspects of email handling are affected by these limitations, such as personalization, interaction with other clients, handling of forms, web integration and organization - but hopefully we can do something about it.

1 - Personalization & Customization:

• Users can't personalize the emails they send.
• Users can't generate professional-looking documents.

There is no reason why users are able to publish their blogs using any template they like to reflect their style, taste, moods and opinions on a public forum, and yet are still constrained to plain text emails when communicating privately with the persons who are closest to them. If mail clients could import and share standard-based HTML+CSS templates, the authors of the thousands of stylesheets that are used to style blog pages would be able to create email templates that match the look of their blogs. An email would then look like a little blog entry written just for you. And personalization does matter: Blogger and MySpace would have never had the success they had if everybody was still using a plain-text browser.

Aside from reflecting the personality of the sender, a template can also illustrate the content of the email. Invitations and announcements are sometimes lost or left unanswered because the bland aspect of the email did not leave enough of an impression on the recipient's mind at the time he received it.

A lot of companies still use paper memos for communications from management and letterhead fax or snail-mail for exchanges with their customers, in part because email looks so bad (not professional enough) and can be so easily lost or ignored (they all look the same).

2 - Interaction between Clients:

• Users can't conveniently reply to nicely formatted email.

Sending rich HTML emails is one thing. Responding to them presents another challenge: how can we identify the user-generated content out of the rest of the email (header, footer, static texts and images)? For now, recipients have to copy the part of text they are responding to and paste it as quotation into their response. An attempt was made 10 years ago to solve the problem, as well as other things, but it wasn't widely adopted by the industry [2].

3 - Handling of Forms:

• Users can't send short forms instead of verbose text mails.
• Users can't receive forms, fill them out and send them back.

Almost 20 years ago, some early mail clients were capable of handling proprietary forms in order to replace office memos and post-it notes. For instance, a manager calling for a meeting would just have to fill in three fields (time, location and subject), while an assistant notifying his boss of a phone call would just have to fill in one field (the name of the caller) and mark one checkbox (either "will call back" or "please call back"). This functionality has disappeared from mainstream mail clients and as a result people waste time typing verbose plain text messages with greetings and sign-offs even for the most trivial tasks, such as sending a reminder or sharing a URL.

If users can't send forms, they can't respond to them either. Oftentimes when receiving a question that requires a yes-or-no answer, users are not comfortable responding with a single "No" because it sounds a bit rude, so they construct a plain text message with a complete sentence in their reply. If the same question came with a response form, they wouldn't hesitate to simply check the "No" checkbox and click "Send" because the form would fulfill the purpose in a completely impersonal manner.

4 - Web Integration:

• Users can't keep copies of blog comments and other forms they fill out on the web.

Another consequence of the technological gap between email and the web is the lack of interaction between the two. People archive years of emails, yet whenever they fill out a web form or add a comment to a blog, they can't keep a trace of it. A better integration could allow users to save their submissions as messages in their Outbox or Sent folder.

5 - Organization:

• Users can't see at a glance what an email is about
• Users can't easily sort their emails

As proof of the neglect they have been subject to for so long, emails now have an interesting characteristic: they are amongst the only data containers left in modern computing that don't tell you anything about the data they contain except for their name (the mail subject). Look at any OS: pictures and movies have icons showing a miniature version of their content, text files have previews, music files have album covers, bookmarks and web sites have Favicons, and everywhere else files have icons corresponding to their type or extension. I'm a programmer and my development tool (XCode) has a UI that's very similar to an emailer (3 panes: folders on the left, files on top, viewer/editor at the bottom) but even there, icons are used to differentiate the elements. All that to say: email programs have an interface that's even less approachable than software development tools.

Email programs offer several ways to differentiate or highlight emails (flags, tags, labels, colors) but they are all on the receiving end. They put the burden of categorizing an email onto its recipients while it is the sender who knows best what's inside. Unfortunately the only marker the sender can use is the 'x-priority' header that hardly anybody uses. An email should have a type, an icon, and even some flags and fields that could be set by the sender (such as 'action required' and 'expiration date').

These extensions are not directly related to HTML-in-Email (they would likely be implemented as RFC822 headers or as a microformat in a MIME part) but they would help a lot in the deployment and interoperability between clients of HTML forms and templates. When a form is used to send a quick reminder, both the sender and the recipient have an interest in making it immediately visible that an action is required before a certain date, without opening the email itself, even though the recipient doesn't use the same emailer as the sender or has never seen before the form in question. I believe the public would be much more likely to adopt forms if there was a way to differentiate them from the other, more verbose, emails. After all, in the physical world, people don't handle their utility bills or the post-it notes that someone left on their monitor the same way as they handle their family letters or a friend's postcard.

III - Needs:

It would not be a major task to enable a system of standards-based templates across desktop clients - it could be as simple as an agreement on a single HTML attribute and its possible values. It also requires the implementation of the 'contentEditable' attribute, but this is already a done deal on all major browsers. What's left - maybe the most difficult - is to persuade the authors of web-based email clients to better embrace the standards.

Here are the 5 points that we might want to address, although only the first 3 would be necessary for a widespread adoption.

1 - Full HTML and CSS:

A large majority of the current desktop and web-based email clients are capable of supporting, or could easily support, full-blown HTML and CSS. Desktop clients don't trim the HTML and they support standards to the same level as their embedded browser engine (IE, Mozilla or Safari). Web-based clients, however, all filter out the HTML and the CSS to some degree [3][4][5][6]. None supports <link> elements or CSS positioning, but Gmail is particularly restrictive as it doesn't even allow embedded stylesheets (all style must be inline).

A task of the workgroup could be to evangelize the authors of web-based clients and convince them that there is no need to impose such restrictions. One of their arguments is to avoid phishing expeditions by imitating the webmail client's interface, but this could be avoided through less detrimental ways [7].

2 - Identification of user-generated content:

When a recipient responds to an HTML email based on a template, the mail program must be able to identify which parts of the received mail were generated by the sender and which parts belong to the static template. A solution could be to set a particular attribute on the user-generated content, but it needs to be agreed upon by the authors of email clients.

Another problem is to tag the quoted-content when composing a reply. This was definitely the topic of an effort mentioned earlier but it did not get much traction within the industry [2]. Email clients usually embed the quoted-content within a BLOCKQUOTE element but they differ in the way they tag it and display it. Yahoo and Gmail give it a certain class (class="replbq" or class="gmail_quote") and force upon it a STYLE attribute. Desktop clients behave better, tagging it with an attribute (type="cite") and relying on their UA stylesheet for the display.

Here again, it would be helpful if the workgroup could talk Yahoo and Gmail into following the standards more closely.

3 - Identification of specific editable elements:

An email template contains several editable elements within a static chrome. Some of these elements can be expected in pretty much all the templates: a subject line, the date, a list of recipients, the body of the email, and a signature. When composing a new message, the email client can differentiate the editable elements from the rest of the template thanks to the 'contentEditable' attribute, but it also needs to identify the type of the elements themselves in order to fill them in automatically.

Here too, a solution could be to set a particular attribute on these elements. In fact, we can have a single attribute for specific editable elements (USERCONTENT="subject", USERCONTENT="body", etc) and any other unidentified user-generated content (USERCONTENT="unknow").

There are two immediate benefits in identifying these elements:
- First, when replying to an email, the program can pre-fill the fields correctly (for instance, only the body of the received mail should be part of the quoted content, not every single piece of user-generated content).

- Second, it is possible to get rid of the traditional Compose window and replace it with a real WYSIWYG editor. The separate "To", "CC" and "Subject" fields are not necessary anymore because they are part of the mail itself.

Another benefit of having a standard to identify fields within the mail is to enable the use of XML - in the long term to actually send the content, but in a shorter term to implement mail templates in a flexible way. A template could consist of an XML document describing the static but customizable fields within the chrome, associated to an XSLT stylesheet that generates the HTML code. This way a first generation of XSLT stylesheets could be designed to produce a layout based on HTML Tables that will be compatible with even the fairly restrictive webmail clients (Yahoo, Gmail, etc...), and later updated to make use of CSS as the clients allow it. The advantages? The customization of templates is very simplified (it's all in XML) and the XSLT stylesheets can be upgraded without requiring the users to re-enter the customized data all over again.

4 - Identification of the message type:

This point isn't strictly necessary but it would help solve the issues described above in "Organization". It involves the definition of a list of email types, plus some flags and fields, in order to better identify emails at the receiving end. Of course, it doesn't really fall under the realm of "HTML-in-Email" because it would be implemented as additional RFC822 headers or a microformat in a MIME part; but since the workgroup will hopefully gather the vendors of HTML-based email clients, it might be a good forum to start the discussion.

The list of email types should stay relatively generic and short enough, with categories such as Question, Report, Scheduled event, Invitation, Personal, Fun, Announcement, News, etc... The list of flags and fields should be limited too, with the examples mentioned above: "Action required", "Expiration date", etc...

5 - Scoped stylesheets:

Templates will come with their own stylesheets, and limiting the scope of these stylesheets will be crucial for web-based clients in order to avoid interferences between their chrome and the content of the emails (not so much so for desktop clients because their embedded browser view is entirely dedicated to displaying the mail). Anyhow, scoped stylesheets are being discussed in other workgroups so the involvement of the HTML-in-Email group might just be communicate our interest and push for a solution.

Conclusion

A lot has been written in recent years about Web 2.0: a lot of buzzwords for sure ("Social Software, Sharing, Collaboration, Convergence, Semantic, Usability"), but a lot of standards recommendations were produced, and a lot of code too. Wherever it leads, Web 2.0 is on its way, but ironically enough the biggest loser in this evolution is the premier "Social, Sharing and Collaboration" software: the emailer. The most widely used communication program is not even mentioned on the Web 2.0 mind-map [8] even though it is the only program everybody has access to that already offers (or could easily offer) a decent native standards-based editor to generate the user content that Web 2.0 relies upon. It's sad because it's a domain where plenty could be done. For instance, instead of sending some elaborate HTML and stylesheets as part of the mail, we could envision a system relying on external stylesheets stored on trusted servers (MySpace or Blogger could typically offer hundreds of styles). There is also much to explore with XML-based emails. Companies have more and more knowledge in emails that don't integrate very well with the rest of their infrastructure. If emails were nicely divided blocks of data, they could be displayed in XML apps within the emailer and their content could easily be archived and shared within a deparment or a company.

I hope this workgroup will not end up imposing restrictions on what email can do. It would sentence the public to either stick with the products we currently have on the market (which are more or less what we have been stuck with for 20 years already) or to face the emergence of disparate proprietary solutions (which will be coming). It would be quite unfortunate for a W3C workgroup to stifle innovation or let the market diverge. That would not be "Leading the World Wide Web to its full potential" [9], especially at a time when the majority of email clients (either web-based or embedding a browser engine) finally support the standards that would allow a new generation of programs to appear.