[Bug 3919] Section 3.1.8 tokens vs words

http://www.w3.org/Bugs/Public/show_bug.cgi?id=3919

           Summary: Section 3.1.8 tokens vs words
           Product: XPath / XQuery / XSLT
           Version: Working drafts
          Platform: PC
        OS/Version: Windows XP
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Full Text
        AssignedTo: jim.melton@acm.org
        ReportedBy: mrys@microsoft.com
         QAContact: public-qt-comments@w3.org


use the term “token” and not “words” in grammar. There is a difference such as
the word "Donaudampfschiffartskapitaensmuetzen" can be tokenized and be
searched with

“Dampf” && “muetze” distance at most 5 tokens

It would read strange with word.