[Bug 6667] [FT] Stemming

http://www.w3.org/Bugs/Public/show_bug.cgi?id=6667





--- Comment #2 from Christian Gruen <christian.gruen@gmail.com>  2009-03-13 14:22:21 ---
Thank you Pat,

just get sure on query "ft-3.4.7-examples-q2.xq" - it performs the following
test:

[1] 'propagating few errors' ftcontains "propagation of errors"
     with stemming with stop words ("a", "the", "of")

If the terms "propagation" and "propagating" will not be stemmed, this query
should equal the following one:

[2] 'propagating few errors' ftcontains "propagation of errors"
     with stop words ("a", "the", "of")

The text token "few" will be ignored due to stopword removal, but I would still
expect the query to yield "false". Do you agree?

In contrast, the following query should yield true:

[3] 'propagate few errors' ftcontains "propagate of errors"
     with stop words ("a", "the", "of")


In my opinion, this query should be simplified in the specification as the
stemming option does not really contribute to explain the asymmetry in the stop
word semantics; what do you think about the following version?

_________________________________

3.4.7 Stop Word Option

[...]

The following expression returns true, because the document contains the phrase
"propagating few errors":

/books/book[@number="1"]//p ftcontains "propagating of errors"
  with stop words ("a", "the", "of") 

Note the asymmetry in the stop word semantics: the property of being a stop
word is only relevant to query terms, not to document terms. Hence, it is
irrelevant for the above-mentioned match whether "few" is a stop word or not,
and on the other hand we do not want the query above to match "propagating"
followed by 2 stop words, or even a sequence of 3 stop words in the document.

[...]
_________________________________


I hope I didn't get it completely wrong - sorry for wasting your time
otherwise..

Christian, BaseX Team 
http://www.basex.org


-- 
Configure bugmail: http://www.w3.org/Bugs/Public/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

Received on Friday, 13 March 2009 14:22:40 UTC