Survey: issue tracking, summarization, and clustering

+www-archive

To: 

Chasen Le Hara	 chasen@chasenlehara.com <mailto:chasen@chasenlehara.com?Subject=Re%3A%20HTML%20version%20issue%20summary%3F&In-Reply-To=%253C97BA8346-3183-441D-BB7A-82D15A593723%40chasenlehara.com%253E&References=%253C97BA8346-3183-441D-BB7A-82D15A593723%40chasenlehara.com%253E> 	
Dan Connolly	 connolly@w3.org <mailto:connolly@w3.org?Subject=Re%3A%20HTML5%20as%20our%20specification%20text%20for%20review%3F%20(formal%20question%20WBS%20%09survey)&In-Reply-To=%253C1177700199.22874.408.camel%40dirk%253E&References=%253C1177700199.22874.408.camel%40dirk%253E> 	
David Dailey	 david.dailey@sru.edu <mailto:david.dailey@sru.edu?Subject=Re%3A%20survey%20of%20top%20web%20sites&In-Reply-To=%253C6.2.5.6.1.20070425133626.01dcb668%40sru.edu%253E&References=%253C6.2.5.6.1.20070425133626.01dcb668%40sru.edu%253E> 	
David McClure	 ?	
Ian Hickson	 ian@hixie.ch <mailto:ian@hixie.ch> 	

cc: 

Chris Wilson	 cwilso@microsoft.com <mailto:cwilso@microsoft.com> 	
David Hyatt	 hyatt@apple.com <mailto:hyatt@apple.com> 	
Doug Schepers	 doug.schepers@vectoreal.com <mailto:doug.schepers@vectoreal.com> 	

I hope what I am doing here is in order. If not, I'm sure one of the chairs will let me know.

We represent the five individuals who responded to the recent survey called by Dan http://lists.w3.org/Archives/Public/public-html/2007Apr/1399.html who showed an interest in helping with "issue tracking, summarization, and clustering ". I hope my spreadsheet work did not screw anything up, and apologies to David McClure whose email address I could not find in the places I knew to look.

I also copied Chris, David Hyatt and Doug. Chris is a co-chair. David looks like he is likely to become an editor and Ian has indicated that as editor he is likely to be involved in all these issues. David, if you wish, please let me know and I'll try to remove you from the above list. Doug, I recall, volunteered to be involved in this initiative which reflects a possible idiosyncracy of my memory. If you know of others, and if the activity I suggest is appropriate, then I suggest they be added to this discussion.

I recall that a little bit of discussion of issue tracking has arisen thus far in the email discussion. Doug asked about good issue-tracking software. Someone mentioned SVN + trac. I replied http://lists.w3.org/Archives/Public/public-html/2007Apr/1389.html that

-----<quote>---------

3a. issue tracking and clustering -- for example the issue of 
enhancements to the <table> object have come up in at least six 
separate occurrences -- only some of those suggestions seem to be 
handled by the WHATWG datagrid or datalist elements. SVN + trac ? 
(http://lists.w3.org/Archives/Public/public-html/2007Apr/1214.html <http://lists.w3.org/Archives/Public/public-html/2007Apr/1214.html> ) I 
think this deals more with the issues once that have been identified. 
If we approach this differently (like as an analysis of customer 
specifications) then "content analysis" might be more appropriate, 
since it would help in the identification and labeling of issues. 
[...]

Now I understand 
that from a chair's perspective, the obligation to address all 
"issues" means that one would not want each expressed opinion or 
concern to require a formal response, so the whole issue of what is 
an "issue" might involve some new nomenclature. But in the long run, 
I think giving a certain amount of editorial independence to the 
group's contributors (a probably outgrowth of such collaborative 
software) would help the chairs, the editors, and the issue trackers 
in the long run.
--</quote>-----

I quote myself above not because I think it is the best or only approach, but rather because it's the only discussion I'm aware of (though Ian may well know of many others) in our WG that has addressed issue tracking methodology per se.

Anyhow what I'd like to suggest is a means by which we might actually begin working on a "suggestion tracking" methodology. I use the term "suggestion tracking" since I am not recommending that all suggestions that WG participants have made qualify as issues worthy of the official "addressing." [Dan I tried to find your message on this subject to include a link, but was unable to find it]. The actual W3C language about the need to address issues should probably be inserted in a working draft somewhere, if this little group of five (or eight or more) actually comes into existence.

I do think it is important that all "suggestions" be identified and either marked as closed, as uncertain, or as issues that the WG will address. This will go a long way toward making sure that the individuals who have devoted large amounts of effort (2500+ email messages so far) know that their "suggestions" have been listened to. This goes a long way toward meeting what I think some of the aims of our WG actually are: to ensure representativeness. All parties seem to concur on this as a basic premise. The more sunlight can be cast on the process the better off we all are.

Let me give an example: in the debate on <indent> as a new tag, some people think the issue is still alive. Others seem to think it has been sufficiently refuted. Summarily siding with either perspective would not be the role of issue tracking, summarization, or clustering as I see it. 

How can we do that? Well, maybe there is software to help. 

1. http://trac.edgewall.org/ Trac is an open source-issue tracking system for software development. My colleagues who know about that sort of thing tell me it is sort of a project management thing for tracking issues once they have been identified, not for identifying them in the first place.

2. content analysis -- When I gathered public testimony pertaining to a $1billion lawsuit in Alaska in the early 1980's, there was no good software for helping me to digest the large volume of public testimony. I offered to design some. Alaska did not buy. I really don't know if there has been much progress since then. I would hope so. That would be one way to proceed that would make the entire process quite visible and accountable.

3. Help desk or software bugs -- issue tracking. I know both of these sorts of systems must in some way allow for the identification of the fibrous and tangled nature of problems. Two problems are often one. One problem often turns into many. Perhaps there is some suitable software that could be adapted to our needs. I suspect everybody here but me probably knows more about this area than I do.

Maybe there isn't such software.

Let me suggest a methodology first, and then see if there is appropriate software later:

Suppose during the next month, we were to divvy up the 4000 (2500 so far plus 1500 for May ) messages in some way. Maybe we all start with March 7th and take turns following threads using (thread#)%(number of contributors).

A. Each message is dealt with by marking it either "new" or "covered by (subsumed under) some prior thread". A cumulative list of the names of those suggesting the recommendation is made. A separate and cumulative list of the names of those "refuting" the issue is also maintained.

B. Once an exhaustive and exclusive list of all "suggestions" has been tallied -- assume for example that the 4000 messages boil down to 400 distinct "proto-issues" -- then that list of 400 proto-issues is used as a sparse-matrix questionnaire. All persons on record as proponents of the feature are asked if they have been convinced that the issue has been sufficiently disputed so as to warrant removing it from the list of "open suggestions" or if they would like to keep it alive for further consideration.

C. The list of 400 proto-issues might then be whittled down to some smaller list (say 300) issues which are still in the minds of some participants still "open". This becomes the list of "open suggestions."

D. The list of open suggestions is then submitted for reanalysis by those identified as the "refuters" of the suggestions. The refuters are asked -- do you still hold your objections.

E. Part 1. All suggestions that are dropped are identified as dropped.

E. Part 2. All suggestions that are proposed without challenge become issues (for further deliberation).

E. Part 3. All contested issues are then sent back to both proposers and detractors (as identified by name and e-mail address). Each side is given one week to file a position paper. Each is given one week to file a response to the opposition's position paper. All papers must be fewer than 500 words.

F. The WG is surveyed. On the basis of that survey and whatever other information they wish to consider, the chairs (or the permanent members of the committee or whatever is appropriate -- it seems like there is some core membership exclusive of invited experts) decide which suggestions become issues worthy of placing on the WG's official agenda.

Whatcha think? Is it worth a try?

respecfully,

David Dailey

Received on Sunday, 29 April 2007 14:42:49 UTC