From: Shwetank Dixit [mailto:shwetank@barrierbreak.com] Sent: Monday, January 30, 2017 11:10 AM Human expert testing is still essential here as many of these require judgement calls on things which might otherwise be passed by existing automated tools in some cases (false positives etc) or in some cases are simply not testable yet with existing tools. Having said that, I think we need the make sure that SCs minimize the use of subjective criteria as much as possible (won't be possible all the time, but we should continue to strive toward it). Not just for the sake of automated testting, but also for better clarity and consistency amongst interpretations by humans. [Jason] It seems to me that a tendency of well informed evaluators to agree in their judgments (i.e., high inter-rater reliability) is good evidence that a requirement is not “subjective” – that is, it refers to a real and well defined property of the content. ________________________________ This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. Thank you for your compliance. ________________________________
Received on Monday, 30 January 2017 16:17:26 UTC