- From: Robert Ma <robertma@chromium.org>
- Date: Tue, 18 Dec 2018 13:33:11 -0500
- To: blink-dev <blink-dev@chromium.org>, ecosystem-infra <ecosystem-infra@chromium.org>
- Message-ID: <CAOPAaNJ_-1abHpEGDve_AUOiiAvtji38TzHkgrwWOiNZxzZBLw@mail.gmail.com>
This research uses the Chromium code coverage toolchain <https://chromium.googlesource.com/chromium/src/+/master/docs/code_coverage.md> to gather and compare the code coverage of Blink provided by web tests & WPT (using run_web_tests.py and content_shell). Three runs were done at the same revision: the full web tests, WPT only (external/wpt), and web tests without WPT. Virtual tests are included/excluded accordingly. The full results can be viewed at: https://storage.googleapis.com/blink-wpt-coverage/201812/index.html Overall, WPT provides lower coverage than other web tests. And here’s a manually curated list of Blink components whose WPT coverage is much (>30%) lower than the full web tests (bold components have an even larger difference than the rest): Accessiblity > BackgroundSync > Contacts > Editing > Forms > ImageCapture > Location > Media>Controls > Media>Session > Media>Recording > PresentationAPI > PushAPI > SVG > SecurityFeature>CredentialManagement > Sensor>DeviceOrientation > Speech > Storage>FileSystem > Storage>Quota > Vibration > WebGL > WebGPU > WebMIDI > WebXR>VR Some of the differences are to be expected (e.g. WebGL/GPU is not included in WPT at the moment). We will use look closer at the list to identify key areas where we can improve testing infrastructure and automation. Meanwhile, suggestions or insights about the data are much appreciated! (Link to details about test setup <https://docs.google.com/document/d/1D4qPWWkGSV-CpPnhAVAMDW3hQQtaJTO6YDVOCkGOUAo/edit#heading=h.gl0ch3w76fw> .) Robert
Received on Tuesday, 18 December 2018 18:33:48 UTC