Re: Schema.org sameAs/Article Test on Wikipedia

Very interesting Adam.  Curious is you can share with us any further detail
regarding *specifically* what's being tested, and how results are being
judged.

That is, you say:
"One motivation for this work is to explore the effect of the changes on
search relevance. More generally, we're also interested in how structured
markup like this on Wikimedia content projects might be beneficial to the
semantic web and machine readability..."

With regard to "the effect of the changes on search relevance", do you mean
how the additional markup impacts the visibility of Wikipedia pages in
enterprise search engines such as Google, Bing and Yandex, or are there
other search environments implicated here?  And what is/are the measurement
protocol/protocols you're using to assess "relevance" (e.g. changes in
search engine ranking, changes in the way a Wikipedia page is presented in
the search results, changes in what information is about a
Wikipedia-referenced entity is represented in a Google Knowledge Panel or
Bing Snapshot)?

And a similar question in regard to your interest in "how markup like this
... might be beneficial to the semantic web and machine readability...."
How are you assessing this impact (asking because, as with "search
relevance", any A/B test is predicated on begin able to measure the
performance of A compared to B, which in turn requires that a measurement
protocol be in place).

Thanks in advance for any further detail  you can provide.

Aaron Bradley
Knowledge Graph Strategist
Electronic Arts


On Tue, Nov 20, 2018 at 5:30 AM Adam Baso <abaso@wikimedia.org> wrote:

> Hello -
>
> I'm Adam Baso, an engineering director at the Wikimedia Foundation. We
> wanted to let the Schema.org community know that we're A/B testing the
> sameAs property on Wikipedia articles having a corresponding Wikidata
> entry. We're running the test on most [1] Wikipedias, including English
> Wikipedia. You can see this in effect for the markup on the English
> article for War of the Polish Succession
> <https://search.google.com/structured-data/testing-tool#url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWar_of_the_Polish_Succession>
> .
>
> We have applied the new property to 50% of Wikipedia pages with
> corresponding Wikidata entries by way of the Article type. Our CDN is being
> populated with this change so you should see more of this markup. We'll be
> running this A/B test for about two months and will be evaluating results
> along the way.
>
> One motivation for this work is to explore the effect of the changes on
> search relevance. More generally, we're also interested in how structured
> markup like this on Wikimedia content projects might be beneficial to the
> semantic web and machine readability, much along the lines of other
> Wikimedia movement initiatives such as Structured Data on Commons
> <https://commons.wikimedia.org/wiki/Commons:Structured_data>.
>
> Any feedback you have on our implementation in this A/B test would be most
> welcome. Please share your feedback with Olga Vasileva (
> ovasileva@wikimedia.org). Thanks!
>
> Regards,
>
> Adam Baso
>
> Engineering Director
>
> Wikimedia Foundation
>
> [1]
>
> The following wikis are excluded from A/B testing due to overlap with
> other A/B tests:
>
> Indonesian: idwiki <https://id.wikipedia.org/wiki/Halaman_Utama>
>
> Portuguese: ptwiki
> <https://pt.wikipedia.org/wiki/Wikip%C3%A9dia:P%C3%A1gina_principal>
>
> Punjabi: pawiki
> <https://pa.wikipedia.org/wiki/%E0%A8%AE%E0%A9%81%E0%A9%B1%E0%A8%96_%E0%A8%B8%E0%A8%AB%E0%A8%BC%E0%A8%BE>
>
> Dutch: nlwiki <https://nl.wikipedia.org/wiki/Hoofdpagina>
>
> Korean: kowiki
> <https://ko.wikipedia.org/wiki/%EC%9C%84%ED%82%A4%EB%B0%B1%EA%B3%BC:%EB%8C%80%EB%AC%B8>
>
> Bhojpuri: bhwiki <https://bh.wikipedia.org>
>
> Cherokee: chrwiki
> <https://chr.wikipedia.org/wiki/%E1%8E%A4%E1%8E%B5%E1%8E%AE%E1%8E%B5%E1%8F%8D%E1%8F%97>
>
> Kazakh: kkwiki
> <https://kk.wikipedia.org/wiki/%D0%91%D0%B0%D1%81%D1%82%D1%8B_%D0%B1%D0%B5%D1%82>
>
> Catalan: cawiki <https://ca.wikipedia.org/wiki/Portada>
>
> French: frwiki
> <https://fr.wikipedia.org/wiki/Wikip%C3%A9dia:Accueil_principal>
>
> Yoruba: yowiki
> <https://yo.wikipedia.org/wiki/Oj%C3%BAew%C3%A9_%C3%80k%E1%BB%8D%CC%81k%E1%BB%8D%CC%81>
> Kalmyk: xalwiki
> <https://xal.wikipedia.org/wiki/%D0%9D%D2%AF%D1%80_%D1%85%D0%B0%D0%BB%D1%85>
>