A Pew Research Analysis of Government Websites

Civic Technology Community Group,

Hello. I am pleased to share a Pew Research analysis of government websites, news websites, Wikipedia, and Twitter.

When Online Content Disappears (https://www.pewresearch.org/data-labs/2024/05/17/when-online-content-disappears/)
by Athena Chapekis, Samuel Bestvater, Emma Remy, and Gonzalo Rivero

With respect to government websites:

We sampled around 500,000 pages from government websites using the Common Crawl March/April 2023 snapshot of the internet, including a mix of different levels of government (federal, state, local and others). We found every link on each page and followed a random selection of those links to their destination to see if the pages they refer to still exist.

Across the government websites we sampled, there were 42 million links. The vast majority of those links (86%) were internal, meaning they link to a different page on the same website. An explainer resource on the IRS website that links to other documents or forms on the IRS site would be an example of an internal link.

Around three-quarters of government webpages we sampled contained at least one on-page link. The typical (median) page contains 50 links, but many pages contain far more. A page in the 90th percentile contains 190 links, and a page in the 99th percentile (that is, the top 1% of pages by number of links) has 740 links.

Other facts about government webpage links:

  *
The vast majority go to secure HTTP pages (and have a URL starting with “https://”).
  *
6% go to a static file, like a PDF document.
  *
16% now redirect to a different URL than the one they originally pointed to.
  *
When we followed these links, we found that 6% point to pages that are no longer accessible. Similar shares of internal and external links are no longer functional.

Overall, 21% of all the government webpages we examined contained at least one broken link. Across every level of government we looked at, there were broken links on at least 14% of pages; city government pages had the highest rates of broken links.


Best regards,
Adam Sobieski