Site stability status

Hi all,

I realized after a talk with Doug that I forgot to give a proper site
status. Here’s the summary.

During the last month we had a few intermittent down times [0][1] and
the cause was a set of software performance problems.

The heart of the problem was because we were using software that were in
their end-of-life. They had known memory leaks problems and weren’t
patched anymore.

My work of the last two weeks was then to rebuild server configuration
to use up to date versions. I had a few quirks with proper set of
software versions, but it should be fixed now.

* MediaWiki from 1.22wmf5 to 1.24wmf16
* Ubuntu server VMs from 10.04 LTS to 14.04 LTS for both app and db
nodes (6 VMs in total)
* MySQL 5.1 to Percona "XTraDB" Cluster MySQL 5.6

There are a few things to tidy up, but the overall stability should be
improved.

The next steps i see for our server setup are:

* Upgrade Piwik and Bug Genie versions
* Create a mirror of our complete setup[2]
* Upgrade Ubuntu 12.04 LTS to 14.04 LTS nodes
* Finish up removing hardcoded passwords, publish on github
* Improve logging system

  [0]: http://status.webplatform.org/post/94249032910
  [1]: http://status.webplatform.org/post/94144481705
  [2]:
http://lists.w3.org/Archives/Public/public-webplatform/2014Aug/0058.html

-- 
Regards,

Renoir Boulanger  |  Developer operations engineer
W3C  |  Web Platform Project

http://w3.org/people/#renoirb  ✪  https://renoirboulanger.com/  ✪  @renoirb
~

Received on Tuesday, 2 September 2014 16:58:48 UTC