- From: Michael Cooper <cooper@w3.org>
- Date: Tue, 3 May 2016 10:52:50 -0400
- To: ARIA Editors <public-aria-editors@w3.org>
- Message-ID: <5728BB42.80202@w3.org>
I've spent the past couple weeks figuring out how to get automated
snapshots of specs posted to the gh-pages branch. I finally have it
mostly figured out, though there's room for improvement. My
understanding is it's desirable to have this working when we split the
repositories so we can set them all up with source versions in the
master branch and up-to-date editors' drafts without respec code running
in the gh-pages branch.
The snapshots are generated by a continuous integration service called
Travis-CI:
https://travis-ci.org/
The service has to be enabled for each repository, and then whenever
there is a push to GitHub it does a build based on a configuration file
called travis.yml stored at the top level of the repository. I've been
testing this in a working branch of the aria-practices repository, so
you can see what the travis.yml file looks like:
https://github.com/w3c/aria-practices/blob/Travis-CI_Test/.travis.yml
The basic process right now is that whenever there is a push to GitHub,
Travis runs the spec through the W3C spec generator:
https://github.com/w3c/spec-generator
It checks out the gh-pages branch (it's using gh-pages_test for now),
saves the generated file in that branch, then commits and pushes the
result. This process basically works and the gh-pages snapshot is
updated within a couple minutes of pushing edits.
This works well enough now to demonstrate that it's a viable solution to
the long-standing problem of out-of-date snapshots. But there are a few
problems and items for further work:
* Currently this depends on a web service nominally run by W3C. My
experience is that service is down at least 50% of the time. If the
service is down when the script runs, it will return no result and
not push to gh-pages. It won't attempt to run again, so there won't
be another attempt to publish a snapshot until somebody pushes
commits again.
* To work around this, I want to have the script run a local copy of
the spec generator. This should be possible but the procedure is
undocumented, so I haven't yet figured it out. Perhaps Shane knows
how? If we get that working we'll have a reliable snapshot
mechanism. That could be a later upgrade though if necessary, the
version using the often-down web service will still be better than
what we've had until now.
* This script only attempts to build the spec file itself, it doesn't
attempt to ensure that images, css, etc. are present as needed in
the gh-pages branch. So we'd manually have to put those in whenever
they're created or updated. We could also have the script do that it
we wanted, but would need some sort of manifest file for each spec I
think so it knows what to copy. That said we may need that manifest
to use the automated publication system, so perhaps this is an
enhancement we want.
* At the moment, commits to gh-pages are made using my GitHub account.
It took me literally years to figure out how to get Travis-CI to
commit to GitHub using any account, so I tested on mine. However,
best practice is to use an account that is used just for automation,
to separate commits made by a human from one made by a bot. One has
to know authentication credentials for the account so can't just use
one that already exists; we might want to create an "aria-github"
account for this purpose.
* Before I merge this .travis.yml into the master branch, I will need
to add a restriction to it saying the build should only run if the
push was made to the master branch. This is so if people branch off
master to do something, and they end up with the .travis.yml in the
branch, Travis knows not to run even though the config is present.
Only edits made in, or merged into, master should be used to
generate snapshots I believe.
I didn't do this in the main aria repository because I didn't know if it
is possible to build only the file to which updates were made, so right
now a commit to one spec would cause all of them to be rebuilt. There
are worse things, but still. But when we split the repositories I think
it will make sense to put versions of this in each repository. I
provided all these details, beyond because I'm a word guy, so you all
would have complete information to decide on that. This is on the agenda
for tomorrow's meeting.
Michael
Received on Tuesday, 3 May 2016 14:52:51 UTC