enable Travis-CI for the main Tahoe-LAFS repo #2072

New issue

Closed

opened 2013-09-01 01:05:06 +00:00 by daira · 21 comments

daira commented

2013-09-01 01:05:06 +00:00

Travis-CI is a virtual-machine-based continuous integration system, which does various neat things like annotating pull requests with the results of testing a build.

I've added a single-purpose travis-tahoe github user to confirm that this works with Tahoe-LAFS. Here are the successful results for that user's fork of the tahoe-lafs repo: https://travis-ci.org/travis-tahoe/tahoe-lafs

Unfortunately Travis requires a github user with admin rights over the repo it is reporting on. We need to decide whether to grant those rights for the main tahoe-lafs/tahoe-lafs repo.

(The Tahoe-LAFS commit to add the configuration file for Travis build testing [is on master already]changeset:b8322c9c53710fb2d82da2d2ab9c9a9df45f0e24/trunk. By itself it has no security consequences.)

[Travis-CI](https://travis-ci.com) is a virtual-machine-based continuous integration system, which does various neat things like [annotating pull requests with the results of testing a build](http://about.travis-ci.org/blog/2012-09-04-pull-requests-just-got-even-more-awesome/). I've added a single-purpose [travis-tahoe](https://github.com/travis-tahoe) github user to confirm that this works with Tahoe-LAFS. Here are the successful results for [that user's fork](https://github.com/travis-tahoe/tahoe-lafs) of the tahoe-lafs repo: <https://travis-ci.org/travis-tahoe/tahoe-lafs> Unfortunately [Travis requires a github user with admin rights over the repo it is reporting on](https://github.com/travis-ci/travis-ci/issues/1390). We need to decide whether to grant those rights for the main [tahoe-lafs/tahoe-lafs](https://github.com/tahoe-lafs/tahoe-lafs) repo. (The Tahoe-LAFS commit to add the configuration file for Travis build testing [is on master already]changeset:b8322c9c53710fb2d82da2d2ab9c9a9df45f0e24/trunk. By itself it has no security consequences.)

daira added the

labels 2013-09-01 01:05:06 +00:00

daira added this to the undecided milestone 2013-09-01 01:05:06 +00:00

daira commented

2013-09-01 02:38:35 +00:00

Author

Also see #1810 #2052.

zooko commented

2013-09-01 15:20:56 +00:00

I posted a request to the travis-ci twitter account:

https://twitter.com/zooko/status/374190039355232257

I posted a request to the travis-ci twitter account: <https://twitter.com/zooko/status/374190039355232257>

daira commented

2013-09-01 16:03:57 +00:00

Author

The corresponding Travis issue is https://github.com/travis-ci/travis-ci/issues/1390.

The corresponding Travis issue is <https://github.com/travis-ci/travis-ci/issues/1390>.

daira commented

2013-09-01 16:14:33 +00:00

Author

I asked on the Travis ticket:

As a workaround, would it work to enable Travis and then afterward revoke the github user's admin permission (leaving it with read permissions, and not revoking the OAuth grant)?

roidrage answered:

You can certainly try that combination. Whether it will work in all scenarios, we can't guarantee.

I think we should try this.

I asked on the Travis ticket: > As a workaround, would it work to enable Travis and then afterward revoke the github user's admin permission (leaving it with read permissions, and not revoking the OAuth grant)? roidrage answered: > You can certainly try that combination. Whether it will work in all scenarios, we can't guarantee. I think we should try this.

daira commented

2013-09-01 16:45:33 +00:00

Author

Brian, please give travis-tahoe temporary admin permission to tahoe-lafs/tahoe-lafs.

warner was assigned by daira

2013-09-01 16:45:33 +00:00

daira commented

2013-09-01 16:47:32 +00:00

Author

Oh hang on, I can test this myself with a different repo, to see whether it will work.

daira commented

2013-09-01 17:35:30 +00:00

Author

Replying to daira:

Oh hang on, I can test this myself with a different repo, to see whether it will work.

Actually I can't, because only an organisation can grant a user other than a repo's owner admin access to that repo. Creating an organisation requires a paid github account.

Replying to [daira](/tahoe-lafs/trac/issues/2072#issuecomment-395107): > Oh hang on, I can test this myself with a different repo, to see whether it will work. Actually I can't, because only an organisation can grant a user other than a repo's owner admin access to that repo. Creating an organisation requires a paid github account.

daira commented

2013-09-01 19:27:59 +00:00

Author

Replying to [daira]comment:8:

Creating an organisation requires a paid github account.

I was mistaken about that.

The result of the testing was that you can revoke Administrative access and the status updating still works, but you cannot revoke write (push) access.

Replying to [daira]comment:8: > Creating an organisation requires a paid github account. I was mistaken about that. The result of the testing was that you can revoke Administrative access and the status updating still works, but you cannot revoke write (push) access.

warner commented

2013-09-04 19:31:13 +00:00

I've used travis-ci on a different project (https://github.com/warner/python-versioneer). It's pretty nice. On that one, I revoked it's admin access after adding the post-commit hook which triggers builds. (from what I could tell, the main use case for that access was to simplify the hook installation, and could be done manually by copy/pasting some tokens from travis-ci into github's repo config page). I haven't explored the "mark a revision as passing" feature or what authority it needs.

You can add an IMG tag to the README which will show the current build status for a single branch. That's a non-authority-requiring way to display build status.

Is travis really capable of running our build and test suite? Its buildslaves are all donated by supporters, and impose a pretty strict timeout. The build-in-a-sandbox feature is pretty nice, but my general impression is that it's most suitable for small projects with simple dependencies (e.g. things which are pip-installable). I'll look at the test results you posted.

I've used travis-ci on a different project (<https://github.com/warner/python-versioneer>). It's pretty nice. On that one, I revoked it's admin access after adding the post-commit hook which triggers builds. (from what I could tell, the main use case for that access was to simplify the hook installation, and could be done manually by copy/pasting some tokens from travis-ci into github's repo config page). I haven't explored the "mark a revision as passing" feature or what authority it needs. You can add an IMG tag to the README which will show the current build status for a single branch. That's a non-authority-requiring way to display build status. Is travis really capable of running our build and test suite? Its buildslaves are all donated by supporters, and impose a pretty strict timeout. The build-in-a-sandbox feature is pretty nice, but my general impression is that it's most suitable for small projects with simple dependencies (e.g. things which are pip-installable). I'll look at the test results you posted.

warner commented

2013-09-04 20:59:07 +00:00

Hm, another thought is to run a small daemon on our hardware (org) that polls or watches travis-ci for build results, then uses the github API to update the build-status data. The github repo admin token would then live on our own box, not on the travis box.

daira commented

2013-09-04 21:09:14 +00:00

Author

Replying to warner:

Is travis really capable of running our build and test suite? Its buildslaves are all donated by supporters, and impose a pretty strict timeout.

The timeout is 50 minutes, or 10 minutes with no output, which is fine for Tahoe-LAFS' test suite.

Do you think the remaining permission issue is important enough to justify the separate daemon? It seems like quite a bit of work. Personally I don't mind that the travis-tahoe user can commit provided that it can only commit as itself, which I think is the case.

If revoking the admin permission after setting the hook is okay, then please give travis-tahoe admin access to tahoe-lafs/tahoe-lafs (in a new Team so that you can change that permission independently).

Replying to [warner](/tahoe-lafs/trac/issues/2072#issuecomment-395110): > Is travis really capable of running our build and test suite? Its buildslaves are all donated by supporters, and impose a pretty strict timeout. The timeout is [50 minutes, or 10 minutes with no output](https://github.com/travis-ci/travis-ci/issues/1362), which is fine for Tahoe-LAFS' test suite. Do you think the remaining permission issue is important enough to justify the separate daemon? It seems like quite a bit of work. Personally I don't mind that the `travis-tahoe` user can commit *provided* that it can only commit as itself, which I think is the case. If revoking the admin permission after setting the hook is okay, then please give `travis-tahoe` admin access to `tahoe-lafs/tahoe-lafs` (in a new Team so that you can change that permission independently).

warner commented

2013-10-02 00:47:17 +00:00

Activating the hook with that token will, I believe, cause travis to start building things when you push a new change.

The incantation to attenuate the oauth token is (fill in USER and ID):

curl -u USER https://api.github.com/authorizations
# note the "id" number for the Travis app. Ignore the Gist app.
curl -X PATCH -d'{"remove_scopes": ["public_repo"]}' -u USER https://api.github.com/authorizations/ID
curl -X PATCH -d'{"add_scopes": ["repo:status"]}' -u USER https://api.github.com/authorizations/ID

curl will ask for your github password with each request (it uses HTTP basic auth, over SSL).

One problem with this scheme is that, eventually (I'm not sure exactly when), your travis session will expire, and if you re-"sign in with github", it will replace the attenuated oauth token with a full-power one, and you'll have to do this dance again.

And of course, this attenuation removes travis's ability to automatically configure your repo's hooks for you. I don't think that's a problem: pasting the travis token into the webhook configuration seems to be enough to trigger builds.

And there's still the window of time (after sign-in, before the PATCH api call) during which travis gets full access to your repos.

FYI, I've set up travis on some of my own projects with somewhat limited authority. To sign into travis at all, you have to grant it an awful lot of power (the basic OAuth request includes write access to your repositories). But there is an incantation you can speak to Github's API to reduce the scope of the travis-ci oauth token to do less. Before or after you attenuate the token like that, you can go to the travis account page and copy down the travis token. Then you can paste this token into the "travis-ci" service hook on one of your repositories. Activating the hook with that token will, I believe, cause travis to start building things when you push a new change. The incantation to attenuate the oauth token is (fill in USER and ID): ``` curl -u USER https://api.github.com/authorizations # note the "id" number for the Travis app. Ignore the Gist app. curl -X PATCH -d'{"remove_scopes": ["public_repo"]}' -u USER https://api.github.com/authorizations/ID curl -X PATCH -d'{"add_scopes": ["repo:status"]}' -u USER https://api.github.com/authorizations/ID ``` `curl` will ask for your github password with each request (it uses HTTP basic auth, over SSL). One problem with this scheme is that, eventually (I'm not sure exactly when), your travis session will expire, and if you re-"sign in with github", it will replace the attenuated oauth token with a full-power one, and you'll have to do this dance again. And of course, this attenuation removes travis's ability to automatically configure your repo's hooks for you. I don't think that's a problem: pasting the travis token into the webhook configuration seems to be enough to trigger builds. And there's still the window of time (after sign-in, before the PATCH api call) during which travis gets full access to your repos.

daira commented

2013-10-02 10:55:30 +00:00

Author

I don't think the approach in comment:395113 gives enough of a security improvement to be worth the complexity or hassle when the token expires. Shall we just give travis-tahoe admin access then revoke it (i.e. change to write access) after the automatic hook setting as described in comment:395112? If I understand correctly, any undesired operations it did with the admin authority (well, other than deleting the repo) would show up in the log for tahoe-lafs/tahoe-lafs that is viewable through the github Web interface. Then, after the revocation, any write to the repo would show up as a push (possibly a force-push) by the travis-tahoe user, so it couldn't write silently.

I don't think the approach in [comment:395113](/tahoe-lafs/trac/issues/2072#issuecomment-395113) gives enough of a security improvement to be worth the complexity or hassle when the token expires. Shall we just give `travis-tahoe` admin access then revoke it (i.e. change to write access) after the automatic hook setting as described in [comment:395112](/tahoe-lafs/trac/issues/2072#issuecomment-395112)? If I understand correctly, any undesired operations it did with the admin authority (well, other than deleting the repo) would show up in the log for `tahoe-lafs/tahoe-lafs` that is viewable through the github Web interface. Then, after the revocation, any write to the repo would show up as a push (possibly a force-push) by the `travis-tahoe` user, so it couldn't write silently.

daira commented

2013-10-02 11:00:56 +00:00

Author

BTW, setting the hook manually seems to leave the travis Web interface in a state where it doesn't know that the repo has been enabled. That's why I don't want to do it that way.

zooko commented

2013-10-02 12:27:47 +00:00

I'm okay with any of the authority-managing mechanisms mentioned above.

zooko commented

2013-10-23 15:15:36 +00:00

What's the state of this ticket? Somebody move this forward, please! I think the current state is "Brian and Daira disagree about how to do it.", so I think the next-step is for the two of them to come to agreement and then go ahead and implement what they've agreed.

daira commented

2013-10-23 15:46:14 +00:00

Author

My last comments were comment:395114 and comment:395115; I was blocked on warner responding to those. Also I don't have the necessary permissions to grant the admin access (which is fine).

My last comments were [comment:395114](/tahoe-lafs/trac/issues/2072#issuecomment-395114) and [comment:395115](/tahoe-lafs/trac/issues/2072#issuecomment-395115); I was blocked on warner responding to those. Also I don't have the necessary permissions to grant the admin access (which is fine).

daira commented

2014-02-26 15:42:30 +00:00

Author

https://github.com/travis-ci/travis-ci/issues/1390 has been fixed, so we should revisit this. I still think it's a good idea to use the travis-tahoe user for defence in depth.

<https://github.com/travis-ci/travis-ci/issues/1390> has been fixed, so we should revisit this. I still think it's a good idea to use the `travis-tahoe` user for defence in depth.

daira modified the milestone from undecided to soon (release n/a)

2014-02-26 15:42:30 +00:00

warner was unassigned by daira

2014-02-26 15:42:30 +00:00

daira self-assigned this 2014-02-26 15:42:30 +00:00

zooko changed title from ~~decide whether to enable Travis-CI for the main Tahoe-LAFS repo~~ to enable Travis-CI for the main Tahoe-LAFS repo

2014-03-19 16:30:35 +00:00

daira commented

2014-09-01 15:06:01 +00:00

Author

warner: please add the travis-tahoe github user as a contributor to tahoe-lafs/tahoe-lafs (or alternatively, give the daira user admin access).

warner: please add the `travis-tahoe` github user as a contributor to `tahoe-lafs/tahoe-lafs` (or alternatively, give the `daira` user admin access).

daira removed their assignment 2014-09-01 15:06:01 +00:00

warner was assigned by daira

2014-09-01 15:06:01 +00:00

daira commented

2014-09-01 15:24:50 +00:00

Author

Replying to daira:

warner: please add the travis-tahoe github user as a contributor to tahoe-lafs/tahoe-lafs (or alternatively, give the daira user admin access).

Oh, actually travis-tahoe will still need admin access temporarily to add the webhook. So please give daira admin access, so that I can grant travis-tahoe admin access, turn on the webhook, and then revoke admin.

Replying to [daira](/tahoe-lafs/trac/issues/2072#issuecomment-395124): > warner: please add the `travis-tahoe` github user as a contributor to `tahoe-lafs/tahoe-lafs` (or alternatively, give the `daira` user admin access). Oh, actually `travis-tahoe` will still need admin access temporarily to add the webhook. So please give `daira` admin access, so that I can grant `travis-tahoe` admin access, turn on the webhook, and then revoke admin.

daira commented

2014-09-09 18:27:12 +00:00

Author

Fixed!

We also have build status badges on the tahoe-lafs.org frontpage and README.rst.

Fixed! We also have build status badges on the [tahoe-lafs.org frontpage](https://tahoe-lafs.org/) and [README.rst](https://github.com/tahoe-lafs/tahoe-lafs).