[Mageia-webteam] Gitorious - LDAP installation-integration Feasibilty

[Michael Scherer]

Le jeudi 06 janvier 2011 à 17:48 +0000, Kosmas Chatzimichalis a écrit :
> Further to the discussion about gitorious setup and integration with ldap:

First, I am likely to be quite undiplomatic ( yeah as usual ), and
basically say "no" so beware :)

> First there is a need for a git setup, as at this moment there are two
> projects using git (mageia-app-db, mageia-maintainers-db).

Sorry, but I would not call this a need, but a personal preference.
2 teams deciding on their own to use whatever suit them is not what I
call a need.

Yes, I would also love git, but that's because I would love it that I
recognize it as a personal preference. While I guess several people does
it too, I am not a fool enough to claim that git is the vcs that
everybody will love. So "setting up $product" is not a need. Once the
community have decided ( ie, more than the few people that participate
on this thread ), yes, this would be a need.

Since the choice of a DVCS is traditionally a total flamewar ( see for
example the gnome migration ), I would prefer to have community opinion
on the subject before calling this a need. Ie, see with developers, see
with I18N, see with sysadmin.

If someone did start to use darcs for a project ( like usually Nanar
does as this is his favorite VCS ), would we be saying "we need to host
darcs ?". Likely. And yet, I would not think we should, because we would
finish by requiring to host everything. This is exactly what is
happening now for the web server, where we will end doing python, php,
perl and ruby. And that's why I think we can do better in term of
ressources planning than just saying "we need that because me and some
others decided".

Ask to developers their opinion, ask to i18n too, and that would be fine
with me. I am pretty sure that most people will be ok, but we cannot be
sure. 

In the case of gnome, the cost of learning to use git was a problem for
translators. We do plan to reduce that with transifex, but some people
asked if tx was mandatory, so you can see that some people may have a
opinion on that, and should in all case be aware of such change, even if
it will likely cause no problem.

> In order to be able to use git, we can either host the projects
> externally, or install git in the mageia servers and host the projects
> there.
> At the moment both projects are hosted externally, but when the mageia
> servers have a git installation they can be transferred there.

So what about the bugs, wiki, etc are you ready to move this to bugzilla
and others software ? 

> For installing git on the mageia servers, we can either install plain
> git or an application like gitorious, as suggested in the list
> previously.

Again "setting gitorious" is not a need, it is a personal choice. No one
has a need of "we need to use this product". It should be "here is what
the community as a whole need, let's find something to fulfill it".

Otherwise, we end with people each pushing their own personal preference
without first discussing with others, causing frustration, likely
duplication, and so causing inadequate installation, more work and less
contribution ( du to the complexity ).

For example, gitorious can be used as review tool. Since Derek already
proposed reviewboard for this
( https://www.mageia.org/pipermail/mageia-sysadm/2010-November/000323.html ), it would be better to first assess our needs, then decide. I do not think having 2 tools to achieve the same thing is a good idea.
( not to mention that there is others tools like this, for example
gerrit ).

Code review is indeed a good idea, and both gitorious and reviewboard
would be usable for that. But that doesn't mean that's a so good idea
that we should setup 2 or 3 systems for that, nor rush to have it right
now. First a process, later a product.

Gitorious is also likely to overlap with the forge project that we are
speaking from time to time ( and that's still unspecified ). As Romain
said, maybe this will be bugzilla + $VCS, and some people asked for
trac, some for redmine. This would also be a lot of duplication.

Gitorious does wiki + git viewer + review. We need to define what
features we need, and in the case of a forge , what we would want for it
( likely adding a bug tracker, a download area ), and see how it
integrate with that.

I like the idea of gitorious, don't get me wrong, I even did a test
installation of the rpm because that's what was planned for mdv. But we
need to do things in order, think on what we want to achieve. And that
mean to think of needs without saying the name of a product, with
community input, with proper process.

Ie, what is gitorious used for ?
Social coding ( ie, review of patch ) ? 
Glorified git repository viewer ?
Others ( like web interface to mange git ) ?

( yes, I know that a long way to say "no" )

> The first option about the plain git installation should be straight
> forward and as simple as: urpmi git-core. Some more information about
> transferring or creating git repositories from one server to another
> can be found here: [1][2]

Again, sorry to disagree, that's not as straightforward as it seems.

First, we need to decide basic things, like "where do people push
personal branch" ( because that sound like a use case for git, or people
would have used git-svn ). 

And "what is our policy around the various thing that people can do with
git", like rewriting history ( that we should forbid I guess ), etc. 

Depending on the usage, we also need various things like a git
repository browser, various checks ( see post-commit hook for svn ),
maybe acl ( and therefor, integration with ldap ), commit mail, etc. 

That's unfortunately slightly not as simple than "urpmi git-core".

And that would requires community input first, like all policies ( or at
least, that's what we should achieve ).

> For the gitorious setup as it is a Ruby on Rails application there are
> some requirements to be able to install it.
> A blog post with full details can be found here: [3]
> During the installation of gitorious there are two options:
> Either install the application from source code or from the available rpm [4].
> Before this stage though a few dependencies and requirements would
> need to be installed.
> Most of them are also relevant to the mageia-maintenainers-db.

I doubt having sphinxsearch or a message queuing server is gonna be
needed for any kind of maintainer db ( I hope not ). And the fact that
most of them are also used by another project is not much helping, since
we didn't even decided on the infrastructure to deploy. More ever, what
is shared is likely the simpler part ( ruby modules ). The problematic
one are more complex to setup ( ie, indexing daemon, git/ssh management
integration, message queuing using a java daemon ).

There is also others issues, for example, the documentation speak of
phusion and mod_passenger. If we use it on alamut ( our current
application server ), since mod_perl is already used for bugzilla,
apache will have ruby and perl interpreter embedded, a combination that
I would prefer to avoid for basic performance reason ( and that's why I
asked for fast cgi support, to at least keep sanity to a acceptable
level ). So we may need to setup more than just a web application, like
2nd ruby application server.

And it would also be nice to take in account the fact that we try to use
postgresql as much as possible, but since that's a rails app, I take for
granted it does ( even if some bug report say it
doesn't :http://gitorious.lighthouseapp.com/projects/53751/tickets/2-group-by-query-error-on-postgresql  ).

> NOTE: As mentioned before, I'm available to help on this stage, as
> I've done similar installations before.

Well, the goal is not to set it as a one time setup. We can perfectly do
that without much problem ( even if for gitorious, the installation
procedure is bigger than the note I took for installing our newest
server ). 

The goal is to make a repeatable installation using puppet ( and a
maintainable one too ), since puppet give us automation ( useful for
testing upgrade, for disaster recovery, for ensuring everything is
correctly setup ), and is backed with a vcs ( useful for automated
communication, for traceability, for sharing our work, and for easier
integration of external contribution ). 

> Finally, about the integration with ldap.
> After a quick search, it seems that is possible and there are people
> out there that have done that.
> There are probably two different ways of achieving this.
> First one seems to be through Apache and mod_ldap and smart http 
> ([5][6])
> Second one seem to be through PAM authentication ([6][7])

You are speaking of git or gitorious ?

If this is for git, yes, we know how to integrate it, that should be
trivial using git + ssh ( at least if no acl are involved and to use it
like we use svn with a single reference repository ).

Now for gitorious, that's more complex. Does it support ldap to the
point that the ssh key is managed by ldap outside of gitorious ? 

I was under the impression that gitorious used a system like gitosis
where the acl and ssh keys are managed for a unique account by
gitorious, duplicating the information we have in our ldap, and that's
not optimal, and in fact would quite a duplication of what we do with
current group based acl.

If it support ldap in the web interface, does it support it fully ?
Ie, if someone change name or email, is it updated ? Can we force people
to go to catdap to register and prevent non ldap login ?

Etc.

So to summarize :
- I think we need community input for git, at least for the policy and
workflow that should be setup, and to confirm it will not cause trouble
in unexpected way.

- gitorious is likely to be more problematic, for all aformentioned
technical and organisational reason I outlined.

-- 
Michael Scherer