[Mageia-webteam] Gitorious - LDAP installation-integration Feasibilty

Mon Jan 10 13:25:28 CET 2011

Le jeudi 06 janvier 2011 à 17:48 +0000, Kosmas Chatzimichalis a écrit :
+> Further to the discussion about gitorious setup and integration with ldap:
+
+First, I am likely to be quite undiplomatic ( yeah as usual ), and
+basically say "no" so beware :)
+
+> First there is a need for a git setup, as at this moment there are two
+> projects using git (mageia-app-db, mageia-maintainers-db).
+
+Sorry, but I would not call this a need, but a personal preference.
+2 teams deciding on their own to use whatever suit them is not what I
+call a need.
+
+Yes, I would also love git, but that's because I would love it that I
+recognize it as a personal preference. While I guess several people does
+it too, I am not a fool enough to claim that git is the vcs that
+everybody will love. So "setting up $product" is not a need. Once the
+community have decided ( ie, more than the few people that participate
+on this thread ), yes, this would be a need.
+
+Since the choice of a DVCS is traditionally a total flamewar ( see for
+example the gnome migration ), I would prefer to have community opinion
+on the subject before calling this a need. Ie, see with developers, see
+with I18N, see with sysadmin.
+
+If someone did start to use darcs for a project ( like usually Nanar
+does as this is his favorite VCS ), would we be saying "we need to host
+darcs ?". Likely. And yet, I would not think we should, because we would
+finish by requiring to host everything. This is exactly what is
+happening now for the web server, where we will end doing python, php,
+perl and ruby. And that's why I think we can do better in term of
+ressources planning than just saying "we need that because me and some
+others decided".
+
+Ask to developers their opinion, ask to i18n too, and that would be fine
+with me. I am pretty sure that most people will be ok, but we cannot be
+sure. 
+
+In the case of gnome, the cost of learning to use git was a problem for
+translators. We do plan to reduce that with transifex, but some people
+asked if tx was mandatory, so you can see that some people may have a
+opinion on that, and should in all case be aware of such change, even if
+it will likely cause no problem.
+
+> In order to be able to use git, we can either host the projects
+> externally, or install git in the mageia servers and host the projects
+> there.
+> At the moment both projects are hosted externally, but when the mageia
+> servers have a git installation they can be transferred there.
+
+So what about the bugs, wiki, etc are you ready to move this to bugzilla
+and others software ? 
+
+> For installing git on the mageia servers, we can either install plain
+> git or an application like gitorious, as suggested in the list
+> previously.
+
+Again "setting gitorious" is not a need, it is a personal choice. No one
+has a need of "we need to use this product". It should be "here is what
+the community as a whole need, let's find something to fulfill it".
+
+Otherwise, we end with people each pushing their own personal preference
+without first discussing with others, causing frustration, likely
+duplication, and so causing inadequate installation, more work and less
+contribution ( du to the complexity ).
+
+For example, gitorious can be used as review tool. Since Derek already
+proposed reviewboard for this
+( https://www.mageia.org/pipermail/mageia-sysadm/2010-November/000323.html ), it would be better to first assess our needs, then decide. I do not think having 2 tools to achieve the same thing is a good idea.
+( not to mention that there is others tools like this, for example
+gerrit ).
+
+Code review is indeed a good idea, and both gitorious and reviewboard
+would be usable for that. But that doesn't mean that's a so good idea
+that we should setup 2 or 3 systems for that, nor rush to have it right
+now. First a process, later a product.
+
+Gitorious is also likely to overlap with the forge project that we are
+speaking from time to time ( and that's still unspecified ). As Romain
+said, maybe this will be bugzilla + $VCS, and some people asked for
+trac, some for redmine. This would also be a lot of duplication.
+
+Gitorious does wiki + git viewer + review. We need to define what
+features we need, and in the case of a forge , what we would want for it
+( likely adding a bug tracker, a download area ), and see how it
+integrate with that.
+
+I like the idea of gitorious, don't get me wrong, I even did a test
+installation of the rpm because that's what was planned for mdv. But we
+need to do things in order, think on what we want to achieve. And that
+mean to think of needs without saying the name of a product, with
+community input, with proper process.
+
+Ie, what is gitorious used for ?
+Social coding ( ie, review of patch ) ? 
+Glorified git repository viewer ?
+Others ( like web interface to mange git ) ?
+
+( yes, I know that a long way to say "no" )
+
+> The first option about the plain git installation should be straight
+> forward and as simple as: urpmi git-core. Some more information about
+> transferring or creating git repositories from one server to another
+> can be found here: [1][2]
+
+Again, sorry to disagree, that's not as straightforward as it seems.
+
+First, we need to decide basic things, like "where do people push
+personal branch" ( because that sound like a use case for git, or people
+would have used git-svn ). 
+
+And "what is our policy around the various thing that people can do with
+git", like rewriting history ( that we should forbid I guess ), etc. 
+
+Depending on the usage, we also need various things like a git
+repository browser, various checks ( see post-commit hook for svn ),
+maybe acl ( and therefor, integration with ldap ), commit mail, etc. 
+
+That's unfortunately slightly not as simple than "urpmi git-core".
+
+And that would requires community input first, like all policies ( or at
+least, that's what we should achieve ).
+
+> For the gitorious setup as it is a Ruby on Rails application there are
+> some requirements to be able to install it.
+> A blog post with full details can be found here: [3]
+> During the installation of gitorious there are two options:
+> Either install the application from source code or from the available rpm [4].
+> Before this stage though a few dependencies and requirements would
+> need to be installed.
+> Most of them are also relevant to the mageia-maintenainers-db.
+
+I doubt having sphinxsearch or a message queuing server is gonna be
+needed for any kind of maintainer db ( I hope not ). And the fact that
+most of them are also used by another project is not much helping, since
+we didn't even decided on the infrastructure to deploy. More ever, what
+is shared is likely the simpler part ( ruby modules ). The problematic
+one are more complex to setup ( ie, indexing daemon, git/ssh management
+integration, message queuing using a java daemon ).
+
+There is also others issues, for example, the documentation speak of
+phusion and mod_passenger. If we use it on alamut ( our current
+application server ), since mod_perl is already used for bugzilla,
+apache will have ruby and perl interpreter embedded, a combination that
+I would prefer to avoid for basic performance reason ( and that's why I
+asked for fast cgi support, to at least keep sanity to a acceptable
+level ). So we may need to setup more than just a web application, like
+2nd ruby application server.
+
+And it would also be nice to take in account the fact that we try to use
+postgresql as much as possible, but since that's a rails app, I take for
+granted it does ( even if some bug report say it
+doesn't :http://gitorious.lighthouseapp.com/projects/53751/tickets/2-group-by-query-error-on-postgresql  ).
+
+> NOTE: As mentioned before, I'm available to help on this stage, as
+> I've done similar installations before.
+
+Well, the goal is not to set it as a one time setup. We can perfectly do
+that without much problem ( even if for gitorious, the installation
+procedure is bigger than the note I took for installing our newest
+server ). 
+
+The goal is to make a repeatable installation using puppet ( and a
+maintainable one too ), since puppet give us automation ( useful for
+testing upgrade, for disaster recovery, for ensuring everything is
+correctly setup ), and is backed with a vcs ( useful for automated
+communication, for traceability, for sharing our work, and for easier
+integration of external contribution ). 
+
+> Finally, about the integration with ldap.
+> After a quick search, it seems that is possible and there are people
+> out there that have done that.
+> There are probably two different ways of achieving this.
+> First one seems to be through Apache and mod_ldap and smart http 
+> ([5][6])
+> Second one seem to be through PAM authentication ([6][7])
+
+You are speaking of git or gitorious ?
+
+If this is for git, yes, we know how to integrate it, that should be
+trivial using git + ssh ( at least if no acl are involved and to use it
+like we use svn with a single reference repository ).
+
+Now for gitorious, that's more complex. Does it support ldap to the
+point that the ssh key is managed by ldap outside of gitorious ? 
+
+I was under the impression that gitorious used a system like gitosis
+where the acl and ssh keys are managed for a unique account by
+gitorious, duplicating the information we have in our ldap, and that's
+not optimal, and in fact would quite a duplication of what we do with
+current group based acl.
+
+If it support ldap in the web interface, does it support it fully ?
+Ie, if someone change name or email, is it updated ? Can we force people
+to go to catdap to register and prevent non ldap login ?
+
+Etc.
+
+So to summarize :
+- I think we need community input for git, at least for the policy and
+workflow that should be setup, and to confirm it will not cause trouble
+in unexpected way.
+
+- gitorious is likely to be more problematic, for all aformentioned
+technical and organisational reason I outlined.
+
+-- 
+Michael Scherer
+
+