diff options
Diffstat (limited to 'zarb-ml/mageia-sysadm/2010-October/000038.html')
| -rw-r--r-- | zarb-ml/mageia-sysadm/2010-October/000038.html | 329 |
1 files changed, 329 insertions, 0 deletions
diff --git a/zarb-ml/mageia-sysadm/2010-October/000038.html b/zarb-ml/mageia-sysadm/2010-October/000038.html new file mode 100644 index 000000000..b21bf9b03 --- /dev/null +++ b/zarb-ml/mageia-sysadm/2010-October/000038.html @@ -0,0 +1,329 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> +<HTML> + <HEAD> + <TITLE> [Mageia-sysadm] [LONG] A not so modest proposal + </TITLE> + <LINK REL="Index" HREF="index.html" > + <LINK REL="made" HREF="mailto:mageia-sysadm%40mageia.org?Subject=Re%3A%20%5BMageia-sysadm%5D%20%5BLONG%5D%20A%20not%20so%20modest%20proposal&In-Reply-To=%3C1287878118.22938.155.camel%40akroma.ephaone.org%3E"> + <META NAME="robots" CONTENT="index,nofollow"> + <META http-equiv="Content-Type" content="text/html; charset=us-ascii"> + <LINK REL="Previous" HREF="000072.html"> + <LINK REL="Next" HREF="000067.html"> + </HEAD> + <BODY BGCOLOR="#ffffff"> + <H1>[Mageia-sysadm] [LONG] A not so modest proposal</H1> + <B>Michael Scherer</B> + <A HREF="mailto:mageia-sysadm%40mageia.org?Subject=Re%3A%20%5BMageia-sysadm%5D%20%5BLONG%5D%20A%20not%20so%20modest%20proposal&In-Reply-To=%3C1287878118.22938.155.camel%40akroma.ephaone.org%3E" + TITLE="[Mageia-sysadm] [LONG] A not so modest proposal">misc at zarb.org + </A><BR> + <I>Sun Oct 24 01:55:18 CEST 2010</I> + <P><UL> + <LI>Previous message: <A HREF="000072.html">[Mageia-sysadm] planning for sysadmin task +</A></li> + <LI>Next message: <A HREF="000067.html">[Mageia-sysadm] [LONG] A not so modest proposal +</A></li> + <LI> <B>Messages sorted by:</B> + <a href="date.html#38">[ date ]</a> + <a href="thread.html#38">[ thread ]</a> + <a href="subject.html#38">[ subject ]</a> + <a href="author.html#38">[ author ]</a> + </LI> + </UL> + <HR> +<!--beginarticle--> +<PRE>Hi, + +so as I said in the previous mail ( and before ), I propose that we use +puppet+svn to manage the servers. Since this may not be obvious to +everybody here is a long mail with a long explanation of each point. +( spoiler, there is a bold proposal in the middle ). + + +So first, what is puppet ? +-------------------------- + +As I didn't found a better explanation than the one of their website, +let's cut and paste : + +"Puppet is an open source data center automation and configuration +management framework. Puppet provides system administrators with a +simplified platform that allows for consistent, transparent, and +flexible systems management." + +If you have heard of cfengine ( used at mdv ), or bcfg2, puppet is +similar. It allows you to describe your computers in a configuration +file and take care of installing and setting them according the +configuration file. For example, you can say 'for server of group web +server, install apache php, and use this config file, and start this +process'. Usually, such system are combined with a regular vcs like svn, +for reason that I will outline later. + +To quickly explain how it work, there is a central server ( called +"puppetmaster" for puppet ) and a agent on each computer. Agents fetch +the config from puppetmaster after a configurable interval of time and +apply it ( ie, install rpm, change configfile, reload software, run +tasks, etc ). + + +So why use a configuration management system and a vcs ? +-------------------------------------------------------- + +First, it provides use with a audit trail. Like for code, we know who +changed what and why. The goal is not to distribute blame, but to be +able to have more information about a change ( ie, if 2 years ago, I +changed some php variable that later found out to break some web +software, the changelog will tell us why it was changed in the first +place, maybe to fix another important software ). It allows us to repeat +a configuration ( useful to migrate a service or to reinstall a +server ), to rollback to previous versions of the configuration, and to +work concurrently. + +It also ensure that process are running which add a extra safety layer +in case of problem. And we can use hooks to check file before applying +them automatically, or send mail to the admin when there is a change. +For example, you can no longer commit broken dns zone since they are +checked them before applying automatically ( provided you write the +proper script, of course ). + + +Why use puppet and not $FOO ? +----------------------------- + +That's a good question. First, let's be honest, I take care of puppet in +mdv, so of course, I am biased. In term of softwares, there is lots of +choice +( <A HREF="http://en.wikipedia.org/wiki/Comparison_of_open_source_configuration_management_software">http://en.wikipedia.org/wiki/Comparison_of_open_source_configuration_management_software</A> ), but reading on the topic on the web, we can safely restrict our discussion to cfengine 2, cfengine 3, puppet, bcfg2, chef. + +Chef is not packaged, nor easy to setup ( imho ) unless you use gems +( which is a bad idea ). The configuration is basically written ruby. It +seems nice, but no one played with it around me ( except people at +CERN ). + +Cfengine 2 is what is used at zarb, and also lightly used at mandriva. I +took a look at mandriva configuration, it is mainly used to prevent +service from starting and to manage fstab. So nothing that could prevent +a migration and in fact, almost nothing to reuse. The version 2 is also +the legacy branch, and I am not sure it is maintained ( guillomovitch +may know better than me ). +So far, I think boklm, blino, me and maybe nanar know it. + +Bcfg2 is not packaged, and use xml for config file, which is imho a bad +point for it. I have some friends that use it, they do not seems unhappy +with it. yet, I found few information about it besides the web site. + +Cfengine3 is not packaged either. And the configuration language is +different from cfengine 2, which would mean that we will likely have to +re-learn it, for those that know the older version. + +Puppet is packaged ( by me ). It is used by several free software group +( afaik, redhat and mozilla among others ), maintained by a enterprise +and there is a healthy community providing modules and software around +it. From what I know, Nanar and I know how to use it. Boklm also started +to look at it. + +So I think we should first use a packaged software, for various obvious +reasons. So the choice is basically between puppet and cfengine2. I +asked to guillomovitch, our zarb.org cfengine expert about the migration +to cfengine3, and he told me he was planning to use puppet instead. And +having used both in production ( either zarb.org or my own server ), I +think puppet is nicer and provides more high level component than +cfengine 2, and is maintained. And since there will be almost no +configuration reuse from Mandriva, I do not think it will be worth to +keep it. + + +Why use svn and not $FOO ? +--------------------------- + +Again a good question. At zarb, we use svn. At mdv, afaik, we used +nothing, cfengine was just here to automate the cluster setup, which is +a similar yet different task than the one we are discussing now. So, as +I prefer DVCS, I tested git + puppet for you, 6 months ago. And after 3 +weeks, I have migrated to svn the whole system. The git hooks are just +too complex for my poor mind ( ie, checking the syntax of puppet, bind, +etc before applying was not easy ). So since svn is quite straight +forward with that, and since we can always use git-svn to have the best +of both world, I suggest to use svn. + +( yes, hg, bzr, cvs and the gazillion others were not suggested ). + +Now, if someone is a git-hooks master, I have no problem with git. + +What would be the work flow ? +----------------------------- + +So the idea is : +- a admin commit some config change in svn repository, +- svn check the syntax if needed ( pre-commit hook ) +- svn extract the configuration with a post-commit hook ( and send a +email ), +- puppetmaster notice the change, +- puppetmaster reload the configuration +- each server get the configuration after some time +- each server apply the configuration, and send mail in case of problem + +We can also add ACLs if needed. + +Could we also use svn to manage the documentation ? +--------------------------------------------------- + +Indeed, that's a good suggestion. I have learned that the documentation +is indeed something that I never find when I need, and I have been often +frustrated by the lack of useful tools to exploit it ( lik grep ). So, +to me, it make sense to have the documentation along the configuration +since I always have a checkout of the configuration on my home, and +since I often update it. ( which is usually not the case with most +wikis, as this requires extranous step ). + + +Where is the bold proposal ? +----------------------------- + +And now, for the very very bold part of the proposal : + +this svn should be public. Ie, people could browse it, changelog should +be sent on this ml ( which is public ), and people could even do +anonymous checkout. + +So I expect three type of reactions at this part of the mail : + +People who say "mhh, I do not understand what he speak about". + +People who say "this make sense, we are doing free software". + +And admins who are screaming "argh, what about the passwords and +security !" + + +So for people who do not understand, well, either reread or ask me on +irc, or by mail. There is no shame into asking questions. + +For people who are screaming, first, no need to scream, I cannot hear +you. And I have of course thought of that, and we can either : + +1) do a search and replace in puppet config for password stored in a +secure location in the svn hooks that do the checkout + +2) use extlookup, a puppet feature that do this, provides we use it +properly +( <A HREF="http://www.devco.net/archives/2009/08/31/complex_data_and_puppet.php">http://www.devco.net/archives/2009/08/31/complex_data_and_puppet.php</A> ). + +Ie, we can store the passwords in a private CSV file somewhere, while +publishing everything else. Obviously, stuff that count as data ( such +as private certificates, gpg keys, etc ) should not be stored in svn, or +not in the public one. And everything that would be considered as +personal information should not be stored in the public svn either. + +And so for those that think, "this make sense", yes, it make sense, but +more than simple making sense, it bring some advantages : + +Sharing is the basis of the free software ethics. And so that show our +commitment to free software. + +Publishing our configuration is also perfectly in line with the values +of the project : + +"We will empower our user base by demystifying advanced technologies" +-> publishing our work as sysadmin is a step toward demystifying + +"Mageia will always adhere to high security and privacy +standards/technologies to protect our users' data." + +-> letting everybody audit is IMHO a high security standard, while +security by obscurity is not. We always say that free software is better +for security for this reason, so we are consistent with this idea. + +"We will cooperate with other OSS distributions and core and kernel +developers with code contribution." + +-> if the configuration is treated like code, so we are cooperating by +giving it. For other distribution that may want to know how we do, be it +big distros or smaller ones. For various project that want to know how +we handle specific part of the infrastructure. + +"We will maintain the vibrancy within our Community, always aiming to +lead the way in collaborative development." + +-> people are eager to help us as sysadmin. I received lots of help +propositions. However for security reasons, I do not think we should +have a group of 20 persons, neither we should have a too complex +organisation for admins. Publishing the config allows us to collaborate +with others by letting them : + - review our changes, as the svn changelog on cooker + - directly send patches, which can be much clearer to us + - directly search when they see a problem ( for very simple problem of +course ) + - help us to communicate with everybody when we fix problem or do some +changes. + +More ever, this allow us to see who is motivated to help us, who is able +to understand our infrastructure, and therefor, this can help us to +recruit new admins in case of need, and allows us to manage the erosion +of our group. This also bring a gentle introduction to new comers, who +can see what happen. + +Finally, using this will allow us to have a forkable infrastructure. And +this would a real innovation I think ( at least, a innovation good +enough to be communicated ). + +I do not plan to let the project go bad, of course, but I also didn't +intend Mandriva to disappear either. + +So in the future, if something goes wrong for any kind of reason, or if +some peoples prefer to work without us, we are offering them what +Mandriva didn't offered, a easy way to fork. And while I do hope it will +not matter much in practice, I think that's a strong statement to +demonstrate that we learned from our errors and that we have evolved. + + +So, WDYT ? +( please do not let me be warnocked +<A HREF="http://en.wikipedia.org/wiki/Warnocked">http://en.wikipedia.org/wiki/Warnocked</A> ) + +PS : baud, as I know you will ask me later and read this mail, yes, you +can use use under CC-BY-SA or what you want. + +-- +Michael Scherer + +</PRE> + + + + + + + + + + + + + + + + + + + + +<!--endarticle--> + <HR> + <P><UL> + <!--threads--> + <LI>Previous message: <A HREF="000072.html">[Mageia-sysadm] planning for sysadmin task +</A></li> + <LI>Next message: <A HREF="000067.html">[Mageia-sysadm] [LONG] A not so modest proposal +</A></li> + <LI> <B>Messages sorted by:</B> + <a href="date.html#38">[ date ]</a> + <a href="thread.html#38">[ thread ]</a> + <a href="subject.html#38">[ subject ]</a> + <a href="author.html#38">[ author ]</a> + </LI> + </UL> + +<hr> +<a href="https://www.mageia.org/mailman/listinfo/mageia-sysadm">More information about the Mageia-sysadm +mailing list</a><br> +</body></html> |