summaryrefslogtreecommitdiffstats
path: root/zarb-ml/mageia-webteam/2011-January/000148.html
diff options
context:
space:
mode:
authorNicolas Vigier <boklm@mageia.org>2013-04-14 13:46:12 +0000
committerNicolas Vigier <boklm@mageia.org>2013-04-14 13:46:12 +0000
commit1be510f9529cb082f802408b472a77d074b394c0 (patch)
treeb175f9d5fcb107576dabc768e7bd04d4a3e491a0 /zarb-ml/mageia-webteam/2011-January/000148.html
parentfa5098cf210b23ab4f419913e28af7b1b07dafb2 (diff)
downloadarchives-1be510f9529cb082f802408b472a77d074b394c0.tar
archives-1be510f9529cb082f802408b472a77d074b394c0.tar.gz
archives-1be510f9529cb082f802408b472a77d074b394c0.tar.bz2
archives-1be510f9529cb082f802408b472a77d074b394c0.tar.xz
archives-1be510f9529cb082f802408b472a77d074b394c0.zip
Add zarb MLs html archivesHEADmaster
Diffstat (limited to 'zarb-ml/mageia-webteam/2011-January/000148.html')
-rw-r--r--zarb-ml/mageia-webteam/2011-January/000148.html324
1 files changed, 324 insertions, 0 deletions
diff --git a/zarb-ml/mageia-webteam/2011-January/000148.html b/zarb-ml/mageia-webteam/2011-January/000148.html
new file mode 100644
index 000000000..fcfcd4ee0
--- /dev/null
+++ b/zarb-ml/mageia-webteam/2011-January/000148.html
@@ -0,0 +1,324 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+ <HEAD>
+ <TITLE> [Mageia-webteam] Forum VM needs
+ </TITLE>
+ <LINK REL="Index" HREF="index.html" >
+ <LINK REL="made" HREF="mailto:mageia-webteam%40mageia.org?Subject=Re%3A%20%5BMageia-webteam%5D%20Forum%20VM%20needs&In-Reply-To=%3C1294921744.6060.0.camel%40akroma.ephaone.org%3E">
+ <META NAME="robots" CONTENT="index,nofollow">
+ <META http-equiv="Content-Type" content="text/html; charset=us-ascii">
+ <LINK REL="Previous" HREF="000150.html">
+ <LINK REL="Next" HREF="000153.html">
+ </HEAD>
+ <BODY BGCOLOR="#ffffff">
+ <H1>[Mageia-webteam] Forum VM needs</H1>
+ <B>Michael Scherer</B>
+ <A HREF="mailto:mageia-webteam%40mageia.org?Subject=Re%3A%20%5BMageia-webteam%5D%20Forum%20VM%20needs&In-Reply-To=%3C1294921744.6060.0.camel%40akroma.ephaone.org%3E"
+ TITLE="[Mageia-webteam] Forum VM needs">misc at zarb.org
+ </A><BR>
+ <I>Thu Jan 13 13:29:04 CET 2011</I>
+ <P><UL>
+ <LI>Previous message: <A HREF="000150.html">[Mageia-webteam] [Mageia-sysadm] Forum VM needs
+</A></li>
+ <LI>Next message: <A HREF="000153.html">[Mageia-webteam] Forum VM needs
+</A></li>
+ <LI> <B>Messages sorted by:</B>
+ <a href="date.html#148">[ date ]</a>
+ <a href="thread.html#148">[ thread ]</a>
+ <a href="subject.html#148">[ subject ]</a>
+ <a href="author.html#148">[ author ]</a>
+ </LI>
+ </UL>
+ <HR>
+<!--beginarticle-->
+<PRE>Le mercredi 12 janvier 2011 &#224; 21:27 +0100, Ma&#226;t a &#233;crit :
+&gt;<i> Hi there,
+</I>&gt;<i>
+</I>&gt;<i> As it seems VM creation takes a little bit of time due
+</I>&gt;<i> to people being under heavy load at work Anne and misc
+</I>&gt;<i> considered the option of creation the Xen VM on one of
+</I>&gt;<i> our servers (we could migrate the VM on atalante later)
+</I>
+The exact technology should not matter much, that's also what puppet is
+made for. Ie, unless we plan to do a migration at the system image
+level, we could simply install the 2nd vm, put puppet, clone the
+computer, migrate the db and ip.
+
+( not that I do not like xen, but I would prefer something else ).
+
+&gt;<i> For that misc asked for Forum needs...
+</I>
+I think I didn't make myself clear. I wanted information to deploy it
+like where is the git stored ( a url, not &quot;it is on a server&quot; ), who
+will need what access, etc. But the information you gave are also
+important ( and bring lots of question as you can see ).
+
+&gt;<i> For the beginning i'll consider that we are going to put everything on the same machine
+</I>&gt;<i> (DB and PHP). This is not rally brilliant to virtualize DB servers but i guess this will
+</I>&gt;<i> not kill the VM in the first monthes as the tables will not be big.
+</I>
+AFAIK, using virtio and proper cache, this should not be much a problem.
+
+&gt;<i> So phpBB needs a LAMP Stack : Apache + PHP5 + MysSQL5 (it prefers to have MySQLi extention)
+</I>
+No specific requirement in term of version, using 2010.1 rpm should be
+ok, I assume ?
+
+&gt;<i> And we'll need with php the optional :
+</I>&gt;<i> -- zlib compression (better having it)
+</I>&gt;<i> -- remote ftp support (well... i'm not in favor even if documentation asks for it)
+</I>We could drop outgoing connexion if needed.
+
+( yes, php make me paranoid in term of security )
+
+&gt;<i> -- XML support (better having it)
+</I>&gt;<i> -- Image Magick support (better having it)
+</I>
+php-image-magick. I do think there is a conspiration to make me have a
+stroke. Security research by a friend of mine on ImageMagick do not make
+feel safe to know we will use it, but if this is required, we have no
+choice.
+
+Just to know, what will it be used for ( I assume this will be used to
+resize avatar ) ?
+
+&gt;<i> -- GD support (same as Image magick)
+</I>
+Does the forum support suhoshin, or various php hardening measures ?
+
+Did you do various testing with a hardened configuration with dangerous
+call disabled ( mainly remote url access for a start, but i also think
+we can use opendir restriction, etc, etc ).
+
+Does it have non regression testing ( so we can enable stuff and see if
+anything break ? ) ?
+
+&gt;<i> For source management git will be used... so we'll need it too :)
+</I>
+Just git clone ?
+I have a puppet module for this, just need tests before I commit.
+
+For git hosting, again, while I am in favor, there is a few questions to
+answer and prepare it, see my previous mail about what is needed.
+
+&gt;<i> As forum have often to face bruteforce having Fail2ban would be really great...
+</I>&gt;<i> for every open service like ssh
+</I>
+On ssh level, and for me, that's a vote in favor of &quot;no&quot;. We use ssh
+keys only for admins, so fail2ban will just cause trouble.
+
+&gt;<i> but also for forums... i'd like to have Fail2ban
+</I>&gt;<i> parse a file of phpBB failed login to trigger a IP low level ban during a
+</I>&gt;<i> few hours or more...
+</I>
+Well, if you give us the configuration, we can see.
+We can also use the trick that Olivier deployed on d-c to avoid numerous
+connexions from the same IP ( in case someone decide to be smart and do
+simultaneous attempts to log ).
+
+&gt;<i> For forum management we'll need :
+</I>
+&gt;&gt;&gt;<i> we
+</I>Traceback (most recent call last):
+ File &quot;&lt;stdin&gt;&quot;, line 1, in &lt;module&gt;
+NameError: name 'we' is not defined
+
+or for those who are not CxO-fluent ( private joke ), who is 'we', in
+term of organisation ( ie, do we need to create a ldap group, etc ) ?
+
+&gt;<i> -- access to sources (read/write)
+</I>I rather keep this automated from git, for security reasons and to avoid
+human errors. I would even add a cron job that does a git diff or
+something similar, to detect if someone uploaded a file manually, or
+touched to it using apache.
+
+In fact, as a security measure, I think the user that will write the
+source should put it read only for apache. Ie, use a separate system
+user for that.
+
+&gt;<i> -- access to data zones (avatars and uploaded things) (read/write)
+</I>You mean apache will need it, no ?
+
+Direct access seems to me a pretty rare event, we can grant access if
+there is a really lots of request, ie if you annoy admin enough to make
+them give it rather than doing themselves.
+
+&gt;<i> -- access to accesslogs and errorlogs (read)
+</I>then this should be merged with the webmasters concept that romain
+explained. For now, we didn't setup anything ( we didn't even split log
+file on alamut, even if this should be trivial ).
+
+&gt;<i> -- ability to change php log levels
+</I>This can be done by php, I think.
+
+&gt;<i> -- access to php logs (read)
+</I>same as accesslogs
+
+&gt;<i> -- console access to database(s) (i'd prefer to avoid completely phpMyadmin on the forum server)
+</I>I would prefer avoid giving console access until there is a real need. I would favor then a remote
+mysql access, and forcing ssl, maybe even limited by fixed ip address if you wish to avoid bruteforce.
+
+( I will not go to the point of proposing to use a vpn too, but
+almost ).
+
+Maybe we could think of some kind of ssh bastion for such access ( or
+maybe that's overkill too ).
+
+&gt;<i> For performance questions : i guess forum opening will trigger a rather vast
+</I>&gt;<i> amount of people coming (at least to register their nicks)... i'd be happy to
+</I>&gt;<i> avoid the server being loaded to death.
+</I>
+Registration will be done on catdap from what I think we agreed on, no ?
+( correct me if I am wrong ).
+
+So we need to work on that part ( starting more processes, and so
+letting us tune that with puppet ( this is hardcoded now, AFAIK ). So
+depending on where we host the forum, we can surely avoid this effect.
+
+&gt;<i> So i'm targetting at least one thousand simultaneous users being active on the
+</I>&gt;<i> forum... that will do for apache tuning.
+</I>
+Ok so let's say 120 simultaneous process for apache, which also mean we
+need to keep apache process as lean as possible ( ie, no unused module
+loaded ). I assume that there is no guarantee on being thread safe from
+php and associated library, so we will use mpm-prefork.
+
+Since the server is isolated and serve only for php hosting, I guess
+using fast-cgi will not bring much to the equation, when compared to
+mod_php.
+
+Let's also assume 30 processes for forum registration on catdap ( if I
+am not wrong on that part, of course ) ? We could surely mitigate the
+potential overload by not announcing this on every possible channel at
+the same time ( ie, first a mail, then a blog post, then
+identica/twitter ).
+
+Should we also maybe need to tune ldap ?
+
+&gt;<i> For database that will mean 800 to 1200 requests per seconds...
+</I>&gt;<i>
+</I>&gt;<i> We'll have 2 - 3 months to see the tables grow and tune the indexes and the memory accordingly.
+</I>
+That mean that we will have to deploy some monitoring, and we didn't
+decided anything ( buchan proposed hobbit, I proposed munin, purely by
+familiarity ).
+
+What metrics would you need so we can work on them in priority ( once we
+start to set up something ) ?
+
+&gt;<i> But i think our needs will stabilize around 4-6 GO for RAM if the forum gets really
+</I>&gt;<i> used (we'll have to tune mysql to keep many requests in cache) apache+mysql all
+</I>&gt;<i> included... if we split later apache and mysql on separate machines the needs on
+</I>&gt;<i> each machine will be obviously lower.
+</I>
+No cache ( squid, varnish ) ?
+No php level cache too ?
+
+( not that it may be requested now )
+
+&gt;<i> For app disk space code is under 50 megs... and with hundred of avatars uploader
+</I>&gt;<i> we will not grow above 1GO
+</I>&gt;<i>
+</I>&gt;<i> For database disk space even after years of activity we'll remain under 5GO
+</I>
+Ok so let's allocate a 10 g partition for the db + ssthat on lvm.
+We should take in account logs, and logs backup ( french law requires 1
+year of logs ).
+
+How many logs are to be expected per day ?
+The only busy webserver I can think of is d-c, but Nanar and I just
+discovered that the configuration is not good.
+So now, that's 5g of log, uncompressed, per month.
+
+
+&gt;<i> We'll need to set up some tables with heavy read and write accesses with InnoDB (not all) :
+</I>&gt;<i> that would be great to have one file per table innodb option enabled
+</I>
+Ok, I guess it should be safe to enable it for all mysql db I guess.
+
+&gt;<i> Nota : i'd like to use https (at least for admin accesses)... so that will mean to enable
+</I>&gt;<i> ssl and open 443 port also
+</I>
+We did not plan to let people use their password under cleartext at all.
+Centralized auth have been setup ( and should be used for forum too ),
+so people will reuse their password, the same used at others part of the
+infrastructure, and that mean svn, or bugzilla, etc. Since people with
+access will use it, no cleartext at all when the password is sent ( or
+over my dead body, after fighting my ghost ).
+
+I guess we can make exception for the cookie, as long as it is not
+shared ( ie, we will have to rethink the scheme if we deploy SSO ).
+
+That also mean that people will complain because of firefox if we do not
+buy a certificate.
+
+
+&gt;<i> That's all for system level... i think directory structures (Which concerns apache web root config) can be dealt with later...
+</I>&gt;<i>
+</I>&gt;<i> Tell me if you have got everything you need for VM creation...
+</I>
+What I needed was more information for forum deployment, not vm
+requirements, I guess I didn't express myself clearly. The requirement
+for the vm in term of memory/disk have been roughly drafted before. What
+I would prefer is a deployment document.
+
+Ideally, I would also prefer that discussion regarding forum development
+occurs on public ml ( for webteam here ) rather than private mail
+exchange.
+
+So @sysadmins, where do we host this ( for temporary creation and setup
+until MLO do it ) :
+
+10 g of disk ( let's say 20 )
+4 to 6 g of ram
+( and I guess 1 or cpu ).
+
+Alamut ?
+
+
+In term of network, do we use a reverse proxy, or do we ask for a ip to
+LO. I would suggest a reverse proxy, easier to setup ( and no fiddling
+with bridge ).
+
+
+What virt technology ?
+
+I am quite biased in favor of libvirtd + kvm + virt-manager, as this is
+supported by upstream and redhat, but maybe other have different
+experiences.
+
+--
+Michael Scherer
+
+
+</PRE>
+
+
+
+
+
+
+
+
+
+
+<!--endarticle-->
+ <HR>
+ <P><UL>
+ <!--threads-->
+ <LI>Previous message: <A HREF="000150.html">[Mageia-webteam] [Mageia-sysadm] Forum VM needs
+</A></li>
+ <LI>Next message: <A HREF="000153.html">[Mageia-webteam] Forum VM needs
+</A></li>
+ <LI> <B>Messages sorted by:</B>
+ <a href="date.html#148">[ date ]</a>
+ <a href="thread.html#148">[ thread ]</a>
+ <a href="subject.html#148">[ subject ]</a>
+ <a href="author.html#148">[ author ]</a>
+ </LI>
+ </UL>
+
+<hr>
+<a href="https://www.mageia.org/mailman/listinfo/mageia-webteam">More information about the Mageia-webteam
+mailing list</a><br>
+</body></html>