[Mageia-dev] Proposal for Mageia: implement bitorrent protocol to allow updates download

Thu Jan 13 02:30:05 CET 2011

Olivier Thauvin a écrit :
+> * andre999 (andr55 at laposte.net) wrote:
+>> Romain d'Alverny a écrit :
+>>>
+>>> On Wed, Jan 12, 2011 at 04:29, Michael Scherer<misc at zarb.org>   wrote:
+>>>
+>>> However, if 1) was to open 5 connections on 5 distinct servers, that
+>>> would make more sense, no?
+>>
+>> Right.
+>> Another way to look at the question :
+>> If 1000 people are downloading from mirrors allowing a total of 2000
+>> connexions, if no-one uses multiple connexions, then 1000 connexions are
+>> wasted.  These unused connexions would likely be from faster mirrors.
+>
+> The upload given by a serveur is split in term of:
+> - connection available
+> - bandwidth available.
+>
+> If the mirrors is connected to Gb (which is more likelly the size of
+> bandwidth for the whole university...), and you split this Gb/s onto
+> 2000 connection you obtain 500kb/s.
+
+Please note that my example was hypothetical, for multiple sites :)
+Sorry if that wasn't clear.  I don't know of _any_ single sites able to 
+do that.  (Although I don't doubt that they exist.)
+(When I started the example, I wrote 20 mirrors, but then erased it 
+because it didn't seem relevant to my point.)
+
+> However, I know only few server in the world really able to read Gb/s
+> from their disk. The top rate I obtain on dc is 400mbis/s (the memory
+> cache help a lot to obtain more).
+>
+> The number of connection is set high because some people download at
+> only 50kb/s and other 2Mbit/s, so the spare bandwidth from the 50kb/s
+> can be given to others. Nothing more.
+
+exactly
+
+> According the graph on distrib-coffee:
+> http://distrib-coffee.ipsl.jussieu.fr/munin/distrib-coffee/distrib-coffee/index.html
+> there is an average of 100 http download at a time, if each connection
+> become 5 (500), you'll reach the limit (200 on this server).
+> What will be the gain on your side ?
+
+In that context, obviously.  The idea wasn't that most sites would have 
+5 connexions - rather to allow up to 5 (depending on the traffic at the 
+moment.  I would expect most sites to have one or 2 connexions.  And 
+preferably use multiple mirrors, as recommended by aria2c documentation.
+
+> I talked here only about http, but apply the same to ftp in the same
+> internet connection, so it mean I have 100mbits/s to share between 400
+> connections.
+>
+> As Michael explain, I voluntary limit the count of connection per IP,
+> before I did this, the server was overload 12 hour a day, which mean
+> stop serving !
+
+A very good idea.  I would do the same thing in your situation.
+Actually, aria2c would work somewhat better if all sites with more 
+limited capacity did the same thing.  Because the benefit comes from 
+making use of the unused capacity (of the moment) of mostly larger 
+mirror sites.  And trying to access sites already at capacity slows 
+somewhat the downloading.
+Unfortunately not all download software monitors the connexion speed to 
+dynamically choose the fastest connexions, as does aria2c.
+
+>> With aria2c, 3 mirrors which support a total of 5 connexions gives me
+>> optimal speed.  (The limit being the speed of my computer.)
+>
+> Can we know the speed of your connection ?
+
+Fast, but I don't know exactly.  It is at the local library, via wi-fi. 
+  (I use it for downloading ISOs because my home connexion has limited 
+bandwidth.)
+It is on the municipal network with telephony and all, and I access it 
+in the evening, when essentially only the library is open.  I have seen 
+another user downloading at over 900 KB/sec while I was downloading, at 
+a somewhat slower speed.  (A while back.)
+
+At the library with up to 5 connexions on 2 or 3 mirrors, it is always 
+over 600 KB/sec, often over 700 KB/sec.  The last time I did a 
+comparison (1 connexion/1 mirror vs max 5 connexions/2 mirrors) for a 
+DVD, the ratio of speed was approx 3,5.  (From summary notes on my 
+computer.)
+
+> Maybe your connection is good enough to provide a mirror for Mageia ?
+> And then you'll be able to test what you're suggesting.
+
+Unfortunately, my home connexion tops out at 400KB/sec (officially 
+512KB/sec), but usually is less than half that.  (Variable-speed slow 
+DSL.)  And the upload speed is severely limited by my FAI, as well as a 
+limited bandwidth.
+Wouldn't mind trying a mirror when better connexions become available.
+(Don't know if I would go for something as big as Mageia, though.)
+
+> What is the result with 5 mirrors and 1 connection per mirror ?
+At the library, it tops out with 3 mirrors and 5 connexions.  The best 
+was over 700KB/sec.  (4 mirrors wasn't any faster.)
+(I borrowed a faster computer from someone once, and it averaged well 
+over 800 KB/sec on 1 mirror/5 connexions, for download of a DVD.)
+
+> Obviously, any mirror in France (I am in France) is able to fulfill
+> my personnal connection... (10mbits/s).
+
+J'aimerais bien ça :)
+Where I live (banlieue de Montréal), I couldn't get more than 2Mbits/s 
+(officially), if that.  And fairly expensive, as well.  With very 
+limited upload speeds.
+
+-- 
+André
+