[Mageia-dev] Proposal for Mageia: implement bitorrent protocol to allow updates download

Tue Jan 18 09:37:12 CET 2011

Motoko-chan a écrit :
+>
+>
+>> However I feel that
+>> 4) It is reasonable for downloaders to use multiple connexions per
+>> mirror, as long as they don't bypass the controls put in place by the
+>> mirror. Not only reasonable, but actually desirable in terms of using
+>> otherwise unused download capacity that occurs from time to time, and
+>> thus enhancing general download access.
+>
+> This method only works in the following situations:
+>
+> - The mirror limits the speed of each individual connection
+> - You use multiple mirrors to download
+> - If applying to a single file across multiple mirrors, the mirrors
+> support transfer of selective portions of the requested file
+>
+> In the first situation, the mirror maintainer usually has a good reason
+> for limiting connection speed. If their limit is very low, you should
+> contact the project they are mirroring and suggest they be dropped as an
+> official mirror. Going around admin-imposed limitations is bad manners
+> at the very least and will either get you banned or cause further
+> restrictions that will harm everyone.
+
+Many mirror sites do restrict multiple connexions (per user), so 
+evidently it is readily done.  If a mirror chooses not to make such 
+restrictions, that is their choice.  It is not good form (to put it 
+mildly) for some mirrors to try to dictate mirror policy for other 
+mirrors, which could have different interests.
+Limiting the connexion speed could be an easy way to ensure that usually 
+everyone who wishes to connect can do so in busy times.  In less busy 
+times, multiple connexions to such mirrors causes no problem.
+Software such as aria2c monitors the download speed of connexions, and 
+will avoid slower connexions, thus balancing use of available download 
+ressources between mirrors.
+
+> In the second situation, you are now taking the bandwidth of multiple
+> mirrors, increasing the overall load on the mirror network. Use of four
+> mirrors means you just took up resources that three other people could
+> have been using. Effectively, you're being greedy to the detriment of
+> the community.
+
+So for 2 hours one uses 4x download ressources instead of 8 hours of x 
+download ressources. In both cases using 8x ressources.
+A simple elementary school calculation.
+For large files, such as ISOs, it makes a big different to the user.
+Globally, it makes no difference collectively to the mirrors.  Except 
+that downloading will tend to be balanced between mirrors if software 
+such as aria2c is used.  And mirrors with free download ressources are 
+more likely to be more fully used.
+(One could always quibble about the overhead per connexion, but that is 
+relatively minor compared to the download itself.)
+
+> The third situation presumes a certain configuration that isn't
+> necessarily true. It also has the same problems as the second situation.
+
+The third case is generally true for ftp mirrors, as well as many http 
+mirrors.  Where it is not, aria2c and similar software will not use the 
+mirror.
+
+Don't forget that the major target of multiple connexions will be the 
+larger/faster mirrors, thus taking load off smaller mirrors.  (Since 
+applications like aria2c favour faster connexions.)
+
+>> There is another factor that may not have been considered. Many
+>> downloaders will be slowed by their internet access or other
+>> non-mirror factors, which will very much increase the likelihood that
+>> mirrors have unused bandwidth at various times. Increasing the utility
+>> -- from both sides -- of multiple connexions.
+>
+> That is an argument against multi-mirror downloads. Most mirror servers
+> will have good bandwidth for their region. In general, the user's
+> connection will be a bottleneck. The overhead of maintaining multiple
+> connections can actually _slow down_ the downloads.
+
+So you believe that unused download bandwith should not be used ?
+Sounds like that would _really_ help downloaders -- and other mirrors.
+
+Among the potential non-mirror factors which could limit the download 
+speed is the capacity of the download path.  Multiple connexions spread 
+the download over multiple paths, minimising this potential bottleneck.
+
+> As for torrents, there exists a large problem in that there are several
+> ways to handle such things (torrent per file, torrent per directory,
+> etc) and on every change you would need the torrent to be regenerated
+> and seeds to be established. If you are doing this on something with a
+> lot of file churn such as a development trunk, you run into a ton of
+> management overhead for handling the torrents. Even if you start with
+> web seeds, you now strain your mirrors by requesting that they be active
+> in seeding a bunch of torrents.
+>
+> Bittorrent makes good sense in one big situation: you have a single
+> large or multiple files that will be widely downloaded and distributed,
+> and the downloaders are likely to leave their download tool (torrent
+> application) running after download. This is especially useful when it's
+> going to be a large, but temporary, demand such as ISO files at release
+> time. While some mirrors might be able to handle a 200-300% increase in
+> traffic, others may not. It's expensive to provision at that capacity
+> when such a thing only happens twice a year for a week or maybe two. In
+> such cases, torrents make very good sense as they help reduce the peak
+> demand on your mirrors.
+
+Evidently you understand the bittorent issue.
+Note that it is essentially those with faster download/upload speeds 
+that act as secondary mirrors that provide the benefit.  Many users 
+can't offer that.
+Interesting that you acknowlege the variation in demand in discussing 
+bittorent.  Which implies that some capacity is not always used.  Which 
+applies to the multiple connexion issue.
+
+> - Michael
+
+Regards :)
+
+-- 
+André
+