1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<TITLE> [Mageia-sysadm] Update procedure on sysadmin side
</TITLE>
<LINK REL="Index" HREF="index.html" >
<LINK REL="made" HREF="mailto:mageia-sysadm%40mageia.org?Subject=Re%3A%20%5BMageia-sysadm%5D%20Update%20procedure%20on%20sysadmin%20side&In-Reply-To=%3C20110722213759.GB30226%40sisay.ephaone.org%3E">
<META NAME="robots" CONTENT="index,nofollow">
<META http-equiv="Content-Type" content="text/html; charset=us-ascii">
<LINK REL="Previous" HREF="003791.html">
<LINK REL="Next" HREF="003793.html">
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1>[Mageia-sysadm] Update procedure on sysadmin side</H1>
<B>Michael scherer</B>
<A HREF="mailto:mageia-sysadm%40mageia.org?Subject=Re%3A%20%5BMageia-sysadm%5D%20Update%20procedure%20on%20sysadmin%20side&In-Reply-To=%3C20110722213759.GB30226%40sisay.ephaone.org%3E"
TITLE="[Mageia-sysadm] Update procedure on sysadmin side">misc at zarb.org
</A><BR>
<I>Fri Jul 22 23:37:59 CEST 2011</I>
<P><UL>
<LI>Previous message: <A HREF="003791.html">[Mageia-sysadm] Update procedure on sysadmin side
</A></li>
<LI>Next message: <A HREF="003793.html">[Mageia-sysadm] Update procedure on sysadmin side
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#3792">[ date ]</a>
<a href="thread.html#3792">[ thread ]</a>
<a href="subject.html#3792">[ subject ]</a>
<a href="author.html#3792">[ author ]</a>
</LI>
</UL>
<HR>
<!--beginarticle-->
<PRE>On Fri, Jul 22, 2011 at 05:53:57PM +0300, Thomas Backlund wrote:
><i> Michael Scherer skrev 20.7.2011 01:26:
</I>><i> >Le mardi 19 juillet 2011 à 22:50 +0200, Michael Scherer a écrit :
</I>><i> >>Hi,
</I>><i> >>
</I>><i> >>while trying to see how to do update ( since the list is quite long :
</I>><i> >><A HREF="https://bugs.mageia.org/buglist.cgi?query_format=advanced&emailassigned_to1=1&order=Importance&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=qa-bugs%40ml.mageia.org&product=Mageia&emailtype1=substring">https://bugs.mageia.org/buglist.cgi?query_format=advanced&emailassigned_to1=1&order=Importance&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=qa-bugs%40ml.mageia.org&product=Mageia&emailtype1=substring</A> ), I searched bash history to find the procedure.
</I>><i> >>
</I>><i> >>So here is what should be used :
</I>><i> >>
</I>><i> >># /root/tmp/mgatools/mga-move-update 1 core me-tv
</I>><i>
</I>><i> Thinking a little about updates...
</I>><i>
</I>><i> Should we hardlink the rpms instead of moving them,
</I>><i> and postpone the removal from *_testing for a few days (or ~a week)
</I>><i> so all mirrors catch up...
</I>><i>
</I>><i> That would save the mirrors from re-downloading the stuff, and make
</I>><i> the updates available faster...
</I>
><i> It does not matter much for a simple update like logrotate, but a
</I>><i> kernel update or a full kde update would definately benefit from it.
</I>
I think that's negligeable when compared to the size of cauldron updates,
and that would be slightly more complex to develop.
This would also mean we keep then in updates_testing, thus having bigger
hdlists, which mean more rpms to parse, and more data to download by mirror
( since hdlists are redownloaded each time, since they are compressed and
I am doubtful about the rsync-friendliness of the whole compression ).
And since hdlists are downloaded by every users doing testing, it may have a
bigger impact.
Let's check some data ( likely bogus, but that's to illustrate ).
Imagine that a hdlist is 1.3 mo bigger due to keeping a kernel in updates testing.
( as said on <A HREF="http://permalink.gmane.org/gmane.linux.mageia.devel/1392">http://permalink.gmane.org/gmane.linux.mageia.devel/1392</A> ).
Let's say 1 update per day, and maybe more depending on when we clean updates_testing,
so that's 10 differents hdlists to download in the week, as we recreate it for
each update. Since the hdlist is compressed, I suspect that rsync may not
be very efficient when downloading it ( ie, do it from 0 each time ).
On the mirror, the hdlist will be downloaded by each mirror or users syncing,
let's say 10 people and 2 mirrors, that make 12.
Each syncing once per day, that make around 100 mo of bandwidth used just for the
hdlist, on one single mirror. Compare that to the size of the update itself, ie 30
mo for the kernel example.
You can do the math also for -debug.
And as Thomas said on irc, the hdlist issue will be likely worst due to kmod()
provides on rpm 4.9.
Now, there is a few assumption that can be discussed, and that would change
everything :
- hdlist and rsync. rsync and gzip do not mix well, but maybe we do have provision
against that. I asked to teuf but he was not sure, maybe nanar or tv can shed a
different light. Maybe we also no longer us hdlist.
- number of users. Given the size of the QA team, maybe 10 is more than our
wildest dream.
And of course, I used a worst case, and for stuff like tetex or vegastrike-data,
it would surely help to do a hardlink. But the hdlist issue would arise mostly
for rpms with lots of files ( kernel, kde ?, various documentation ? ), so that's
something that we cannot decide without at least doing some stats.
IE, while it would help for vegastrike ( best case ), we never updated it on
Mandriva, while there was lots of kernel update ( worst case, and on steroid due
to the kernel multiplication ).
My point is not that we should or should not do this, but rather that we should
first measure the impact, based on real data rather than trying to optimize
based on guess, because it may make things worst for mirrors ( more bandwidth used
in the long run ) and for us ( more complexity on the setup ).
><i> the downside is of course more complex code on staging/primary mirror
</I>
Yes, and I fear this may not be so trivial to do (ie, how do we know that
we need to remove a rpm from update_testing, since we cannot base on the creation
date due to the hardlink )
--
Michael Scherer
</PRE>
<!--endarticle-->
<HR>
<P><UL>
<!--threads-->
<LI>Previous message: <A HREF="003791.html">[Mageia-sysadm] Update procedure on sysadmin side
</A></li>
<LI>Next message: <A HREF="003793.html">[Mageia-sysadm] Update procedure on sysadmin side
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#3792">[ date ]</a>
<a href="thread.html#3792">[ thread ]</a>
<a href="subject.html#3792">[ subject ]</a>
<a href="author.html#3792">[ author ]</a>
</LI>
</UL>
<hr>
<a href="https://www.mageia.org/mailman/listinfo/mageia-sysadm">More information about the Mageia-sysadm
mailing list</a><br>
</body></html>
|
