Start creating a new set of tools for generating Mageia ISO images.

The aim is to share as much code as possible between the tool used
to generate the Live ISOs and the tool used to generate the classic
installer ISOs.

This is derived from the user/martinw/use-grub2 branch of draklive.

1 files changed, 2 insertions, 643 deletions

diff --git a/NEWS b/NEWS
index 9bcbaac..ef4e846 100644
--- a/NEWS
+++ b/NEWS
@@ -1,643 +1,2 @@
-- remove obsolete code for generating old draklive-style initrds
-
-2.6:
-
-- use systemctl to disable services and timers
-- disable audit logging as it floods the logs
-
-2.5:
-
-- remove control characters from boot/help.msg
-- remove obsolete 'requires bootsplash'
-
-2.4:
-
-- use overlayfs instead of unionfs
-- replace placeholder with actual disk label in EFI grub.cfg file
-- don't include region in the product name when region is 'all'
-- generate sha512 checksum and DATE.txt in ISO master step
-- change syslinux 'harddisk' menu entry to use chain loading
-- mount /dev in chroot so /dev/null is available
-- automatically generate the EFI boot image (efiboot.img)
-- replace incomplete syslinux/bootlogo with gfxboot
-- create mgagfxbootXXXX temporary directory in cwd
-
-2.3:
-
-- generate proper efi isohybrid
-- use xorriso for proper efi metadata
-- add noiswmd on kernel command line
-- add efiboot.img on x86_64 isos
-- add missing isolinux menu files
-
-2.2:
-
-- umount_system_fs(): unmount /run too
-
-2.1:
-
-- adapt for usbfs death, change /proc/bus/usb to /sys/kernel/debug/usb
-- drop harcoded /etc/dracut.conf.d/60-live.conf
-- unmount /dev when needed
-
-2.0:
-
-* adapt for creating Mageia 2 livecds
-  we now rely on udev, systemd and dracut
-
-1.4:
-
-- use gfxboot.c32 as graphical boot loader
-- add more space in initrd for plymouth
-- adapt for Mageia naming scheme (https://wiki.mageia.org/en/Product_naming)
-
-1.3:
-
-* build/chroot
-- add a -<set> suffix to chroot/build directories if
-  live->{settings}{set} is defined
-
-* master
-- link iso image with -<set> prefix if live->{settings}{set} is defined
-
-
-1.2:
-
-* master
-- use mgaiso-addmd5 instead of mkcd --addmd5
-
-* misc
-- umount /proc and /sys in chroot before cleaning or compressing
-
-
-1.1:
-
-* boot
-- do not harcode default gfxboot theme,
-  use distribution name from /etc/product.id
-
-* loopback
-- add xz compression support for squashfs
-
-
-1.0:
-
-* install
-- mount /proc and /sys in post-install (to be friendly for packages installation, like Mythware)
-- add trailing / not to copy resolv.conf as /etc if chroot is empty
-- rename setarch option as targetarch
-- do not try to use setarch on ARM
-
-* initrd
-- boost initrd size for Flash
-- force initrd re-creation by unlinking old one
-- mount /proc and /sys for both classical and media-specific initrd
-- mount /proc and /sys for gfxboot files too
-- use busybox.static if present in chroot (else use normal busybox)
-
-* boot
-- copy grub files from arched lib dir (reported by boklm)
-
-* loopbacks
-- no need for squashfs_lzma and squashfs3 now that support has been merged in the kernel
-- set block size in call to mksquashfs
-    This improves compressed image size by about 10% and was dropped when
-    switching to squashfs4
-- add back squashfs3 support (for older distros)
-- use our own progress framework again for squashfs compression
-
-* master
-- add -fatfirst option when hybridifying an image
-- fix settling for udev when formatting master
-- log writing partition table
-- handle writing to mmcblk devices (hackish)
-
-* vm-image
-- do not build vm image by default for masters, do it only if vm_type
-  settings is set (check qemu-img for valid types, like qcow2, vmdk, ...)
-
-* record
-- delete existing files on media when recording, unless --keep-files option is used
-- handle recording CD-Rom images on both CD-Rom and disks, since they are hybrid
-
-* replicator
-- do not include whole boot dir in replicator image, boot/vmlinuz would
-  be duplicate in replicator, which sometimes confuses genisoimage, only
-  include boot/syslinux for replicator
-- add --record-replicator option
-- allow to record USB replicator
-- allow to record CD-Rom replicator
-
-* misc
-- add tool to dump and check Mandriva Flash keys
-- add --tarball to build chroot tarball
-
-
-0.9:
-
-* config
-- automatically export draklive settings to DRAKLIVE_ environment variables for install
-- allow to specify mount options in fstab in media->{mount_options}
-- make it possible to specify mount options for tmpfs mounts
-- allow to skip writing fstab with live->{system}{skip_fstab}
-- allow to skip writing bootloader config with live->{system}{skip_bootloader_config}
-- allow to skip bootloader install with live->{system}{skip_bootloader_install}
-- allow not to create initrd with live->{system}{no_initrd}
-- allow to select which settings fields are used to build name, in live->{name_fields}
-
-* install
-- generate lst.names files list
-- make sure zh_TW and zh_CN will be included in language list
-- clean resolv.conf later (so that network can be used in post)
-
-* initrd
-- dropped splashy support, added plymouth support
-  (using bootloader::add_boot_splash)
-- don't mount /dev/pts since this causes huge slowdowns
-- mount tmpfs partitions with 0755 perms (#51565):
-  When using the obsolete --attach-to-session plymouth option, /dev/pts
-  has to be mounted. But for some reason, nash interacts badly with
-  mounted /dev/pts and freezes for dozen of seconds when running a builtin
-  command. Remote --attach-to-session from plymouth which lets us get rid
-  of /dev/pts mounting.
-- add hack to use modules list from chroot
-  (to handle different IDE modules name in build and target environments)
-- detect some modules as built-in and do not wrongly abort
-- do not create initrd symlink or append splash if no initrd has been created
-- make sure /proc and /sys are available when building initrd (useful
-  to see error messages at least, from Paulo Ricardo Zanoni)
-- use bootloader module to build initrd and create kernel symlinks
-  (Gdium friendly)
-
-* bootloader
-- default to grub bootloader for harddisk storage
-- fix installing grub to separate /boot partition
-- use per-media additional boot_entries
-- do not add default boot entry if media->{boot_entries} is specified
-  (to allow having a custom default entry)
-- fix installing grub to hidden /boot in master image
-- set LD_LIBRARY_PATH to chroot libraries when running grub
-- die in bootloader step if selected kernel does not exist
-- do not add initrd in grub menu if it does not exist
-- allow to install bootloader on separate /boot partition
-- make sure the grub install script is executable
-- do not overwrite grub conf for "classical" boot when installing
-  bootloader
-- use same bootloader install code for disks and USB masters
-- remove hardcoded splashy code and use back bootloader::add_boot_splash
-- run switch-themes -u
-- use bootloader module to find kernel (Gdium friendly)
-- update gfxboot theme before copying gfxboot files
-- fix getting bootsplash theme (fixes gfxboot theme)
-- do not add vga mode on kernel command line if forced in append
-  (for drakx-based replicator)
-- fix setting splash for classical bootloader
-- create media specific gfxmenu if needed only
-- always copy gfxmenu in build boot dir (needed for replicator)
-- do not create bootloader files if bootloader install is to be skipped
-- fix writing media specific boot for disk masters
-
-* master
-- use -fatfirst option when hybridifying an image
-- use DrakX partitioning/formatting code
-- run udevsettle, like done in diskdrake::interactive::write_partitions
-- allow to set custom media geometry in media->{geom}
-- allow to format disk devices
-- add ext4 and swap support in mkfs
-- use ext4 by default for harddisk
-- set label at mkfs time
-- fix setting label on fat
-- fix setting label for USB devices
-- add default label for harddisk storage
-- always compute master size from pre-computed partition sizes
-- use apparent size when computing loopbacks size (system loopback can be sparse)
-- supplement / partition label with default one if needed
-- allow to add an OEM rescue partition in the master (quite hackish, using live->{oem_rescue})
-- preset fs_type for OEM_RESCUE partition too
-- allow to set inode size in media->{inode_size} for ext2/ext3 file systems
-- preserve timestamps when copying files
-
-* image
-- allow to compress master images as gzip instead of bzip2 by setting
-  compression_method=gzip in settings.cfg (from Paulo Ricardo Zanoni)
-
-* vm-image
-- create vmdk virtual machine images (for VMware, VirtualBox, qemu)
-
-* replicator
-- copy syslinux dir for replicator too
-- always create syslinux msg files (useful for cdrom replicator)
-- allow drakx-based replicator
-  (by setting live->{settings}{replicator_type} to "drakx")
-- write image size in master list file
-- write bootloader config for replicator media
-
-* record
-- use full disk device when recording harddisk/oem_rescue/replicator/USB masters
-- allow to mount multiple partitions before recording target master
-- do not try to use configured media source when it is not a label, it
-  could be a device (/dev/sda1) that should not be touched on the
-  build machine, force it to be passed as an option
-
-
-0.8:
-
-* loop
-- use lzma by default (should be an option)
-- use legacy mksquashfs3 instead of the new mksquashfs
-- use 1MB squashfs blocks, this gives us about 10% better compression
-- fix loopback extension in /etc/sysconfig/draklive-resize
-
-* bootloader
-- remove unneeded locale files from gfxboot bootlogo file (causes boot issues
-  on some machines)
-- add 'harddrive' boot entry to chainload to the harddrive bootloader
-- use latest syslinux + gfxboot COM module
-- do not crash if there is no gfxmenu file
-- use live->{system}{append}
-- specify root in bootloader append for harddisk installs
-- create "classical" bootloader files (grub menu and gfxmenu)
-- upper-case FAT labels again (we switched to blkid in nash)
-- fix installing probe-modules for x86_64 (spotted by imr)
-
-* master
-- allow to create harddisk masters (for OEM images)
-- hybridify generated ISOs so that they can be dumped on USB keys
-- create bigger vfat and disk masters
-- setup loop device with its real size (so that mkfs does not make a too large fs)
-- write a /boot/grub/install.sh script when installing grub, and run
-  grub in chroot (requires to bind-mount the image inside its mount point)
-- use again du --apparent-size for some sparse files
-
-* images
-- build compressed harddisk image (for now bzipped, could be extracted
-  through partimage later)
-
-* replicator
-- allow to create replicator images (cdrom and USB)
-  using stage1 and restore scripts to dump OEM master on final systems
-- add --replicator to build replicator image
-- allow to customize replicator append in live->{replicator}{append}
-
-* initrd
-- splashy support
-- generate "classical" initrd (for harddisk installs)
-- handle non-gzipped additional modules
-- fix checking missing modules (do it after parsing additional modules)
-- mount /dev as tmpfs after pivot_root, so that finit's /dev/initctl
-  does not get forgotten when start_udev remounts /dev
-- remove deprecated mkdevices call
-- do not use nash-mount to mount tmpfs (run by busybox when using fallback)
-- do not include nls modules for ext2/ext3 filesystems
-- add support for external pre-pivot-root command in initrd
-  (live->{system}{initrd_pre_pivot})
-- skip non-dirs when mount moving to new root (avoid error messages)
-
-* config
-- allow to split version out of name (to follow name-product-version naming)
-
-* chroot
-- use media source instead of /dev/root in fstab, since the later
-  confuses mkinitrd
-- write live->{system}{files} with no_install option in
-  /etc/draklive-install.d/remove.d/draklive
-
-* code architecture
-- split code and MDV::Draklive submodules
-- add initial mkinitrd-draklive prototype
-  (to allow rebuilding live initrd on installed system)
-- add a default mount (for harddisk master)
-- extract predefined and custom mounts/fs in MDV::Draklive::Mounts and
-  MDV::Draklive::CustomMedia modules (breaks config files compatibility)
-
-
-0.7:
-
-* initrd
-- do not copy and load twice additional modules if they were listed in
-  extra modules
-- use nash-mount instead of mount
-- create /etc/blkid and use "showlabels --removable" to get CD-Rom
-  labels in blkid cache
-- adapt to new probe-modules syntax for storage bus
-- use stage1's probe-modules instead of dropped nash's insmod
-- move initrd modules in /lib/modules/`uname -r`
-- gzip initrd modules
-- use depmod to generate modules.dep
-- do not print excluding modules warning if not needed
-- check that there is enough space left in initrd
-- umount /proc/bus/usb before pivot_root in initrd
-- add firewire controllers (bus and disk) in CDROM live (#31356)
-- use libraries from /lib instead of /lib/i686 (#38649) and /lib/tls
-  (#21683) to be able to boot on processors without cmov
-
-* bootloader
-- remove hardcoded fastboot option (new initscripts do not fsck rw /)
-
-* master
-- handle genisoimage progress more nicely
-- insert mkcd checksum before computing md5/sha1 checksums
-- hide mkcd output
-
-* dist
-- rename "images" directory as "dist"
-- create packages lst in dist
-- write a .langs file with human-readable langs list
-- write full list of rpm packages and list of rpm leaves, sorted by package size
-
-* misc
-- do not try to use consolehelper to display "must be root" message,
-  it's broken outside of X
-
-
-0.6:
-
-* config
-- use config and additional files from live->{settings}{config_root},
-  defaulting to current directory, or to /etc/draklive if no config file
-  is present in current directory
-- allow to configure files root directory with --config-root
-- use /var/lib/draklive/build as default build dir and
-  /var/lib/draklive/chroot as default chroot dir
-
-* bootloader
-- improve gfxboot support by copying files from both Mandriva and
-  system-chosen theme
-- install gfxboot files in /isolinux on CD image
-- gfxboot support for grub
-- drop unused F1 binding when gfxboot is used
-- use livecd mode in gfxboot (thanks Pixel \o/)
-- copy gfxmenu instead of moving it (#31879)
-- enable gfxboot only if live->{system}{gfxboot} is set
-- remove extra media support
-- remove optional media support
-- allow to specify bootloader timeout in
-  live->{media}{bootloader_timeout}
-
-* initrd
-- rename debug command line option as initrd_debug not to conflict
-  with kernel option
-- handle module names with underscores, using new module dependencies
-  functions from list_modules (such as modname2path)
-- add and use probe-modules helper from stage1 to load modules
-- drop useless ide-generic hack, probe-modules will load it as
-  fallback only
-- drop usb-storage hack, done by probe-modules
-- do not explicitely load storage module in linuxrc
-  (done by probe-modules helper)
-- add modules.dep, modules.description, modules.alias, ldetect-lst
-  tables, pci.ids and fallback-modules.alias for probe-modules
-- add .ko.gz symlinks to .ko files (used by probe-modules, gzopen
-  transparently opens uncompressed files)
-- load dependencies of extra modules only
-- install modules in /modules instead of /lib in initrd
-- add partition "loop" type to be able to mount partitions in initrd
-- create empty fstab and mtab for fsck programs not to complain
-- run fsck when mounting ext2 partition
-- move mount actions in a storage_fs hash
-- be less verbose when compressing modules
-- be less verbose when printing skipped modules
-- allow to exclude modules from initrd in
-  live->{system}{exclude_modules}
-- use busybox's mkdir since nash's "mkdir -p" does not have the
-  standard behavior
-- pre-create devfsd compatibility loop devices (since busybox is still
-  built with devfsd support)
-- pre-create more loop devices
-- allow to specify module options in live->{system}{module_options}
-- allow to delay mount of specific loopbacks
-- do not add disk/raw category for cdrom media
-- allow to specify ro unionfs branches
-- fix unionfs branches list by removing trailing colon (for unionfs2)
-- don't check symlinks when looking for kernels
-  (fix matching vmlinuz-tmb-desktop586)
-- run mkdevices to create nodes for existing devices
-- move modules mount point in new root
-- do not mess / tree if no live modules are used
-- allow to copy per-fs files in initrd
-- copy additional modules after plain modules, so that they can overwrite them
-- do not uppercase label anymore for vfat partitions, nash with vol_id
-  seems to handle it fine
-
-* chroot
-- be less verbose when running patch dry runs
-- don't hardcode list of files to be removed in chroot, they must be
-  listed in live->{system}{remove_files} now (dbus/modprobe/udev files
-  for example)
-- allow to use real glob patterns in live->{system}{remove_files}
-- copy resolv.conf in chroot for urpmi to work (and clean it at end of
-  post-install)
-- do not run service_harddrake stop anymore, it does not exist in 2008.0
-- run postInstall after copying packages
-
-* loopbacks
-- allow to create loopback modules from live->{loopbacks}{modules}
-  (to be used for separate i18n loopbacks)
-- allow to include loopback modules in master images by defining
-  modules sets in live->{packs} and selecting them from live->{settings}{pack}
-- write a loopbacks/excluded.tar.bz2 tarball containing files that
-  have been excluded from other loopbacks
-- write excluded.tar.bz2 in /tarballs, so that it does not get
-  included in final ISO image
-- remove multi squash union, better replaced with modules
-- add "modules" source type, and use it for squash_union and
-  volatile_squash_union mounts (which are now custom mounts)
-- allow to exclude files in loopbacks with live->{system}{exclude_files}
-- pass -no-progress option to mksquashfs, draklive handles its own
-  progress bar
-- fix division by zero in progress code /o\
-
-* master
-- drop some genisoimage verbosity
-- do not allow to create USB master, it is not correctly implemented yet
-- name images after live name instead of live.iso/live.img
-- compute md5sum and sha1sum for ISO images
-  (in addition to the mkcd checksum)
-- do not copy extra media files in boot images
-- fix extra files selection in cdrom master
-- compute media dirs on the fly instead of hardcoding at start
-  (to support CD-Rom boot images with non-hidden media directories)
-
-* record
-- make device option used for boot-only as well
-- do not set label if boot-only
-- use bootloader setting from optional media if specified
-- umount usb device if loopback copy fails
-- unlink list file if loopbacks copy fails as well
-- mark as read-only boot and loopbacks on vfat USB media
-- mark as hidden files beginning with a dot, autorun files, boot and
-  loopbacks read-only on vfat USB media
-- only run mattrib for vfat
-- run mattrib with MTOOLS_SKIP_CHECK (like mlabel)
-
-* misc
-- print usage if no option is specified
-- log files copy
-- add newlines in die messages, not to trigger the infamous drakbug
-- make warn messages less verbose
-- use live->{settings}{name/product/media/desktop/region} in
-  build/chroot names
-- write a packages.lst file containing the installed packages list
-- add a glob wrapper with dotglob behavior, and expand only if needed
-- custom NFS is now in custom{fs} instead of custom{mounts}
-- reorder init so that arch is set before calling get_builddir and
-  get_system_root (#35105)
-- chomp arch returned by rpm to fix directory names when arch is not
-  in settings.cfg (#35105)
-
-
-0.5:
-
-* chroot
-- use glob() instead of glob_() to support again non-wildcarded
-  filenames in live->{system}{files} and live->{media}{files}
-
-* loopbacks
-- use noappend option in mksquashfs to overwrite previous loopbacks
-
-
-0.4:
-
-* config
-- make loopback fs configurable
-- allow to define fallbacks directories
-  (will be mounted only if the mountpoint is not already mounted)
-- XiB are now used instead of XB when specified fs size is expanded
-- allow to specify min_size in custom squash_union helper
-- introduce "plain" mount type (instead of empty mount type)
-- split build/media prefixes
-- allow to specify additional mount options for storage media
-  (such as umask=...)
-- add live-{system}{hide_media_dirs} and live->{media}{files}
-  (see details in "master" section)
-- allow to use glob patterns in live->{system}{files} and
-  live->{media}{files}
-
-* bootloader
-- use grub-install and grub files from chroot, and make it use custom
-  boot directory
-
-* chroot
-- create draklive-resize configuration file and do not build loopback
-  fs if min_size is specified
-
-* loopbacks
-- use tmpfs as fallback if the ext2 loopback can not be mounted
-- repair loopbacks non-interactively
-
-* master
-- set MTOOLS_SKIP_CHECK to 1 when running mlabel (some USB pens are
-  misdetected by mtools)
-- allow to hide media dirs by adding '.' in prefixes when
-  live-{system}{hide_media_dirs} is set
-- use mattrib to hide media dirs from windows
-- install grub on both master device and partition (fix boot on buggy
-  Intel BIOS)
-- allow to copy files directly to the final image/media using
-  live->{media}{files} (implemented for USB only)
-
-
-0.3:
-
-* interface
-- make --boot-only copy only boot files (instead of using boot media)
-- add --boot-image option to create a boot image for the selected
-  method
-- make string command line options mandatory
-- add --dump-config option
-- allow to load settings from command line using --define options
-- drop --region option (use --define region=)
-
-* config
-- make the system.loop size configurable
-- handle GigaBytes for loopback sizes
-- rename live->{post} as live->{system}{initrd_post}
-- export region as DRAKLIVE_REGION for auto_install
-- allow to provide stage2 updates with live->{system}{stage2_updates}
-- allow to make the initrd reread a device partition table using
-  live->{media}{rereadpt}
-  (useful if USB partition changes after some PIN input)
-- allow to customize bootloader title in live->{media}{title}
-- default to config/live.cfg config path
-- allow to load live->{settings} from a file and default to
-  config/settings.cfg
-- use region/repository/root/workdir variables from live->{settings}
-- rename "workdir" setting as "builddir"
-- rename "root" setting as "chroot"
-- automatically add arch to specified repository
-
-* chroot installation
-- remove /etc/dbus-1/machine-id
-- really don't apply patches if they look already applied
-- workaround buggy installation of directories that are not owned by
-  any packages (using 022 umask)
-- do not try to install packages if the additional media is only aimed
-  at adding extra sources (warly)
-- remove urpmi media before starting additional media installation
-- install additional media just before they are required
-- get drakx-in-chroot from the repository
-- run install/rpm commands for targetted architecture
-
-* bootloader
-- unlink grub device map so that grub rechecks the map
-- die when grub or rsync fail
-- don't install bootloader on non-block device
-- partial gfxboot support (#26430)
-- build bootloader files for boot methods listed in
-  media->{extra_boot}
-- display help file if not bootlogo is available
-- add missing newlines in syslinux/grub configuration files
-
-* loopbacks
-- don't use sort file if it doesn't exist
-- adapt mksquashfs output parsing to latest release
-- die if mksquashfs fails
-- don't pass float to mkfs.vfat
-- run fsck on rw loopback files before mounting them
-- don't reserv blocks on ext2/3 filesystems, we don't create root fs
-- compute an approximative size for USB master images
-
-* master/record
-- don't use a timeout for rsync copy (should fix USB mastering)
-- pass -F option to mke2fs only for non-block devices
-- use genisoimage
-- die if genisoimage can not be run
-- use wodim instead of cdrecord
-
-* copy wizard
-- don't create a master for onthefly USB recording
-- don't show Previous button in copy wizard (#25868)
-- allow to select a master image in the copy wizard
-- update progress during live USB recording
-- add an end step
-- warn if an error occurs during live copy
-- use 788 as default vga mode
-
-* misc:
-- don't add splash/vga settings on cmdline if no vga mode is defined
-- umount any nfs volume in /mnt (warly)
-- use the 'never' timeout value instead of hardcoded values
-- automatically select storage type if only one is available
-- define directory mounts in their mount order, and reverse the order
-  when mounting unionfs
-- guess arch using rpm _target_cpu macro
-- add arch in chroot and builddir paths
-- include desktop in builddir/chroot name if specified
-
-
-0.2:
-
-- remove modprobe.preload.d files
-- remove /etc/udev/rules.d/61-*_config.rules files
-- use patch batch mode (-t) not to apply already applied patches and die if a patch can't be applied
-- run shell in initrd when the "debug" option is on cmdline
-- allow to add additionnal boot entries
-- umount filesystem which could stay mounted in the chroot before cleaning (warly)
-- default splash image is now splash.xpm.gz for grub
-- rename draklive.desktop as draklive-copy-wizard.desktop
-
-
-0.1:
-
-- use system's mount to mount NFS loopbacks
-- sort squashfs loopback if a config/distrib.sort file is present
+- new tools for creating both classic installer and live ISOs
+  o based on user/martinw/use-grub2 branch of draklive