Clone a node/computer using KA method CLONING WILL ERASE ALL CLIENT NODES DATA !!! USE WITH CARE !! Clone a computer over the network Goal of duplication is to easily deploy a computer over network without taking care of numbers of computer. In this documentation, we call golden node the node we want to clone. We can duplicate SCSI or IDE hard drive, and duplication support multiple filesystem (reiserfs, ext2, ext3, ext4, xfs, jfs). This method came from a very old project called CLIC, and was used under IGGI project, all Mandrake Clustering products, and now it is used under XtreemOS project. Now it should be available in 2010 spring, and all futur product. WARNING: all data on client nodes will be ERASED ! We duplicate partitions of HDD's golden node, and the process will do an fdisk command on the client node, so ALL YOUR DATA will be erased on client nodes. KA method With KA method you can quickly duplicate a node using a desc file describing partitions. KA method only duplicate data on partitions, so if you have 80go HDD disk, and only 10go on it, KA only duplicates 10go, and not the whole disk. KA method doesn't not support RAID software. Drawbacks: KA method doesn't support RAID software (use dolly to do that) all data on client nodes are erased you need a PXE, DHDCP and TFTP server you must re-create same partition table as the golden node (even if size can differ) even if it has been tested, it's still an experimental method cloning script are old, and need a full rewrite now it's only works with the Mageia installer (need to patch it to support a KA method) if a node crash while doing a duplication, the duplication process stop (or became very unstable) using fdisk to erase and re-format the HDD is not a good way to proceed UUID support is not really done (fstab use old /dec/sdX) you can only clone Linux filesystems (if you want to duplicate another kinf of FS, it's up to you to modify the scripts) of course various other things ! HOW it works Steps The clone process works in three steps PXE boot to retrieve stage1: the computer boot on PXE mode, retrieve vmlinuz and an initrd image. The computer is in stage1 mode, and is able to get the stage2 throug KA. Network is up. get stage2: the computer gets the stage2 with the KA method. The stage2 contains all necessary tools to recognize your hardware (the most important things is to detect your HDD and your network card), and all necessary tools/scripts to finalize the cloning process. Duplication process: the computer auto-probes needed modules to be able to access the HDD. A basic log server is launched on the client node to be able to run command and get status of the KA duplication process. The computer reconfigure the modprobe.conf and restore the booloader (grub or lilo) Needed files All needed files are available in Mageia cooker. install/stage2/rescue.sqfs: this is the stage2 file with all needed files to detect and probe modules, and launch the third step of the duplication process. This file will be used on the golden node. isolinux/alt0/vmlinuz: linux kernel, needed in the /var/lib/tftpboot/X86PC/linux/images/ directory of the PXE server isolinux/alt0/all.rdz: stage1 and all needed modules and tools. Step 1: PXE, TFTP, DHCPD services To easily clone a computer node, we use PXE technology to boot a kernel, and an initrd image wich contains all needed modules for network and media storage. Documentation about PXE can be found here: PXE doc. Please, keep in mind setting such services can DISTURB your current network architecture. PXE parameters on server Mageia installer supports various methods to install a computer. With PXE configuration file you can specify wich method you want to use to install your node, or add a specific option at boot prompt. Edit your default PXE configuration file to add your custom entry (/var/lib/tftpboot/X86PC/linux/pxelinux.cfg/default). PROMPT 1 DEFAULT local DISPLAY messages TIMEOUT 50 F1 help.txt label local LOCALBOOT 0 label kamethod KERNEL images/vmlinuz APPEND initrd=images/all.rdz ramdisk_size=64000 vga=788 \ automatic=method:ka,interface:eth0,network:dhcp root=/dev/ram3 rw kamethod At boot prompt no you can boot: DEFAULT local: default boot will be local one, change it with the name of a LABEL local: boot local kamethod: automatic mode, get stage2 through KA. Network interface is set to eth0. Auto setup the network with DHCP, and use the KA technology to launch the replication method. TFTP server TFTP server should be activated in /etc/xinetd.d/tftp file, and the xinetd service started. service tftp { disable= no socket_type= dgram protocol= udp wait= yes user= root server= /usr/sbin/in.tftpd server_args = -s /var/lib/tftpboot per_source= 11 cps= 100 2 flags= IPv4 } PXE configuration ,,,, service=X86PC,0,2,linux,Mageia x86 service=IA64PC,0,2,linux,Mageia IA64 service=X86PC,0,0,local,Local boot # tftpd base dir tftpdbase=/ # domain=guibland.com domain= ]]> DHCPD configuration IE of an /etc/dhcpd.conf configuration file. Change IPADDR_TFTP with the IP address of the TFTP serrver, and the NET value. Don't forget to adjust the domain-name and the domain-name-servers. ddns-update-style none; allow booting; allow bootp; authoritative; # Definition of PXE-specific options # Code 1: Multicast IP address of bootfile # Code 2: UDP port that client should monitor for MTFTP responses # Code 3: UDP port that MTFTP servers are using to listen for MTFTP requests # Code 4: Number of secondes a client must listen for activity before trying # to start a new MTFTP transfer # Code 5: Number of secondes a client must listen before trying to restart # a MTFTP transfer # define Option for the PXE class option space PXE; option PXE.mtftp-ip code 1 = ip-address; option PXE.mtftp-cport code 2 = unsigned integer 16; option PXE.mtftp-sport code 3 = unsigned integer 16; option PXE.mtftp-tmout code 4 = unsigned integer 8; option PXE.mtftp-delay code 5 = unsigned integer 8; option PXE.discovery-control code 6 = unsigned integer 8; option PXE.discovery-mcast-addr code 7 = ip-address; #Define options for pxelinux option space pxelinux; option pxelinux.magic code 208 = string; option pxelinux.configfile code 209 = text; option pxelinux.pathprefix code 210 = text; option pxelinux.reboottime code 211 = unsigned integer 32; site-option-space "pxelinux"; option pxelinux.magic f1:00:74:7e; option pxelinux.reboottime 30; #Class that determine the options for Etherboot 5.x requests class "Etherboot" { #if The vendor-class-identifier equal Etherboot-5.0 match if substring (option vendor-class-identifier, 0, 13) = "Etherboot-5.0"; # filename define the file retrieve by the client, there nbgrub # our tftp is chrooted so is just the path to the file filename "/etherboot/nbgrub"; #Used by etherboot to detect a valid pxe dhcp server option vendor-encapsulated-options 3c:09:45:74:68:65:72:62:6f:6f:74:ff; # Set the "vendor-class-identifier" field to "PXEClient" in dhcp answer # if this field is not set the pxe client will ignore the answer ! option vendor-class-identifier "Etherboot-5.0"; vendor-option-space PXE; option PXE.mtftp-ip 0.0.0.0; # IP of you TFTP server next-server IPADDR_TFTP; } # create the Class PXE class "PXE" { # if the "vendor-class-identifier" is set to "PXEClient" in the client dhcp request match if substring(option vendor-class-identifier, 0, 9) = "PXEClient"; filename "/X86PC/linux/linux.0"; option vendor-class-identifier "PXEClient"; vendor-option-space PXE; option PXE.mtftp-ip 0.0.0.0; next-server IPADDR_TFTP; } #host node20 { # hardware ethernet 00:40:CA:8C:B6:E9; # fixed-address node20; #} subnet NET.0 netmask 255.255.255.0 { option subnet-mask 255.255.255.0; option routers IPADDR_GW; default-lease-time 288000; max-lease-time 864000; option domain-name "guibland.com"; option domain-name-servers IPADDR_DNS; next-server IPADDR_TFTP; pool { range NET.30 NET.40; } } Setup a node as a golden node The rescue.sqfs file You need the rescue disk (wich contains the /ka directory), Just extract this file, and copy all directory in /mnt/ka. [root@guibpiv ~]# mkdir /mnt/ka [root@guibpiv ~]# cd /mnt/ka/ [root@guibpiv ka]# unsquashfs rescue.sqfs [root@guibpiv ka]# mv squashfs-root/* . [root@guibpiv ka]# ls bin/ dev/ etc/ ka/ lib/ modules/ proc/ sbin/ squashfs-root/ tmp/ usr/ var/ Go in the /mnt/ka/ka directory, and see all new files available. All those files are needed to do a KA duplication process. We will explain now the rule of each of them. You can modify all them, those files will be copied in the directory /tmp/stage2 of the client node of the duplication process (second step). ka-d.sh This is the master script to declare a node as a golden node. This script takes a lot of arguments. This script should be run on the host wich have the /mnt/ka directory. -h, --help : display this message -n num : specify the number of (destination) nodes -x dir : exclude directory -X sdb|sdc : exclude sdb for the replication -m drive : copy the master boot record (for windows) of this drive (not really tested yet) -M drive file : use 'file' as master boot record (must be 446 bytes long) for the specified drive -D partition : also copy partition 'partition' -p drive pdesc : use 'pdesc' file as partition scheme (see doc) for the specified drive -d delay : delay beteween the release of 2 clients (1/10 second) -r 'grub|lilo' : choose the bootloader (you can add mkinitrd options) ie: ka-d.sh -n 3 -p sda /tmp/desc -X 'sdb|sdc' -r 'grub --with=ata_piix --with=piix' replication.conf This file contains all variables needed by other scripts. It also tries to get information like IP address. fdisk_to_desc This script generate the description table of the hard drive disk in the /tmp/desc file. This file must follow some rules: one line per partition, with two fields : type of partition and size in megabytes. The type can be linux, swap, extended. Other types can be obtained by appending their hexadecimal number to 'type'. For example linux is the same as type83. The size is either a number of megabytes, or the keyword fill (to take all available space). The logical partitions must have the logical keyword. Do a man ka-d for more help. gen_modprobe_conf.pl This script creates a basic output like the content of the/etc/modprobe.conf file. Drawbacks this file must be updated for each new modules available in the kernel (based on the kernel/list_modules.pm file). ka-d-client The ka-d-client binary file is used to get stage2 with the KA method, and after get the whole system. The important argument is the -s session name. A KA can only connect to a specific session (getstage2, kainstall ...). The code source is available in the ka-deploy SRPM. ka-d-server The ka-d-server binary file is used to be a KA golden node server. Like the ka-d-client the session arguments is an important parameter (-s session_name). The session name will be getstage2 to retrieve the stage2 (after the PXE boot) and will be kainstall1 at duplication process step. If you want to do more than one duplication process of nodes at the same time, you should synchronize the ka_sesion name between the server and the client. The code source is available in the ka-deploy SRPM. ka_replication.sh Script launched on the KA client (after getting stage2 and probing modules), to do the full process of the Ka duplication. This script call other scripts to prepare the node (prepare_node.sh), configure the bootloader (make_initrd_grub or make_initrd_lilo). store_log.sh Basic script to store the log of the KA duplication process on an FTP server. Adjust to feet your need, and uncomment the line #store_log.sh in the /mnt/ka/ka/ka_replication.sh file. bootable_flag.sh Script to set bootable an HDD using fdisk. First arg must be the HDD device. make_initrd_grub Restore and reload the Grub bootloader in the /mnt/disk directory. It's a very basic script, and perhaps use the restore_bootloader of the Mageia Rescue should be a better idea. make_initrd_lilo Restore and reload the lilo bootloader in the /mnt/disk directory. Again it's a very basic script, perhaps we should use the restore_bootloader of the Mageia Rescue. prepare_node.sh This script remove in the futur system the old network's udev rules, old dhcp cache files, launch the script gen_modprobe_conf.pl to regenerate an up to date /etc/modprobe.conf in the new system, and launch the script to restore the bootloader. If you want to do more action on the installed, system, you can modify this script. send_status.pl Very basic perl script to open the port 12345, and paste the content of the /tmp/ka* file. It also permit the execution of commands on node, if user send a message from the golden node with the exec prefix. status_node.pl Script to connect to a client node, first arg must be the IP address of the node. You can run command on the node with the exec prefix. The golden node, KA server Now, it is time to build a description of the node partitions. You can use the script /mnt/ka/ka/fdisk_to_desc as root user, or your favorite text editor, you can write a file like this one: linux 3500 extended fill logical swap 500 logical linux fill This file describes your partition table and the sample above can be considered as a default one for a recommended installation. There is a 3.5GB / partition, a 500 MB swap partition, and /var fills the rest, of course you can adjust sizes accoding to your system. Type the following to start the ka replication server as root user on the golden node: -r "grub --with=jfs --with=ata_piix": use grub bootloader and --with=jfs --with=piix mkinitrd option in the chrooted system after the KA deploiement -n nb_nodes: specify how many nodes are clients -p sda desc: specify the name of the hdd -x /tmp: exclude /tmp directory -X sdb: exclude sdb hdd for the duplication Now the golden node is waiting for clients nodes to start replication. KA client node PXE server (kamethod) We have to configure the PXE to boot by default on kamethod. To do this just edit /var/lib/tftpboot/X86PC/linux/pxelinux.cfg/default and set DEFAULT to kamethod: DEFAULT kamethod So, next time a node boots, the PXE server will force the node to boot using the kamethod entry. Stage1 KA method, node waiting stage2 Now, you boot all remaining nodes. The replication process will start once all nodes are up and waiting on the KA screen. If the nodes can't reach the golden node, running the KA server the message Can't reach a valid KA server will appear. Each node will try five times to reach the KA server, after that the node will reboot. As the node boots on kamethod, it will retry until it finds it. Stage2, the duplication process Once all the nodes have found the KA server, the first duplication process will start. This step duplicates the stage2 from the /mnt/ka directory of the golden node, in the client's nodes memory (/dev/ram3 formated as ext2). Then, nodes chroot their memories (the /tmp/stage2 directory), and launch the drvinst command from the stage2, to probe all needed their modules (drivers). Then, the second step of the duplication starts. The duplication process will clone your drives following the description you have made (/tmp/desc of the golden node). Nodes will rewrite their partition table, then format their filesystems (ReiserFs, XFS, ext2/3/4, JFS). All new partitions will be mounted in the /mnt/disk directory. Then, the drive duplication process will begin. On a fast Ethernet switch you can reach speeds of 10MBytes/sec. Prepare the node At the end of the duplication process, each node will chroot its partitions and rebuild its /boot/initrd.img, and /etc/modprobe.conf files. This step ensures that your node will reboot using its potential SCSI drives and adjusting its network card driver. Before rebooting, each node reinstalls lilo/grub. All your node are now ready, and are clone of master node. PXE server to local boot Don't forget to change the default PXE boot to local so node after replication will boot localy. Step by step from scratch KA duplication We will use a PIV 3gz box as golden node, with a SATA hard drive, and an Intel 82540EM Gigabit Ethernet Controller card. This golden box will be the: PXE, DHCPD, TFTP server. Client nodes are basic PIV 2.8gz, with a Realtek Semiconductor 8139 network card, and a IDE hard drive disk PE2650 dual XEON 2.4gz, SCSI Hard Drive disk, and NetXtreme BCM5701 Gigabit Ethernet cards Both nodes are configured to boot on their network card. Golden node side Prepapre the golden node, install all needed tools. Create the /mnt/ka directory, and put all stuff in it (this directory will be sent to all client nodes and use to finish the duplication process) cd pub/linux/Mageia/distrib/cauldron/i586/install/stage2/ lftp distrib-coffee.ipsl.jussieu.fr:/pub/linux/Mageia/distrib/cauldron/i586/install/stage2> pget rescue.sqfs 19132416 bytes transferred in 78 seconds (241.1K/s) [root@localhost ~]# urpmi squashfs-tools http://192.168.1.253/cooker/i586/media/main/release/squashfs-tools-4.0-3.20091221.1mdv2010.1.i586.rpm installing squashfs-tools-4.0-3.20091221.1mdv2010.1.i586.rpm from /var/cache/urpmi/rpms Preparing... ############################################################################ 1/1: squashfs-tools ############################################################################ [root@localhost ~]# unsquashfs rescue.sqfs Parallel unsquashfs: Using 2 processors 988 inodes (1222 blocks) to write [============================================================================================-] 1222/1222 100% created 550 files created 93 directories created 60 symlinks created 371 devices created 1 fifos [root@localhost ~]# cd squashfs-root/ [root@localhost squashfs-root]# ls bin/ dev/ etc/ ka/ lib/ modules/ proc/ sbin/ tmp/ usr/ var/ [root@localhost squashfs-root]# mv * /mnt/ka/ ]]> Install all needed packages to be able to be a PXE, DHCPD and TFTP server Configure all services mtu 1500 qdisc pfifo_fast state UP qlen 100 link/ether 00:17:31:19:a0:78 brd ff:ff:ff:ff:ff:ff inet 10.0.1.42/24 brd 10.0.1.255 scope global eth0 inet6 fe80::217:31ff:fe19:a078/64 scope link valid_lft forever preferred_lft forever [root@localhost ~]# vi /etc/pxe.conf # which interface to use interface=eth0 default_address=10.0.1.42 # the multicast ip address to listen on multicast_address=224.0.1.2 # mtftp info mtftp_address=10.0.1.42 mtftp_client_port=1758 mtftp_server_port=1759 # the port to listen on listen_port=4011 # enable multicast? use_multicast=1 # enable broadcast? use_broadcast=0 # user prompt prompt=Press F8 to view menu ... prompt_timeout=2 # what services to provide, priority in ordering # CSA = Client System Architecture # service=,,,, service=X86PC,0,2,linux,Mageia x86 service=IA64PC,0,2,linux,Mageia IA64 service=X86PC,0,0,local,Local boot # tftpd base dir tftpdbase=/ # domain name domain=guibland.com [root@localhost ~]# vi /etc/xinetd.d/tftp service tftp { disable = no socket_type = dgram protocol = udp wait = yes user = root server = /usr/sbin/in.tftpd server_args = -s /var/lib/tftpboot per_source = 11 cps = 100 2 flags = IPv4 } [root@localhost ~]# cp /etc/dhcpd.conf.pxe.single /etc/dhcpd.conf cp: overwrite `/etc/dhcpd.conf'? y [root@localhost ~]# cat /etc/resolv.conf nameserver 10.0.1.253 search guibland.com [root@localhost ~]# cat /etc/dhcpd.conf # for explanation in french go to : http://www.delafond.org/traducmanfr/man/man5/dhcpd.conf.5.html ddns-update-style none; allow booting; allow bootp; # Your dhcp server is not master on your network ! #not authoritative; # Your dhcpd server is master on your network ! #authoritative; authoritative; #Interface where dhcpd is active #DHCPD_INTERFACE = "eth0"; # Definition of PXE-specific options # Code 1: Multicast IP address of bootfile # Code 2: UDP port that client should monitor for MTFTP responses # Code 3: UDP port that MTFTP servers are using to listen for MTFTP requests # Code 4: Number of secondes a client must listen for activity before trying # to start a new MTFTP transfer # Code 5: Number of secondes a client must listen before trying to restart # a MTFTP transfer # define Option for the PXE class option space PXE; option PXE.mtftp-ip code 1 = ip-address; option PXE.mtftp-cport code 2 = unsigned integer 16; option PXE.mtftp-sport code 3 = unsigned integer 16; option PXE.mtftp-tmout code 4 = unsigned integer 8; option PXE.mtftp-delay code 5 = unsigned integer 8; option PXE.discovery-control code 6 = unsigned integer 8; option PXE.discovery-mcast-addr code 7 = ip-address; #Define options for pxelinux option space pxelinux; option pxelinux.magic code 208 = string; option pxelinux.configfile code 209 = text; option pxelinux.pathprefix code 210 = text; option pxelinux.reboottime code 211 = unsigned integer 32; site-option-space "pxelinux"; # These lines should be customized to your setup #option pxelinux.configfile "configs/common"; #option pxelinux.pathprefix "/pxelinux/files/"; #filename "/pxelinux/pxelinux.bin"; option pxelinux.magic f1:00:74:7e; option pxelinux.reboottime 30; #if exists dhcp-parameter-request-list { # Always send the PXELINUX options # append dhcp-parameter-request-list 208, 209, 210, 211; # append dhcp-parameter-request-list 208,211; # } #Class that determine the options for Etherboot 5.x requests class "Etherboot" { #if The vendor-class-identifier equal Etherboot-5.0 match if substring (option vendor-class-identifier, 0, 13) = "Etherboot-5.0"; # filename define the file retrieve by the client, there nbgrub # our tftp is chrooted so is just the path to the file filename "/etherboot/nbgrub"; #Used by etherboot to detect a valid pxe dhcp server option vendor-encapsulated-options 3c:09:45:74:68:65:72:62:6f:6f:74:ff; # Set the "vendor-class-identifier" field to "PXEClient" in dhcp answer # if this field is not set the pxe client will ignore the answer ! option vendor-class-identifier "Etherboot-5.0"; vendor-option-space PXE; option PXE.mtftp-ip 0.0.0.0; # IP of you TFTP server next-server 10.0.1.42; } # create the Class PXE class "PXE" { # if the "vendor-class-identifier" is set to "PXEClient" in the client dhcp request match if substring(option vendor-class-identifier, 0, 9) = "PXEClient"; # filename define the file retrieve by the client, there pxelinux.0 # our tftp is chrooted so is just the path to the file # If you prefer use grub, use pxegrub compiled for your ethernet card. #filename "/PXEClient/pxegrub"; filename "/X86PC/linux/linux.0"; # Set the "vendor-class-identifier" field to "PXEClient" in dhcp answer # if this field is not set the pxe client will ignore the answer ! option vendor-class-identifier "PXEClient"; vendor-option-space PXE; option PXE.mtftp-ip 0.0.0.0; # IP of you TFTP server next-server 10.0.1.42; } # Tags uses by dhcpnode and setup_add_nodes_to_dhcp # TAG: NODE_LIST_ADMIN_BEGIN # TAG: NODE_LIST_ADMIN_END # TAG: MY_ADMIN_BEGIN subnet 10.0.1.0 netmask 255.255.255.0 { option subnet-mask 255.255.255.0; option routers 10.0.1.253; default-lease-time 288000; max-lease-time 864000; option domain-name "guibland.com"; option domain-name-servers 10.0.1.253; next-server 10.0.1.42; pool { range 10.0.1.110 10.0.1.120; } } # TAG: MY_ADMIN_END [root@localhost ~]# service xinetd restart Stopping xinetd [FAILED] Starting xinetd [ OK ] [root@localhost ~]# service pxe restart Stopping PXE server [FAILED] Dhcp server is not running on this machine ! Be sure that a valid PXE Dhcp server is running on your network Starting PXE server [ OK ] [root@localhost ~]# service dhcpd restart Shutting down dhcpd: [ OK ] Starting dhcpd: [ OK ] ]]> KA listen only listen on eth0, and need a FQDN. So if it is not the case, ka-d-server will try to open a port on 0.0.0.0 IP address, wich cause an error. You can fix it easely setting an valid hostname in /etc/hosts file. Don't forget to kill ka-d-server with crtl+C key, after testing it will open a port on a valid IP address. =0' failed. Aborted [root@node42 ~]# cat /etc/hosts 127.0.0.1 localhost.localdomain localhost 10.0.1.42 node42.guibland.com [root@node42 ~]# ka-d-server Compiled : May 4 2010 20:33:07 ARGS=+ka-d-server+ Server IP = 10.0.1.42 command = (cd /; tar --create --one-file-system --sparse /) I want 1 clients Socket 3 on port 30765 on node42.guibland.com ready. Socket 4 on port 30764 on node42.guibland.com ready. [root@node42 ~]# ^C ]]> We need to describe the partition table of our golden node, to send it to client nodes. Set default PXE boot to kamethod ls -rw-r--r-- 1 ftp ftp 15613654 Apr 23 17:26 all.rdz -rw-r--r-- 1 ftp ftp 2279584 Apr 23 17:26 vmlinuz lftp distrib-coffee.ipsl.jussieu.fr:/pub/linux/Mageia/distrib/cauldron/i586/isolinux/alt0> mget * 17893238 bytes transferred in 77 seconds (227.9K/s) Total 2 files transferred lftp distrib-coffee.ipsl.jussieu.fr:/pub/linux/Mageia/distrib/cauldron/i586/isolinux/alt0> [root@node42 images]# pwd /var/lib/tftpboot/X86PC/linux/images [root@node42 images]# ls all.rdz vmlinuz ]]> Now it's time to launch the duplication process `/tmp/ka-d3156/pfstab.tmp' + Mount points : /dev/sda2 / ext3 relatime 1 1 /dev/sda5 /home ext4 relatime 1 2 /dev/sda1 swap swap defaults 0 0 + Hard drives : sda + Reading partition table description for sda Added partition 1 : type 82 Added partition 2 : type 83 Added partition 5 : type 83 + Included mount points : / /home + Bootloader is: grub +++ Sending Stage2 +++ Compiled : May 4 2010 20:33:07 ARGS=+ka-d-server+-s+getstage2+-n+2+-e+(cd /mnt/ka; tar --create --one-file-system --sparse . )+ Server IP = 10.0.1.42 command = (cd /mnt/ka; tar --create --one-file-system --sparse . ) I want 2 clients Socket 4 on port 30765 on node42.guibland.com ready. Socket 5 on port 30764 on node42.guibland.com ready. got UDP packet from 10.0.1.111 Session name matches Sending UDP reply to 10.0.1.111 Accepting connection from 10.0.1.111 Clients : want_data 0 / connected 0 client says hello ! Client sends options Client accepts data got UDP packet from 10.0.1.110 Session name matches Sending UDP reply to 10.0.1.110 Accepting connection from 10.0.1.110 Clients : want_data 1 / connected 0 client says hello ! Client sends options Client accepts data Added client 10.0.1.110, daddy = 10.0.1.42 Added client 10.0.1.111, daddy = 10.0.1.110 Accepting connection from 10.0.1.110 checking connection auth10.0.1.42 reports 10.0.1.110 has opened data connection Client 10.0.1.110 reports data position : 0 10.0.1.42 reports 10.0.1.110 has been accepted Welcome son, you are number 1 (MAX 4) Client got client 10.0.1.110 reports 10.0.1.111 has opened data connection Client 10.0.1.111 reports data position : 0 sending auth for 10.0.1.111 to 10.0.1.110 Client got client 10.0.1.110 reports 10.0.1.111 has been accepted Let's go! Total data read = 43 Megs, BUF: 34M FREE = 0M startpos = 8MM End of data flow Dropping children Dropping child 10.0.1.110 All children dropped Client says dad disconnected Client says he has finished Client has finished transfer Busy clients: 1 -- connected : 2 Client says dad disconnected Client says he has finished Client has finished transfer Busy clients: 0 -- connected : 2 Peer closed connection on socket 7 close_connection(7) Busy clients: 0 -- connected : 1 Peer closed connection on socket 6 close_connection(6) Busy clients: 0 -- connected : 0 All clients left, I quit Total data sent = 44 Megs, in 1543 packets Transfer time = 0.858 seconds, throughput = 51.307 Mbytes/second The pipeline was emptied in 3.250 seconds - Sending partition/filesystem/mount points informations... +++ Running ka-deploy +++ Compiled : May 4 2010 20:33:07 ARGS=+ka-d-server+-s+kainstall1+-n+2+-e+(cd /tmp/ka-d3156 && tar c *)+ Server IP = 10.0.1.42 command = (cd /tmp/ka-d3156 && tar c *) I want 2 clients Socket 4 on port 30765 on node42.guibland.com ready. Socket 5 on port 30764 on node42.guibland.com ready. got UDP packet from 10.0.1.110 Session name matches Sending UDP reply to 10.0.1.110 Accepting connection from 10.0.1.110 Clients : want_data 0 / connected 0 client says hello ! Client sends options Client accepts data got UDP packet from 10.0.1.111 Session name matches Sending UDP reply to 10.0.1.111 Accepting connection from 10.0.1.111 Clients : want_data 1 / connected 0 client says hello ! Client sends options Client accepts data Added client 10.0.1.110, daddy = 10.0.1.42 Added client 10.0.1.111, daddy = 10.0.1.110 Accepting connection from 10.0.1.110 checking connection auth10.0.1.42 reports 10.0.1.110 has opened data connection Client 10.0.1.110 reports data position : 0 10.0.1.42 reports 10.0.1.110 has been accepted Welcome son, you are number 1 (MAX 4) Client got client 10.0.1.110 reports 10.0.1.111 has opened data connection Client 10.0.1.111 reports data position : 0 sending auth for 10.0.1.111 to 10.0.1.110 Client got client 10.0.1.110 reports 10.0.1.111 has been accepted Let's go! Total data read = 0 Megs, BUF: 0M FREE = 34M startpos = 0M End of data flow Dropping children Dropping child 10.0.1.110 All children dropped Client says dad disconnected Client says he has finished Client has finished transfer Busy clients: 1 -- connected : 2 Client says dad disconnected Client says he has finished Client has finished transfer Busy clients: 0 -- connected : 2 Peer closed connection on socket 6 close_connection(6) Busy clients: 0 -- connected : 1 Peer closed connection on socket 7 close_connection(7) Busy clients: 0 -- connected : 0 All clients left, I quit Total data sent = 0 Megs, in 1 packets Transfer time = 0.006 seconds, throughput = 1.698 Mbytes/second The pipeline was emptied in 0.038 seconds WAITING node (partition/format) - Sending Linux filesystem... +++ Running ka-deploy +++ Compiled : May 4 2010 20:33:07 ARGS=+ka-d-server+-s+kainstall2+-n+2+-e+(cd /; tar --create --one-file-system --sparse / /home)+ Server IP = 10.0.1.42 command = (cd /; tar --create --one-file-system --sparse / /home) I want 2 clients Socket 4 on port 30765 on node42.guibland.com ready. Socket 5 on port 30764 on node42.guibland.com ready. got UDP packet from 10.0.1.110 Session name matches Sending UDP reply to 10.0.1.110 Accepting connection from 10.0.1.110 Clients : want_data 0 / connected 0 client says hello ! Client sends options Client accepts data got UDP packet from 10.0.1.111 Session name matches Sending UDP reply to 10.0.1.111 Accepting connection from 10.0.1.111 Clients : want_data 1 / connected 0 client says hello ! Client sends options Client accepts data Added client 10.0.1.110, daddy = 10.0.1.42 Added client 10.0.1.111, daddy = 10.0.1.110 Accepting connection from 10.0.1.110 checking connection auth10.0.1.42 reports 10.0.1.110 has opened data connection Client got client 10.0.1.110 reports 10.0.1.111 has opened data connection Client 10.0.1.110 reports data position : 0 10.0.1.42 reports 10.0.1.110 has been accepted Welcome son, you are number 1 (MAX 4) Client 10.0.1.111 reports data position : 0 sending auth for 10.0.1.111 to 10.0.1.110 Client got client 10.0.1.110 reports 10.0.1.111 has been accepted Let's go! Total data read = 789 Megs, BUF: 34M FREE = 0M startpos = 754M End of data flow Dropping children Dropping child 10.0.1.110 All children dropped Client says dad disconnected Client says he has finished Client has finished transfer Busy clients: 1 -- connected : 2 Client says dad disconnected Client says he has finished Client has finished transfer Busy clients: 0 -- connected : 2 Peer closed connection on socket 7 close_connection(7) Busy clients: 0 -- connected : 1 Peer closed connection on socket 6 close_connection(6) Busy clients: 0 -- connected : 0 All clients left, I quit Total data sent = 792 Megs, in 25445 packets Transfer time = 69.904 seconds, throughput = 11.343 Mbytes/second The pipeline was emptied in 4.002 seconds ]]> KA client side To get the log of the client node, launch /mnt/ka/ka/status_node.pl IPADD on the golden node. ------| Ka |---- Install starting... 10.0.1.111> Current session is -s kainstall1 10.0.1.111> Receiving partitions information...OK 10.0.1.111> Cleaning hard drive... 10.0.1.111> ==> /tmp/kacmd <== 10.0.1.111> Starting log server.. 10.0.1.111> 10.0.1.111> ==> /tmp/ka_log-10.0.1.111-20100507-10h32 <== 10.0.1.111> OK 10.0.1.111> Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel 10.0.1.111> Building a new DOS disklabel with disk identifier 0x59be1427. 10.0.1.111> Changes will remain in memory only, until you decide to write them. 10.0.1.111> After that, of course, the previous content won't be recoverable. 10.0.1.111> 10.0.1.111> 10.0.1.111> The number of cylinders for this disk is set to 1116. 10.0.1.111> There is nothing wrong with that, but this is larger than 1024, 10.0.1.111> and could in certain setups cause problems with: 10.0.1.111> 1) software that runs at boot time (e.g., old versions of LILO) 10.0.1.111> 2) booting and partitioning software from other OSs 10.0.1.111> (e.g., DOS FDISK, OS/2 FDISK) 10.0.1.111> Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) 10.0.1.111> 10.0.1.111> Command (m for help): The partition table has been altered! 10.0.1.111> 10.0.1.111> Calling ioctl() to re-read partition table. 10.0.1.111> Syncing disks. 10.0.1.111> Writing partition table for sda using fdisk...OK 10.0.1.111> Formatting /dev/sda2 as ext3...OK 10.0.1.111> Formatting /dev/sda5 as ext4...OK 10.0.1.111> Formatting /dev/sda1 as swap...OK 10.0.1.111> - Mounting /dev/sda2 as /mnt/disk/ ......OK 10.0.1.111> - Mounting /dev/sda5 as /mnt/disk/home ......OK 10.0.1.111> ++++++++++++++++++++++++++ 10.0.1.111> none on /sys type sysfs (rw,relatime) 10.0.1.111> none on /proc/bus/usb type usbfs (rw,relatime) 10.0.1.111> /dev/ram3 on /tmp/stage2 type ext2 (rw,relatime,errors=continue) 10.0.1.111> /dev/sda2 on /mnt/disk type ext3 (rw,relatime,errors=continue,data=ordered) 10.0.1.111> /dev/sda5 on /mnt/disk/home type ext4 (rw,relatime,barrier=1,data=ordered) 10.0.1.111> ++++++++++++++++++++++++++ 10.0.1.111> Linux copy is about to start 10.0.1.111> Server IP is 10.0.1.42 10.0.1.111> Buffers names :pipe Total data received = 21 Megs (10.830 Mbytes/sec); BUF :0M 10.0.1.111> ==> /tmp/kacmd <== 10.0.1.111> Starting log server.. 10.0.1.111> 10.0.1.111> ==> /tmp/ka_log-localhost-20100507-11h17 <== Total data received = 788 Megs (10.796 Mbytes/sec); BUF :0M End of data flow 10.0.1.111> Flushing buffers 10.0.1.111> Total data received = 792 Megs, in 574479 packets 10.0.1.111> Elapsed time = 73.413 seconds, throughput = 10.801 Mbytes/second 10.0.1.111> Syncing disks...OK 10.0.1.111> Linux copy done. 10.0.1.111> Creating excluded directories 10.0.1.111> - bootloader is grub (user choice) 10.0.1.111> `/tmp/partfiles/pfstab' -> `/mnt/disk/etc/fstab' 10.0.1.111> - Removing duplicated dhcp cache 10.0.1.111> - Writing modprobe.conf 10.0.1.111> ******************** 10.0.1.111> install scsi_hostadapter /sbin/modprobe aic7xxx; /bin/true 10.0.1.111> install scsi_hostadapter /sbin/modprobe pata_serverworks; /bin/true 10.0.1.111> alias eth0 tg3 10.0.1.111> alias eth1 tg3 10.0.1.111> ******************** 10.0.1.111> - Remove persistent udev rules 10.0.1.111> removed `/mnt/disk/etc/udev/rules.d/70-persistent-net.rules' 10.0.1.111> - Fix /dev in /mnt/disk 10.0.1.111> `/ka2/udev_creation.sh' -> `/mnt/disk/sbin/udev_creation.sh' 10.0.1.111> Starting udev: [ OK ] 10.0.1.111> umount: /mnt/disk/dev: device is busy. 10.0.1.111> (In some cases useful info about processes that use 10.0.1.111> the device is found by lsof(8) or fuser(1)) 10.0.1.111> - Running mkinitrd 10.0.1.111> - Looking for default grub menu 10.0.1.111> - erase old initrd.img link 10.0.1.111> removed `/mnt/disk/boot/initrd.img' 10.0.1.111> initrd will be : /boot/initrd-2.6.33.3-desktop-1mnb.img 10.0.1.111> running: chroot /mnt/disk /sbin/mkinitrd -v -f /boot/initrd-2.6.33.3-desktop-1mnb.img 2.6.33.3-desktop-1mnb 10.0.1.111> Creating initramfs 10.0.1.111> Looking for driver for /dev/sda2 in /sys/block/sda/sda2 10.0.1.111> Looking for deps of module scsi:t-0x00: crc-t10dif scsi_mod sd_mod 10.0.1.111> Looking for deps of module pci:v00009005d000000CFsv00001028sd00000121bc01sc00i00: scsi_mod scsi_transport_spi aic7xxx 10.0.1.111> Looking for deps of module pci:v00008086d00000309sv00000000sd00000000bc06sc04i00: pci_hotplug shpchp 10.0.1.111> Looking for driver for /dev/sda1 in /sys/block/sda/sda1 10.0.1.111> Using modules: usbhid ehci-hcd ohci-hcd uhci-hcd ext3 crc-t10dif scsi_mod sd_mod scsi_transport_spi aic7xxx pci_hotplug shpchp pata_serverworks 10.0.1.111> Building initrd in /tmp/initrd.uuIIkZ 10.0.1.111> /sbin/nash -> /tmp/initrd.uuIIkZ/bin/nash 10.0.1.111> /usr/lib/libnash.so.6.0.93 -> /tmp/initrd.uuIIkZ/usr/lib/libnash.so.6.0.93 10.0.1.111> /lib/libdevmapper.so.1.02 -> /tmp/initrd.uuIIkZ/lib/libdevmapper.so.1.02 10.0.1.111> /lib/libreadline.so.6 -> /tmp/initrd.uuIIkZ/lib/libreadline.so.6 10.0.1.111> /lib/libreadline.so.6.1 -> /tmp/initrd.uuIIkZ/lib/libreadline.so.6.1 10.0.1.111> /lib/libncurses.so.5 -> /tmp/initrd.uuIIkZ/lib/libncurses.so.5 10.0.1.111> /lib/libncurses.so.5.7 -> /tmp/initrd.uuIIkZ/lib/libncurses.so.5.7 10.0.1.111> /lib/libc.so.6 -> /tmp/initrd.uuIIkZ/lib/libc.so.6 10.0.1.111> /lib/libc-2.11.1.so -> /tmp/initrd.uuIIkZ/lib/libc-2.11.1.so 10.0.1.111> /lib/ld-linux.so.2 -> /tmp/initrd.uuIIkZ/lib/ld-linux.so.2 10.0.1.111> /lib/ld-2.11.1.so -> /tmp/initrd.uuIIkZ/lib/ld-2.11.1.so 10.0.1.111> /lib/libdl.so.2 -> /tmp/initrd.uuIIkZ/lib/libdl.so.2 10.0.1.111> /lib/libdl-2.11.1.so -> /tmp/initrd.uuIIkZ/lib/libdl-2.11.1.so 10.0.1.111> /lib/libudev.so.0 -> /tmp/initrd.uuIIkZ/lib/libudev.so.0 10.0.1.111> /lib/libudev.so.0.7.0 -> /tmp/initrd.uuIIkZ/lib/libudev.so.0.7.0 10.0.1.111> /usr/lib/libparted.so.0 -> /tmp/initrd.uuIIkZ/usr/lib/libparted.so.0 10.0.1.111> /usr/lib/libparted.so.0.0.1 -> /tmp/initrd.uuIIkZ/usr/lib/libparted.so.0.0.1 10.0.1.111> /lib/libuuid.so.1 -> /tmp/initrd.uuIIkZ/lib/libuuid.so.1 10.0.1.111> /lib/libuuid.so.1.3.0 -> /tmp/initrd.uuIIkZ/lib/libuuid.so.1.3.0 10.0.1.111> /lib/libblkid.so.1 -> /tmp/initrd.uuIIkZ/lib/libblkid.so.1 10.0.1.111> /lib/libblkid.so.1.1.0 -> /tmp/initrd.uuIIkZ/lib/libblkid.so.1.1.0 10.0.1.111> /lib/libpopt.so.0 -> /tmp/initrd.uuIIkZ/lib/libpopt.so.0 10.0.1.111> /lib/libpopt.so.0.0.0 -> /tmp/initrd.uuIIkZ/lib/libpopt.so.0.0.0 10.0.1.111> /lib/libresolv.so.2 -> /tmp/initrd.uuIIkZ/lib/libresolv.so.2 10.0.1.111> /lib/libresolv-2.11.1.so -> /tmp/initrd.uuIIkZ/lib/libresolv-2.11.1.so 10.0.1.111> /usr/lib/libelf.so.1 -> /tmp/initrd.uuIIkZ/usr/lib/libelf.so.1 10.0.1.111> /usr/lib/libelf-0.146.so -> /tmp/initrd.uuIIkZ/usr/lib/libelf-0.146.so 10.0.1.111> /lib/libm.so.6 -> /tmp/initrd.uuIIkZ/lib/libm.so.6 10.0.1.111> /lib/libm-2.11.1.so -> /tmp/initrd.uuIIkZ/lib/libm-2.11.1.so 10.0.1.111> /lib/libgcc_s.so.1 -> /tmp/initrd.uuIIkZ/lib/libgcc_s.so.1 10.0.1.111> /lib/libgcc_s-4.4.3.so.1 -> /tmp/initrd.uuIIkZ/lib/libgcc_s-4.4.3.so.1 10.0.1.111> /usr/lib/libbdevid.so.6.0.93 -> /tmp/initrd.uuIIkZ/usr/lib/libbdevid.so.6.0.93 10.0.1.111> /sbin/modprobe -> /tmp/initrd.uuIIkZ/bin/modprobe 10.0.1.111> /lib/libmodprobe.so.0 -> /tmp/initrd.uuIIkZ/lib/libmodprobe.so.0 10.0.1.111> /lib/libmodprobe.so.0.0.0 -> /tmp/initrd.uuIIkZ/lib/libmodprobe.so.0.0.0 10.0.1.111> /lib/libz.so.1 -> /tmp/initrd.uuIIkZ/lib/libz.so.1 10.0.1.111> /lib/libz.so.1.2.3 -> /tmp/initrd.uuIIkZ/lib/libz.so.1.2.3 10.0.1.111> /sbin/rmmod -> /tmp/initrd.uuIIkZ/bin/rmmod 10.0.1.111> /bin/ln -> /tmp/initrd.uuIIkZ/bin/ln 10.0.1.111> resolving for MODULES 10.0.1.111> and that has items of usbhid ehci-hcd ohci-hcd uhci-hcd ext3 crc-t10dif scsi_mod sd_mod scsi_transport_spi aic7xxx pci_hotplug shpchp pata_serverworks 10.0.1.111> Looking for deps of module usbhid: usbcore hid 10.0.1.111> Looking for deps of module ehci-hcd: usbcore 10.0.1.111> Looking for deps of module ohci-hcd: usbcore 10.0.1.111> Looking for deps of module uhci-hcd: usbcore 10.0.1.111> Looking for deps of module ext3: jbd 10.0.1.111> Looking for deps of module crc-t10dif 10.0.1.111> Looking for deps of module scsi_mod 10.0.1.111> Looking for deps of module sd_mod: crc-t10dif scsi_mod 10.0.1.111> Looking for deps of module scsi_transport_spi: scsi_mod 10.0.1.111> Looking for deps of module aic7xxx: scsi_mod scsi_transport_spi 10.0.1.111> Looking for deps of module pci_hotplug 10.0.1.111> Looking for deps of module shpchp: pci_hotplug 10.0.1.111> Looking for deps of module pata_serverworks: scsi_mod libata 10.0.1.111> Looking for deps of module aic7xxx: scsi_mod scsi_transport_spi 10.0.1.111> Looking for deps of module crc-t10dif 10.0.1.111> Looking for deps of module ehci-hcd: usbcore 10.0.1.111> Looking for deps of module ext3: jbd 10.0.1.111> Looking for deps of module hid 10.0.1.111> Looking for deps of module jbd 10.0.1.111> Looking for deps of module libata: scsi_mod 10.0.1.111> Looking for deps of module ohci-hcd: usbcore 10.0.1.111> Looking for deps of module pata_serverworks: scsi_mod libata 10.0.1.111> Looking for deps of module pci_hotplug 10.0.1.111> Looking for deps of module scsi_mod 10.0.1.111> Looking for deps of module scsi_transport_spi: scsi_mod 10.0.1.111> Looking for deps of module sd_mod: crc-t10dif scsi_mod 10.0.1.111> Looking for deps of module shpchp: pci_hotplug 10.0.1.111> Looking for deps of module uhci-hcd: usbcore 10.0.1.111> Looking for deps of module usbcore 10.0.1.111> Looking for deps of module usbhid: usbcore hid 10.0.1.111> resolving for availmodules 10.0.1.111> and that has items of 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/scsi/aic7xxx/aic7xxx.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/aic7xxx.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/lib/crc-t10dif.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/crc-t10dif.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/usb/host/ehci-hcd.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/ehci-hcd.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/fs/ext3/ext3.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/ext3.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/hid/hid.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/hid.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/fs/jbd/jbd.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/jbd.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/ata/libata.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/libata.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/usb/host/ohci-hcd.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/ohci-hcd.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/ata/pata_serverworks.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/pata_serverworks.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/pci/hotplug/pci_hotplug.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/pci_hotplug.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/scsi/scsi_mod.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/scsi_mod.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/scsi/scsi_transport_spi.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/scsi_transport_spi.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/scsi/sd_mod.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/sd_mod.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/pci/hotplug/shpchp.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/shpchp.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/usb/host/uhci-hcd.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/uhci-hcd.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/usb/core/usbcore.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/usbcore.ko.gz 10.0.1.111> /lib/modules/2.6.33.3-desktop-1mnb/kernel/drivers/hid/usbhid/usbhid.ko.gz -> /tmp/initrd.uuIIkZ/lib/modules/2.6.33.3-desktop-1mnb/usbhid.ko.gz 10.0.1.111> /etc/sysconfig/keyboard -> /tmp/initrd.uuIIkZ/etc/sysconfig/keyboard 10.0.1.111> /bin/loadkeys -> /tmp/initrd.uuIIkZ/bin/loadkeys 10.0.1.111> /etc/sysconfig/console/default.kmap -> /tmp/initrd.uuIIkZ/etc/sysconfig/console/default.kmap 10.0.1.111> /etc/sysconfig/i18n -> /tmp/initrd.uuIIkZ/etc/sysconfig/i18n 10.0.1.111> /bin/setfont -> /tmp/initrd.uuIIkZ/bin/setfont 10.0.1.111> /usr/lib/kbd/consolefonts/lat0-16.psfu.gz -> /tmp/initrd.uuIIkZ/usr/lib/kbd/consolefonts/lat0-16.psfu.gz 10.0.1.111> /lib/udev/console_init -> /tmp/initrd.uuIIkZ/lib/udev/console_init 10.0.1.111> probing for drm modules for pci device /sys/bus/pci/devices/0000:00:0e.0 10.0.1.111> Adding graphics device /sys/bus/pci/devices/0000:00:0e.0 10.0.1.111> Looking for deps of module pci:v00001002d00004752sv00001028sd00000121bc03sc00i00 10.0.1.111> resolving for GRAPHICSMODS 10.0.1.111> and that has items of 10.0.1.111> Adding module usbhid 10.0.1.111> Adding module ehci-hcd 10.0.1.111> Adding module ohci-hcd 10.0.1.111> Adding module uhci-hcd 10.0.1.111> Adding module ext3 10.0.1.111> Adding module crc-t10dif 10.0.1.111> Adding module scsi_mod 10.0.1.111> Adding module sd_mod 10.0.1.111> Adding module scsi_transport_spi 10.0.1.111> Adding module aic7xxx 10.0.1.111> Adding module pci_hotplug 10.0.1.111> Adding module shpchp 10.0.1.111> Adding module pata_serverworks 10.0.1.111> /usr/sbin/resume -> /tmp/initrd.uuIIkZ/bin/resume 10.0.1.111> /usr/lib/suspend/resume -> /tmp/initrd.uuIIkZ/usr/lib/suspend/resume 10.0.1.111> /usr/lib/liblzo2.so.2 -> /tmp/initrd.uuIIkZ/usr/lib/liblzo2.so.2 10.0.1.111> /usr/lib/liblzo2.so.2.0.0 -> /tmp/initrd.uuIIkZ/usr/lib/liblzo2.so.2.0.0 10.0.1.111> /lib/libpthread.so.0 -> /tmp/initrd.uuIIkZ/lib/libpthread.so.0 10.0.1.111> /lib/libpthread-2.11.1.so -> /tmp/initrd.uuIIkZ/lib/libpthread-2.11.1.so 10.0.1.111> /lib/libply.so.2 -> /tmp/initrd.uuIIkZ/lib/libply.so.2 10.0.1.111> /lib/libply.so.2.0.0 -> /tmp/initrd.uuIIkZ/lib/libply.so.2.0.0 10.0.1.111> /lib/librt.so.1 -> /tmp/initrd.uuIIkZ/lib/librt.so.1 10.0.1.111> /lib/librt-2.11.1.so -> /tmp/initrd.uuIIkZ/lib/librt-2.11.1.so 10.0.1.111> /lib/libply-splash-core.so.2 -> /tmp/initrd.uuIIkZ/lib/libply-splash-core.so.2 10.0.1.111> /lib/libply-splash-core.so.2.0.0 -> /tmp/initrd.uuIIkZ/lib/libply-splash-core.so.2.0.0 10.0.1.111> /etc/suspend.conf -> /tmp/initrd.uuIIkZ/etc/suspend.conf 10.0.1.111> This initrd uses dynamic shared objects. 10.0.1.111> Adding dynamic linker configuration files. 10.0.1.111> /etc/ld.so.conf -> /tmp/initrd.uuIIkZ/etc/ld.so.conf 10.0.1.111> Running ldconfig 10.0.1.111> Installation finished. No error reported. 10.0.1.111> This is the contents of the device map /boot/grub/device.map. 10.0.1.111> Check if this is correct or not. If any of the lines is incorrect, 10.0.1.111> fix it and re-run the script `grub-install'. 10.0.1.111> 10.0.1.111> (hd0) /dev/sda 10.0.1.111> Umounting /dev/sda5...OK 10.0.1.111> Umounting /dev/sda2...OK 10.0.1.111> ftp: connect: Connection refused 10.0.1.111> Local directory now /tmp 10.0.1.111> Not connected. ]]> Post duplication process Now client nodes reboots, so we have to switch PXE to a local boot. Nodes hsould be up, we can to see their hardware.
n1883' href='#n1883'>1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233
<?php
/**
*
* This file is part of the phpBB Forum Software package.
*
* @copyright (c) phpBB Limited <https://www.phpbb.com>
* @license GNU General Public License, version 2 (GPL-2.0)
*
* For full copyright and license information, please see
* the docs/CREDITS.txt file.
*
*/

/**
*/
if (!defined('IN_PHPBB'))
{
	exit;
}

/*
	Ability to simply add own rules by doing three things:
		1) Add an appropriate constant
		2) Add a new check array to the global_privmsgs_rules variable and the condition array (if one is required)
		3) Implement the rule logic in the check_rule() function
		4) Add a new language variable to ucp.php

		The user is then able to select the new rule. It will be checked against and handled as specified.
		To add new actions (yes, checks can be added here too) to the rule management, the core code has to be modified.
*/

define('RULE_IS_LIKE', 1);		// Is Like
define('RULE_IS_NOT_LIKE', 2);	// Is Not Like
define('RULE_IS', 3);			// Is
define('RULE_IS_NOT', 4);		// Is Not
define('RULE_BEGINS_WITH', 5);	// Begins with
define('RULE_ENDS_WITH', 6);	// Ends with
define('RULE_IS_FRIEND', 7);	// Is Friend
define('RULE_IS_FOE', 8);		// Is Foe
define('RULE_IS_USER', 9);		// Is User
define('RULE_IS_GROUP', 10);	// Is In Usergroup
define('RULE_ANSWERED', 11);	// Answered
define('RULE_FORWARDED', 12);	// Forwarded
define('RULE_TO_GROUP', 14);	// Usergroup
define('RULE_TO_ME', 15);		// Me

define('ACTION_PLACE_INTO_FOLDER', 1);
define('ACTION_MARK_AS_READ', 2);
define('ACTION_MARK_AS_IMPORTANT', 3);
define('ACTION_DELETE_MESSAGE', 4);

define('CHECK_SUBJECT', 1);
define('CHECK_SENDER', 2);
define('CHECK_MESSAGE', 3);
define('CHECK_STATUS', 4);
define('CHECK_TO', 5);

/**
* Global private message rules
* These rules define what to do if a rule is hit
*/
$global_privmsgs_rules = array(
	CHECK_SUBJECT	=> array(
		RULE_IS_LIKE		=> array('check0' => 'message_subject'),
		RULE_IS_NOT_LIKE	=> array('check0' => 'message_subject'),
		RULE_IS				=> array('check0' => 'message_subject'),
		RULE_IS_NOT			=> array('check0' => 'message_subject'),
		RULE_BEGINS_WITH	=> array('check0' => 'message_subject'),
		RULE_ENDS_WITH		=> array('check0' => 'message_subject'),
	),

	CHECK_SENDER	=> array(
		RULE_IS_LIKE		=> array('check0' => 'username'),
		RULE_IS_NOT_LIKE	=> array('check0' => 'username'),
		RULE_IS				=> array('check0' => 'username'),
		RULE_IS_NOT			=> array('check0' => 'username'),
		RULE_BEGINS_WITH	=> array('check0' => 'username'),
		RULE_ENDS_WITH		=> array('check0' => 'username'),
		RULE_IS_FRIEND		=> array('check0' => 'friend'),
		RULE_IS_FOE			=> array('check0' => 'foe'),
		RULE_IS_USER		=> array('check0' => 'author_id'),
		RULE_IS_GROUP		=> array('check0' => 'author_in_group'),
	),

	CHECK_MESSAGE	=> array(
		RULE_IS_LIKE		=> array('check0' => 'message_text'),
		RULE_IS_NOT_LIKE	=> array('check0' => 'message_text'),
		RULE_IS				=> array('check0' => 'message_text'),
		RULE_IS_NOT			=> array('check0' => 'message_text'),
	),

	CHECK_STATUS	=> array(
		RULE_ANSWERED		=> array('check0' => 'pm_replied'),
		RULE_FORWARDED		=> array('check0' => 'pm_forwarded'),
	),

	CHECK_TO		=> array(
		RULE_TO_GROUP		=> array('check0' => 'to', 'check1' => 'bcc', 'check2' => 'user_in_group'),
		RULE_TO_ME			=> array('check0' => 'to', 'check1' => 'bcc'),
	)
);

/**
* This is for defining which condition fields to show for which Rule
*/
$global_rule_conditions = array(
	RULE_IS_LIKE		=> 'text',
	RULE_IS_NOT_LIKE	=> 'text',
	RULE_IS				=> 'text',
	RULE_IS_NOT			=> 'text',
	RULE_BEGINS_WITH	=> 'text',
	RULE_ENDS_WITH		=> 'text',
	RULE_IS_USER		=> 'user',
	RULE_IS_GROUP		=> 'group'
);

/**
* Get all folder
*/
function get_folder($user_id, $folder_id = false)
{
	global $db, $user, $template;
	global $phpbb_root_path, $phpEx;

	$folder = array();

	// Get folder information
	$sql = 'SELECT folder_id, COUNT(msg_id) as num_messages, SUM(pm_unread) as num_unread
		FROM ' . PRIVMSGS_TO_TABLE . "
		WHERE user_id = $user_id
			AND folder_id <> " . PRIVMSGS_NO_BOX . '
		GROUP BY folder_id';
	$result = $db->sql_query($sql);

	$num_messages = $num_unread = array();
	while ($row = $db->sql_fetchrow($result))
	{
		$num_messages[(int) $row['folder_id']] = $row['num_messages'];
		$num_unread[(int) $row['folder_id']] = $row['num_unread'];
	}
	$db->sql_freeresult($result);

	// Make sure the default boxes are defined
	$available_folder = array(PRIVMSGS_INBOX, PRIVMSGS_OUTBOX, PRIVMSGS_SENTBOX);

	foreach ($available_folder as $default_folder)
	{
		if (!isset($num_messages[$default_folder]))
		{
			$num_messages[$default_folder] = 0;
		}

		if (!isset($num_unread[$default_folder]))
		{
			$num_unread[$default_folder] = 0;
		}
	}

	// Adjust unread status for outbox
	$num_unread[PRIVMSGS_OUTBOX] = $num_messages[PRIVMSGS_OUTBOX];

	$folder[PRIVMSGS_INBOX] = array(
		'folder_name'		=> $user->lang['PM_INBOX'],
		'num_messages'		=> $num_messages[PRIVMSGS_INBOX],
		'unread_messages'	=> $num_unread[PRIVMSGS_INBOX]
	);

	// Custom Folder
	$sql = 'SELECT folder_id, folder_name, pm_count
		FROM ' . PRIVMSGS_FOLDER_TABLE . "
			WHERE user_id = $user_id";
	$result = $db->sql_query($sql);

	while ($row = $db->sql_fetchrow($result))
	{
		$folder[$row['folder_id']] = array(
			'folder_name'		=> $row['folder_name'],
			'num_messages'		=> $row['pm_count'],
			'unread_messages'	=> ((isset($num_unread[$row['folder_id']])) ? $num_unread[$row['folder_id']] : 0)
		);
	}
	$db->sql_freeresult($result);

	$folder[PRIVMSGS_OUTBOX] = array(
		'folder_name'		=> $user->lang['PM_OUTBOX'],
		'num_messages'		=> $num_messages[PRIVMSGS_OUTBOX],
		'unread_messages'	=> $num_unread[PRIVMSGS_OUTBOX]
	);

	$folder[PRIVMSGS_SENTBOX] = array(
		'folder_name'		=> $user->lang['PM_SENTBOX'],
		'num_messages'		=> $num_messages[PRIVMSGS_SENTBOX],
		'unread_messages'	=> $num_unread[PRIVMSGS_SENTBOX]
	);

	// Define Folder Array for template designers (and for making custom folders usable by the template too)
	foreach ($folder as $f_id => $folder_ary)
	{
		$folder_id_name = ($f_id == PRIVMSGS_INBOX) ? 'inbox' : (($f_id == PRIVMSGS_OUTBOX) ? 'outbox' : 'sentbox');

		$template->assign_block_vars('folder', array(
			'FOLDER_ID'			=> $f_id,
			'FOLDER_NAME'		=> $folder_ary['folder_name'],
			'NUM_MESSAGES'		=> $folder_ary['num_messages'],
			'UNREAD_MESSAGES'	=> $folder_ary['unread_messages'],

			'U_FOLDER'			=> ($f_id > 0) ? append_sid("{$phpbb_root_path}ucp.$phpEx", 'i=pm&amp;folder=' . $f_id) : append_sid("{$phpbb_root_path}ucp.$phpEx", 'i=pm&amp;folder=' . $folder_id_name),

			'S_CUR_FOLDER'		=> ($f_id === $folder_id) ? true : false,
			'S_UNREAD_MESSAGES'	=> ($folder_ary['unread_messages']) ? true : false,
			'S_CUSTOM_FOLDER'	=> ($f_id > 0) ? true : false)
		);
	}

	if ($folder_id !== false && $folder_id !== PRIVMSGS_HOLD_BOX && !isset($folder[$folder_id]))
	{
		trigger_error('UNKNOWN_FOLDER');
	}

	return $folder;
}

/**
* Delete Messages From Sentbox
* we are doing this here because this saves us a bunch of checks and queries
*/
function clean_sentbox($num_sentbox_messages)
{
	global $db, $user, $config;

	// Check Message Limit
	if ($user->data['message_limit'] && $num_sentbox_messages > $user->data['message_limit'])
	{
		// Delete old messages
		$sql = 'SELECT t.msg_id
			FROM ' . PRIVMSGS_TO_TABLE . ' t, ' . PRIVMSGS_TABLE . ' p
			WHERE t.msg_id = p.msg_id
				AND t.user_id = ' . $user->data['user_id'] . '
				AND t.folder_id = ' . PRIVMSGS_SENTBOX . '
			ORDER BY p.message_time ASC';
		$result = $db->sql_query_limit($sql, ($num_sentbox_messages - $user->data['message_limit']));

		$delete_ids = array();
		while ($row = $db->sql_fetchrow($result))
		{
			$delete_ids[] = $row['msg_id'];
		}
		$db->sql_freeresult($result);
		delete_pm($user->data['user_id'], $delete_ids, PRIVMSGS_SENTBOX);
	}
}

/**
* Check Rule against Message Information
*/
function check_rule(&$rules, &$rule_row, &$message_row, $user_id)
{
	global $user, $config;

	if (!isset($rules[$rule_row['rule_check']][$rule_row['rule_connection']]))
	{
		return false;
	}

	$check_ary = $rules[$rule_row['rule_check']][$rule_row['rule_connection']];

	$result = false;

	$check0 = $message_row[$check_ary['check0']];

	switch ($rule_row['rule_connection'])
	{
		case RULE_IS_LIKE:
			$result = preg_match("/" . preg_quote($rule_row['rule_string'], '/') . '/i', $check0);
		break;

		case RULE_IS_NOT_LIKE:
			$result = !preg_match("/" . preg_quote($rule_row['rule_string'], '/') . '/i', $check0);
		break;

		case RULE_IS:
			$result = ($check0 == $rule_row['rule_string']);
		break;

		case RULE_IS_NOT:
			$result = ($check0 != $rule_row['rule_string']);
		break;

		case RULE_BEGINS_WITH:
			$result = preg_match("/^" . preg_quote($rule_row['rule_string'], '/') . '/i', $check0);
		break;

		case RULE_ENDS_WITH:
			$result = preg_match("/" . preg_quote($rule_row['rule_string'], '/') . '$/i', $check0);
		break;

		case RULE_IS_FRIEND:
		case RULE_IS_FOE:
		case RULE_ANSWERED:
		case RULE_FORWARDED:
			$result = ($check0 == 1);
		break;

		case RULE_IS_USER:
			$result = ($check0 == $rule_row['rule_user_id']);
		break;

		case RULE_IS_GROUP:
			$result = in_array($rule_row['rule_group_id'], $check0);
		break;

		case RULE_TO_GROUP:
			$result = (in_array('g_' . $message_row[$check_ary['check2']], $check0) || in_array('g_' . $message_row[$check_ary['check2']], $message_row[$check_ary['check1']]));
		break;

		case RULE_TO_ME:
			$result = (in_array('u_' . $user_id, $check0) || in_array('u_' . $user_id, $message_row[$check_ary['check1']]));
		break;
	}

	if (!$result)
	{
		return false;
	}

	switch ($rule_row['rule_action'])
	{
		case ACTION_PLACE_INTO_FOLDER:
			return array('action' => $rule_row['rule_action'], 'folder_id' => $rule_row['rule_folder_id']);
		break;

		case ACTION_MARK_AS_READ:
		case ACTION_MARK_AS_IMPORTANT:
			return array('action' => $rule_row['rule_action'], 'pm_unread' => $message_row['pm_unread'], 'pm_marked' => $message_row['pm_marked']);
		break;

		case ACTION_DELETE_MESSAGE:
			global $db, $auth;

			// Check for admins/mods - users are not allowed to remove those messages...
			// We do the check here to make sure the data we use is consistent
			$sql = 'SELECT user_id, user_type, user_permissions
				FROM ' . USERS_TABLE . '
				WHERE user_id = ' . (int) $message_row['author_id'];
			$result = $db->sql_query($sql);
			$userdata = $db->sql_fetchrow($result);
			$db->sql_freeresult($result);

			$auth2 = new \phpbb\auth\auth();
			$auth2->acl($userdata);

			if (!$auth2->acl_get('a_') && !$auth2->acl_get('m_') && !$auth2->acl_getf_global('m_'))
			{
				return array('action' => $rule_row['rule_action'], 'pm_unread' => $message_row['pm_unread'], 'pm_marked' => $message_row['pm_marked']);
			}

			return false;
		break;

		default:
			return false;
	}

	return false;
}

/**
* Update user PM count
*/
function update_pm_counts()
{
	global $user, $db;

	// Update unread count
	$sql = 'SELECT COUNT(msg_id) as num_messages
		FROM ' . PRIVMSGS_TO_TABLE . '
		WHERE pm_unread = 1
			AND folder_id <> ' . PRIVMSGS_OUTBOX . '
			AND user_id = ' . $user->data['user_id'];
	$result = $db->sql_query($sql);
	$user->data['user_unread_privmsg'] = (int) $db->sql_fetchfield('num_messages');
	$db->sql_freeresult($result);

	// Update new pm count
	$sql = 'SELECT COUNT(msg_id) as num_messages
		FROM ' . PRIVMSGS_TO_TABLE . '
		WHERE pm_new = 1
			AND folder_id IN (' . PRIVMSGS_NO_BOX . ', ' . PRIVMSGS_HOLD_BOX . ')
			AND user_id = ' . $user->data['user_id'];
	$result = $db->sql_query($sql);
	$user->data['user_new_privmsg'] = (int) $db->sql_fetchfield('num_messages');
	$db->sql_freeresult($result);

	$db->sql_query('UPDATE ' . USERS_TABLE . ' SET ' . $db->sql_build_array('UPDATE', array(
		'user_unread_privmsg'	=> (int) $user->data['user_unread_privmsg'],
		'user_new_privmsg'		=> (int) $user->data['user_new_privmsg'],
	)) . ' WHERE user_id = ' . $user->data['user_id']);

	// Ok, here we need to repair something, other boxes than privmsgs_no_box and privmsgs_hold_box should not carry the pm_new flag.
	if (!$user->data['user_new_privmsg'])
	{
		$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
			SET pm_new = 0
			WHERE pm_new = 1
				AND folder_id NOT IN (' . PRIVMSGS_NO_BOX . ', ' . PRIVMSGS_HOLD_BOX . ')
				AND user_id = ' . $user->data['user_id'];
		$db->sql_query($sql);
	}
}

/**
* Place new messages into appropriate folder
*/
function place_pm_into_folder(&$global_privmsgs_rules, $release = false)
{
	global $db, $user, $config;

	if (!$user->data['user_new_privmsg'])
	{
		return array('not_moved' => 0, 'removed' => 0);
	}

	$user_message_rules = (int) $user->data['user_message_rules'];
	$user_id = (int) $user->data['user_id'];

	$action_ary = $move_into_folder = array();
	$num_removed = 0;

	// Newly processing on-hold messages
	if ($release)
	{
		$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
			SET folder_id = ' . PRIVMSGS_NO_BOX . '
			WHERE folder_id = ' . PRIVMSGS_HOLD_BOX . "
				AND user_id = $user_id";
		$db->sql_query($sql);
	}

	// Get those messages not yet placed into any box
	$retrieve_sql = 'SELECT t.*, p.*, u.username, u.user_id, u.group_id
		FROM ' . PRIVMSGS_TO_TABLE . ' t, ' . PRIVMSGS_TABLE . ' p, ' . USERS_TABLE . " u
		WHERE t.user_id = $user_id
			AND p.author_id = u.user_id
			AND t.folder_id = " . PRIVMSGS_NO_BOX . '
			AND t.msg_id = p.msg_id';

	// Just place into the appropriate arrays if no rules need to be checked
	if (!$user_message_rules)
	{
		$result = $db->sql_query($retrieve_sql);

		while ($row = $db->sql_fetchrow($result))
		{
			$action_ary[$row['msg_id']][] = array('action' => false);
		}
		$db->sql_freeresult($result);
	}
	else
	{
		$user_rules = $zebra = $check_rows = array();
		$user_ids = $memberships = array();

		// First of all, grab all rules and retrieve friends/foes
		$sql = 'SELECT *
			FROM ' . PRIVMSGS_RULES_TABLE . "
			WHERE user_id = $user_id";
		$result = $db->sql_query($sql);
		$user_rules = $db->sql_fetchrowset($result);
		$db->sql_freeresult($result);

		if (sizeof($user_rules))
		{
			$sql = 'SELECT zebra_id, friend, foe
				FROM ' . ZEBRA_TABLE . "
				WHERE user_id = $user_id";
			$result = $db->sql_query($sql);

			while ($row = $db->sql_fetchrow($result))
			{
				$zebra[$row['zebra_id']] = $row;
			}
			$db->sql_freeresult($result);
		}

		// Now build a bare-bone check_row array
		$result = $db->sql_query($retrieve_sql);

		while ($row = $db->sql_fetchrow($result))
		{
			$check_rows[] = array_merge($row, array(
				'to'				=> explode(':', $row['to_address']),
				'bcc'				=> explode(':', $row['bcc_address']),
				'friend'			=> (isset($zebra[$row['author_id']])) ? $zebra[$row['author_id']]['friend'] : 0,
				'foe'				=> (isset($zebra[$row['author_id']])) ? $zebra[$row['author_id']]['foe'] : 0,
				'user_in_group'		=> array($user->data['group_id']),
				'author_in_group'	=> array())
			);

			$user_ids[] = $row['user_id'];
		}
		$db->sql_freeresult($result);

		// Retrieve user memberships
		if (sizeof($user_ids))
		{
			$sql = 'SELECT *
				FROM ' . USER_GROUP_TABLE . '
				WHERE ' . $db->sql_in_set('user_id', $user_ids) . '
					AND user_pending = 0';
			$result = $db->sql_query($sql);

			while ($row = $db->sql_fetchrow($result))
			{
				$memberships[$row['user_id']][] = $row['group_id'];
			}
			$db->sql_freeresult($result);
		}

		// Now place into the appropriate folder
		foreach ($check_rows as $row)
		{
			// Add membership if set
			if (isset($memberships[$row['author_id']]))
			{
				$row['author_in_group'] = $memberships[$row['user_id']];
			}

			// Check Rule - this should be very quick since we have all information we need
			$is_match = false;
			foreach ($user_rules as $rule_row)
			{
				if (($action = check_rule($global_privmsgs_rules, $rule_row, $row, $user_id)) !== false)
				{
					$is_match = true;
					$action_ary[$row['msg_id']][] = $action;
				}
			}

			if (!$is_match)
			{
				$action_ary[$row['msg_id']][] = array('action' => false);
			}
		}

		unset($user_rules, $zebra, $check_rows, $user_ids, $memberships);
	}

	// We place actions into arrays, to save queries.
	$sql = $unread_ids = $delete_ids = $important_ids = array();

	foreach ($action_ary as $msg_id => $msg_ary)
	{
		// It is allowed to execute actions more than once, except placing messages into folder
		$folder_action = $message_removed = false;

		foreach ($msg_ary as $pos => $rule_ary)
		{
			if ($folder_action && $rule_ary['action'] == ACTION_PLACE_INTO_FOLDER)
			{
				continue;
			}

			switch ($rule_ary['action'])
			{
				case ACTION_PLACE_INTO_FOLDER:
					// Folder actions have precedence, so we will remove any other ones
					$folder_action = true;
					$move_into_folder[(int) $rule_ary['folder_id']][] = $msg_id;
				break;

				case ACTION_MARK_AS_READ:
					if ($rule_ary['pm_unread'])
					{
						$unread_ids[] = $msg_id;
					}
				break;

				case ACTION_DELETE_MESSAGE:
					$delete_ids[] = $msg_id;
					$message_removed = true;
				break;

				case ACTION_MARK_AS_IMPORTANT:
					if (!$rule_ary['pm_marked'])
					{
						$important_ids[] = $msg_id;
					}
				break;
			}
		}

		// We place this here because it could happen that the messages are doubled if a rule marks a message and then moves it into a specific
		// folder. Here we simply move the message into the INBOX if it gets not removed and also not put into a custom folder.
		if (!$folder_action && !$message_removed)
		{
			$move_into_folder[PRIVMSGS_INBOX][] = $msg_id;
		}
	}

	// Do not change the order of processing
	// The number of queries needed to be executed here highly depends on the defined rules and are
	// only gone through if new messages arrive.

	// Delete messages
	if (sizeof($delete_ids))
	{
		$num_removed += sizeof($delete_ids);
		delete_pm($user_id, $delete_ids, PRIVMSGS_NO_BOX);
	}

	// Set messages to Unread
	if (sizeof($unread_ids))
	{
		$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
			SET pm_unread = 0
			WHERE ' . $db->sql_in_set('msg_id', $unread_ids) . "
				AND user_id = $user_id
				AND folder_id = " . PRIVMSGS_NO_BOX;
		$db->sql_query($sql);
	}

	// mark messages as important
	if (sizeof($important_ids))
	{
		$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
			SET pm_marked = 1 - pm_marked
			WHERE folder_id = ' . PRIVMSGS_NO_BOX . "
				AND user_id = $user_id
				AND " . $db->sql_in_set('msg_id', $important_ids);
		$db->sql_query($sql);
	}

	// Move into folder
	$folder = array();

	if (sizeof($move_into_folder))
	{
		// Determine Full Folder Action - we need the move to folder id later eventually
		$full_folder_action = ($user->data['user_full_folder'] == FULL_FOLDER_NONE) ? ($config['full_folder_action'] - (FULL_FOLDER_NONE*(-1))) : $user->data['user_full_folder'];

		$sql_folder = array_keys($move_into_folder);
		if ($full_folder_action >= 0)
		{
			$sql_folder[] = $full_folder_action;
		}

		$sql = 'SELECT folder_id, pm_count
			FROM ' . PRIVMSGS_FOLDER_TABLE . '
			WHERE ' . $db->sql_in_set('folder_id', $sql_folder) . "
				AND user_id = $user_id";
		$result = $db->sql_query($sql);

		while ($row = $db->sql_fetchrow($result))
		{
			$folder[(int) $row['folder_id']] = (int) $row['pm_count'];
		}
		$db->sql_freeresult($result);

		unset($sql_folder);

		if (isset($move_into_folder[PRIVMSGS_INBOX]))
		{
			$sql = 'SELECT COUNT(msg_id) as num_messages
				FROM ' . PRIVMSGS_TO_TABLE . "
				WHERE user_id = $user_id
					AND folder_id = " . PRIVMSGS_INBOX;
			$result = $db->sql_query($sql);
			$folder[PRIVMSGS_INBOX] = (int) $db->sql_fetchfield('num_messages');
			$db->sql_freeresult($result);
		}
	}

	// Here we have ideally only one folder to move into
	foreach ($move_into_folder as $folder_id => $msg_ary)
	{
		$dest_folder = $folder_id;
		$full_folder_action = FULL_FOLDER_NONE;

		// Check Message Limit - we calculate with the complete array, most of the time it is one message
		// But we are making sure that the other way around works too (more messages in queue than allowed to be stored)
		if ($user->data['message_limit'] && $folder[$folder_id] && ($folder[$folder_id] + sizeof($msg_ary)) > $user->data['message_limit'])
		{
			$full_folder_action = ($user->data['user_full_folder'] == FULL_FOLDER_NONE) ? ($config['full_folder_action'] - (FULL_FOLDER_NONE*(-1))) : $user->data['user_full_folder'];

			// If destination folder itself is full...
			if ($full_folder_action >= 0 && ($folder[$full_folder_action] + sizeof($msg_ary)) > $user->data['message_limit'])
			{
				$full_folder_action = $config['full_folder_action'] - (FULL_FOLDER_NONE*(-1));
			}

			// If Full Folder Action is to move to another folder, we simply adjust the destination folder
			if ($full_folder_action >= 0)
			{
				$dest_folder = $full_folder_action;
			}
			else if ($full_folder_action == FULL_FOLDER_DELETE)
			{
				// Delete some messages. NOTE: Ordered by msg_id here instead of message_time!
				$sql = 'SELECT msg_id
					FROM ' . PRIVMSGS_TO_TABLE . "
					WHERE user_id = $user_id
						AND folder_id = $dest_folder
					ORDER BY msg_id ASC";
				$result = $db->sql_query_limit($sql, (($folder[$dest_folder] + sizeof($msg_ary)) - $user->data['message_limit']));

				$delete_ids = array();
				while ($row = $db->sql_fetchrow($result))
				{
					$delete_ids[] = $row['msg_id'];
				}
				$db->sql_freeresult($result);

				$num_removed += sizeof($delete_ids);
				delete_pm($user_id, $delete_ids, $dest_folder);
			}
		}

		//
		if ($full_folder_action == FULL_FOLDER_HOLD)
		{
			$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
				SET folder_id = ' . PRIVMSGS_HOLD_BOX . '
				WHERE folder_id = ' . PRIVMSGS_NO_BOX . "
					AND user_id = $user_id
					AND " . $db->sql_in_set('msg_id', $msg_ary);
			$db->sql_query($sql);
		}
		else
		{
			$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . "
				SET folder_id = $dest_folder, pm_new = 0
				WHERE folder_id = " . PRIVMSGS_NO_BOX . "
					AND user_id = $user_id
					AND pm_new = 1
					AND " . $db->sql_in_set('msg_id', $msg_ary);
			$db->sql_query($sql);

			if ($dest_folder != PRIVMSGS_INBOX)
			{
				$sql = 'UPDATE ' . PRIVMSGS_FOLDER_TABLE . '
					SET pm_count = pm_count + ' . (int) $db->sql_affectedrows() . "
					WHERE folder_id = $dest_folder
						AND user_id = $user_id";
				$db->sql_query($sql);
			}
		}
	}

	if (sizeof($action_ary))
	{
		// Move from OUTBOX to SENTBOX
		// We are not checking any full folder status here... SENTBOX is a special treatment (old messages get deleted)
		$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
			SET folder_id = ' . PRIVMSGS_SENTBOX . '
			WHERE folder_id = ' . PRIVMSGS_OUTBOX . '
				AND ' . $db->sql_in_set('msg_id', array_keys($action_ary));
		$db->sql_query($sql);
	}

	// Update new/unread count
	update_pm_counts();

	// Now check how many messages got not moved...
	$sql = 'SELECT COUNT(msg_id) as num_messages
		FROM ' . PRIVMSGS_TO_TABLE . "
		WHERE user_id = $user_id
			AND folder_id = " . PRIVMSGS_HOLD_BOX;
	$result = $db->sql_query($sql);
	$num_not_moved = (int) $db->sql_fetchfield('num_messages');
	$db->sql_freeresult($result);

	return array('not_moved' => $num_not_moved, 'removed' => $num_removed);
}

/**
* Move PM from one to another folder
*/
function move_pm($user_id, $message_limit, $move_msg_ids, $dest_folder, $cur_folder_id)
{
	global $db, $user;
	global $phpbb_root_path, $phpEx;

	$num_moved = 0;

	if (!is_array($move_msg_ids))
	{
		$move_msg_ids = array($move_msg_ids);
	}

	if (sizeof($move_msg_ids) && !in_array($dest_folder, array(PRIVMSGS_NO_BOX, PRIVMSGS_OUTBOX, PRIVMSGS_SENTBOX)) &&
		!in_array($cur_folder_id, array(PRIVMSGS_NO_BOX, PRIVMSGS_OUTBOX)) && $cur_folder_id != $dest_folder)
	{
		// We have to check the destination folder ;)
		if ($dest_folder != PRIVMSGS_INBOX)
		{
			$sql = 'SELECT folder_id, folder_name, pm_count
				FROM ' . PRIVMSGS_FOLDER_TABLE . "
				WHERE folder_id = $dest_folder
					AND user_id = $user_id";
			$result = $db->sql_query($sql);
			$row = $db->sql_fetchrow($result);
			$db->sql_freeresult($result);

			if (!$row)
			{
				trigger_error('NOT_AUTHORISED');
			}

			if ($message_limit && $row['pm_count'] + sizeof($move_msg_ids) > $message_limit)
			{
				$message = sprintf($user->lang['NOT_ENOUGH_SPACE_FOLDER'], $row['folder_name']) . '<br /><br />';
				$message .= sprintf($user->lang['CLICK_RETURN_FOLDER'], '<a href="' . append_sid("{$phpbb_root_path}ucp.$phpEx", 'i=pm&amp;folder=' . $row['folder_id']) . '">', '</a>', $row['folder_name']);
				trigger_error($message);
			}
		}
		else
		{
			$sql = 'SELECT COUNT(msg_id) as num_messages
				FROM ' . PRIVMSGS_TO_TABLE . '
				WHERE folder_id = ' . PRIVMSGS_INBOX . "
					AND user_id = $user_id";
			$result = $db->sql_query($sql);
			$num_messages = (int) $db->sql_fetchfield('num_messages');
			$db->sql_freeresult($result);

			if ($message_limit && $num_messages + sizeof($move_msg_ids) > $message_limit)
			{
				$message = sprintf($user->lang['NOT_ENOUGH_SPACE_FOLDER'], $user->lang['PM_INBOX']) . '<br /><br />';
				$message .= sprintf($user->lang['CLICK_RETURN_FOLDER'], '<a href="' . append_sid("{$phpbb_root_path}ucp.$phpEx", 'i=pm&amp;folder=inbox') . '">', '</a>', $user->lang['PM_INBOX']);
				trigger_error($message);
			}
		}

		$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . "
			SET folder_id = $dest_folder
			WHERE folder_id = $cur_folder_id
				AND user_id = $user_id
				AND " . $db->sql_in_set('msg_id', $move_msg_ids);
		$db->sql_query($sql);
		$num_moved = $db->sql_affectedrows();

		// Update pm counts
		if ($num_moved)
		{
			if (!in_array($cur_folder_id, array(PRIVMSGS_INBOX, PRIVMSGS_OUTBOX, PRIVMSGS_SENTBOX)))
			{
				$sql = 'UPDATE ' . PRIVMSGS_FOLDER_TABLE . "
					SET pm_count = pm_count - $num_moved
					WHERE folder_id = $cur_folder_id
						AND user_id = $user_id";
				$db->sql_query($sql);
			}

			if ($dest_folder != PRIVMSGS_INBOX)
			{
				$sql = 'UPDATE ' . PRIVMSGS_FOLDER_TABLE . "
					SET pm_count = pm_count + $num_moved
					WHERE folder_id = $dest_folder
						AND user_id = $user_id";
				$db->sql_query($sql);
			}
		}
	}
	else if (in_array($cur_folder_id, array(PRIVMSGS_NO_BOX, PRIVMSGS_OUTBOX)))
	{
		trigger_error('CANNOT_MOVE_SPECIAL');
	}

	return $num_moved;
}

/**
* Update unread message status
*/
function update_unread_status($unread, $msg_id, $user_id, $folder_id)
{
	if (!$unread)
	{
		return;
	}

	global $db, $user, $phpbb_container;

	$phpbb_notifications = $phpbb_container->get('notification_manager');

	$phpbb_notifications->mark_notifications_read('notification.type.pm', $msg_id, $user_id);

	$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . "
		SET pm_unread = 0
		WHERE msg_id = $msg_id
			AND user_id = $user_id
			AND folder_id = $folder_id
			AND pm_unread = 1";
	$db->sql_query($sql);

	// If the message is already marked as read, we just skip the rest to avoid negative PM count
	if (!$db->sql_affectedrows())
	{
		return;
	}

	$sql = 'UPDATE ' . USERS_TABLE . "
		SET user_unread_privmsg = user_unread_privmsg - 1
		WHERE user_id = $user_id";
	$db->sql_query($sql);

	if ($user->data['user_id'] == $user_id)
	{
		$user->data['user_unread_privmsg']--;

		// Try to cope with previous wrong conversions...
		if ($user->data['user_unread_privmsg'] < 0)
		{
			$sql = 'UPDATE ' . USERS_TABLE . "
				SET user_unread_privmsg = 0
				WHERE user_id = $user_id";
			$db->sql_query($sql);

			$user->data['user_unread_privmsg'] = 0;
		}
	}
}

function mark_folder_read($user_id, $folder_id)
{
	global $db;

	$sql = 'SELECT msg_id
		FROM ' . PRIVMSGS_TO_TABLE . '
		WHERE folder_id = ' . ((int) $folder_id) . '
			AND user_id = ' . ((int) $user_id) . '
			AND pm_unread = 1';
	$result = $db->sql_query($sql);

	while ($row = $db->sql_fetchrow($result))
	{
		update_unread_status(true, $row['msg_id'], $user_id, $folder_id);
	}
	$db->sql_freeresult($result);
}

/**
* Handle all actions possible with marked messages
*/
function handle_mark_actions($user_id, $mark_action)
{
	global $db, $user, $phpbb_root_path, $phpEx;

	$msg_ids		= request_var('marked_msg_id', array(0));
	$cur_folder_id	= request_var('cur_folder_id', PRIVMSGS_NO_BOX);
	$confirm		= (isset($_POST['confirm'])) ? true : false;

	if (!sizeof($msg_ids))
	{
		return false;
	}

	switch ($mark_action)
	{
		case 'mark_important':

			$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . "
				SET pm_marked = 1 - pm_marked
				WHERE folder_id = $cur_folder_id
					AND user_id = $user_id
					AND " . $db->sql_in_set('msg_id', $msg_ids);
			$db->sql_query($sql);

		break;

		case 'delete_marked':

			global $auth;

			if (!$auth->acl_get('u_pm_delete'))
			{
				trigger_error('NO_AUTH_DELETE_MESSAGE');
			}

			if (confirm_box(true))
			{
				delete_pm($user_id, $msg_ids, $cur_folder_id);

				$success_msg = (sizeof($msg_ids) == 1) ? 'MESSAGE_DELETED' : 'MESSAGES_DELETED';
				$redirect = append_sid("{$phpbb_root_path}ucp.$phpEx", 'i=pm&amp;folder=' . $cur_folder_id);

				meta_refresh(3, $redirect);
				trigger_error($user->lang[$success_msg] . '<br /><br />' . sprintf($user->lang['RETURN_FOLDER'], '<a href="' . $redirect . '">', '</a>'));
			}
			else
			{
				$s_hidden_fields = array(
					'cur_folder_id'	=> $cur_folder_id,
					'mark_option'	=> 'delete_marked',
					'submit_mark'	=> true,
					'marked_msg_id'	=> $msg_ids
				);

				confirm_box(false, 'DELETE_MARKED_PM', build_hidden_fields($s_hidden_fields));
			}

		break;

		default:
			return false;
	}

	return true;
}

/**
* Delete PM(s)
*/
function delete_pm($user_id, $msg_ids, $folder_id)
{
	global $db, $user, $phpbb_root_path, $phpEx, $phpbb_container, $phpbb_dispatcher;

	$user_id	= (int) $user_id;
	$folder_id	= (int) $folder_id;

	if (!$user_id)
	{
		return false;
	}

	if (!is_array($msg_ids))
	{
		if (!$msg_ids)
		{
			return false;
		}
		$msg_ids = array($msg_ids);
	}

	if (!sizeof($msg_ids))
	{
		return false;
	}

	/**
	* Get all info for PM(s) before they are deleted
	*
	* @event core.delete_pm_before
	* @var	int	user_id	 ID of the user requested the message delete
	* @var	array	msg_ids	array of all messages to be deleted
	* @var	int	folder_id	ID of the user folder where the messages are stored
	* @since 3.1.0-b5
	*/
	$vars = array('user_id', 'msg_ids', 'folder_id');
	extract($phpbb_dispatcher->trigger_event('core.delete_pm_before', compact($vars)));

	// Get PM Information for later deleting
	$sql = 'SELECT msg_id, pm_unread, pm_new
		FROM ' . PRIVMSGS_TO_TABLE . '
		WHERE ' . $db->sql_in_set('msg_id', array_map('intval', $msg_ids)) . "
			AND folder_id = $folder_id
			AND user_id = $user_id";
	$result = $db->sql_query($sql);

	$delete_rows = array();
	$num_unread = $num_new = $num_deleted = 0;
	while ($row = $db->sql_fetchrow($result))
	{
		$num_unread += (int) $row['pm_unread'];
		$num_new += (int) $row['pm_new'];

		$delete_rows[$row['msg_id']] = 1;
	}
	$db->sql_freeresult($result);
	unset($msg_ids);

	if (!sizeof($delete_rows))
	{
		return false;
	}

	$db->sql_transaction('begin');

	// if no one has read the message yet (meaning it is in users outbox)
	// then mark the message as deleted...
	if ($folder_id == PRIVMSGS_OUTBOX)
	{
		// Remove PM from Outbox
		$sql = 'DELETE FROM ' . PRIVMSGS_TO_TABLE . "
			WHERE user_id = $user_id AND folder_id = " . PRIVMSGS_OUTBOX . '
				AND ' . $db->sql_in_set('msg_id', array_keys($delete_rows));
		$db->sql_query($sql);

		// Update PM Information for safety
		$sql = 'UPDATE ' . PRIVMSGS_TABLE . " SET message_text = ''
			WHERE " . $db->sql_in_set('msg_id', array_keys($delete_rows));
		$db->sql_query($sql);

		// Set delete flag for those intended to receive the PM
		// We do not remove the message actually, to retain some basic information (sent time for example)
		$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
			SET pm_deleted = 1
			WHERE ' . $db->sql_in_set('msg_id', array_keys($delete_rows));
		$db->sql_query($sql);

		$num_deleted = $db->sql_affectedrows();
	}
	else
	{
		// Delete private message data
		$sql = 'DELETE FROM ' . PRIVMSGS_TO_TABLE . "
			WHERE user_id = $user_id
				AND folder_id = $folder_id
				AND " . $db->sql_in_set('msg_id', array_keys($delete_rows));
		$db->sql_query($sql);
		$num_deleted = $db->sql_affectedrows();
	}

	// if folder id is user defined folder then decrease pm_count
	if (!in_array($folder_id, array(PRIVMSGS_INBOX, PRIVMSGS_OUTBOX, PRIVMSGS_SENTBOX, PRIVMSGS_NO_BOX)))
	{
		$sql = 'UPDATE ' . PRIVMSGS_FOLDER_TABLE . "
			SET pm_count = pm_count - $num_deleted
			WHERE folder_id = $folder_id";
		$db->sql_query($sql);
	}

	// Update unread and new status field
	if ($num_unread || $num_new)
	{
		$set_sql = ($num_unread) ? 'user_unread_privmsg = user_unread_privmsg - ' . $num_unread : '';

		if ($num_new)
		{
			$set_sql .= ($set_sql != '') ? ', ' : '';
			$set_sql .= 'user_new_privmsg = user_new_privmsg - ' . $num_new;
		}

		$db->sql_query('UPDATE ' . USERS_TABLE . " SET $set_sql WHERE user_id = $user_id");

		$user->data['user_new_privmsg'] -= $num_new;
		$user->data['user_unread_privmsg'] -= $num_unread;
	}

	$phpbb_notifications = $phpbb_container->get('notification_manager');

	$phpbb_notifications->delete_notifications('notification.type.pm', array_keys($delete_rows));

	// Now we have to check which messages we can delete completely
	$sql = 'SELECT msg_id
		FROM ' . PRIVMSGS_TO_TABLE . '
		WHERE ' . $db->sql_in_set('msg_id', array_keys($delete_rows));
	$result = $db->sql_query($sql);

	while ($row = $db->sql_fetchrow($result))
	{
		unset($delete_rows[$row['msg_id']]);
	}
	$db->sql_freeresult($result);

	$delete_ids = array_keys($delete_rows);

	if (sizeof($delete_ids))
	{
		// Check if there are any attachments we need to remove
		if (!function_exists('delete_attachments'))
		{
			include($phpbb_root_path . 'includes/functions_admin.' . $phpEx);
		}

		delete_attachments('message', $delete_ids, false);

		$sql = 'DELETE FROM ' . PRIVMSGS_TABLE . '
			WHERE ' . $db->sql_in_set('msg_id', $delete_ids);
		$db->sql_query($sql);
	}

	$db->sql_transaction('commit');

	return true;
}

/**
* Delete all PM(s) for a given user and delete the ones without references
*
* @param	int		$user_id	ID of the user whose private messages we want to delete
*
* @return	boolean		False if there were no pms found, true otherwise.
*/
function phpbb_delete_user_pms($user_id)
{
	global $db, $user, $phpbb_root_path, $phpEx;

	$user_id = (int) $user_id;

	if (!$user_id)
	{
		return false;
	}

	return phpbb_delete_users_pms(array($user_id));
}

/**
* Delete all PM(s) for given users and delete the ones without references
*
* @param	array		$user_ids	IDs of the users whose private messages we want to delete
*
* @return	boolean		False if there were no pms found, true otherwise.
*/
function phpbb_delete_users_pms($user_ids)
{
	global $db, $user, $phpbb_root_path, $phpEx, $phpbb_container;

	$user_id_sql = $db->sql_in_set('user_id', $user_ids);
	$author_id_sql = $db->sql_in_set('author_id', $user_ids);

	// Get PM Information for later deleting
	// The two queries where split, so we can use our indexes
	$undelivered_msg = $delete_ids = array();

	// Part 1: get PMs the user received
	$sql = 'SELECT msg_id
		FROM ' . PRIVMSGS_TO_TABLE . '
		WHERE ' . $user_id_sql;
	$result = $db->sql_query($sql);

	while ($row = $db->sql_fetchrow($result))
	{
		$msg_id = (int) $row['msg_id'];
		$delete_ids[$msg_id] = $msg_id;
	}
	$db->sql_freeresult($result);

	// Part 2: get PMs the users sent, but are yet to be received.
	// We cannot simply delete them. First we have to check
	// whether another user already received and read the message.
	$sql = 'SELECT msg_id
		FROM ' . PRIVMSGS_TO_TABLE . '
		WHERE ' . $author_id_sql . '
			AND folder_id = ' . PRIVMSGS_NO_BOX;
	$result = $db->sql_query($sql);

	while ($row = $db->sql_fetchrow($result))
	{
		$msg_id = (int) $row['msg_id'];
		$undelivered_msg[$msg_id] = $msg_id;
	}
	$db->sql_freeresult($result);

	if (empty($delete_ids) && empty($undelivered_msg))
	{
		return false;
	}

	$db->sql_transaction('begin');

	$phpbb_notifications = $phpbb_container->get('notification_manager');

	if (!empty($undelivered_msg))
	{
		// A pm is delivered, if for any recipient the message was moved
		// from their NO_BOX to another folder. We do not delete such
		// messages, but only delete them for users, who have not yet
		// received them.
		$sql = 'SELECT msg_id
			FROM ' . PRIVMSGS_TO_TABLE . '
			WHERE ' . $author_id_sql . '
				AND folder_id <> ' . PRIVMSGS_NO_BOX . '
				AND folder_id <> ' . PRIVMSGS_OUTBOX . '
				AND folder_id <> ' . PRIVMSGS_SENTBOX;
		$result = $db->sql_query($sql);

		$delivered_msg = array();
		while ($row = $db->sql_fetchrow($result))
		{
			$msg_id = (int) $row['msg_id'];
			$delivered_msg[$msg_id] = $msg_id;
			unset($undelivered_msg[$msg_id]);
		}
		$db->sql_freeresult($result);

		$undelivered_user = array();

		// Count the messages we delete, so we can correct the user pm data
		$sql = 'SELECT user_id, COUNT(msg_id) as num_undelivered_privmsgs
			FROM ' . PRIVMSGS_TO_TABLE . '
			WHERE ' . $author_id_sql . '
				AND folder_id = ' . PRIVMSGS_NO_BOX . '
					AND ' . $db->sql_in_set('msg_id', array_merge($undelivered_msg, $delivered_msg)) . '
			GROUP BY user_id';
		$result = $db->sql_query($sql);

		while ($row = $db->sql_fetchrow($result))
		{
			$num_pms = (int) $row['num_undelivered_privmsgs'];
			$undelivered_user[$num_pms][] = (int) $row['user_id'];

			if (sizeof($undelivered_user[$num_pms]) > 50)
			{
				// If there are too many users affected the query might get
				// too long, so we update the value for the first bunch here.
				$sql = 'UPDATE ' . USERS_TABLE . '
					SET user_new_privmsg = user_new_privmsg - ' . $num_pms . ',
						user_unread_privmsg = user_unread_privmsg - ' . $num_pms . '
					WHERE ' . $db->sql_in_set('user_id', $undelivered_user[$num_pms]);
				$db->sql_query($sql);
				unset($undelivered_user[$num_pms]);
			}
		}
		$db->sql_freeresult($result);

		foreach ($undelivered_user as $num_pms => $undelivered_user_set)
		{
			$sql = 'UPDATE ' . USERS_TABLE . '
				SET user_new_privmsg = user_new_privmsg - ' . $num_pms . ',
					user_unread_privmsg = user_unread_privmsg - ' . $num_pms . '
				WHERE ' . $db->sql_in_set('user_id', $undelivered_user_set);
			$db->sql_query($sql);
		}

		if (!empty($delivered_msg))
		{
			$sql = 'DELETE FROM ' . PRIVMSGS_TO_TABLE . '
				WHERE folder_id = ' . PRIVMSGS_NO_BOX . '
					AND ' . $db->sql_in_set('msg_id', $delivered_msg);
			$db->sql_query($sql);

			$phpbb_notifications->delete_notifications('notification.type.pm', $delivered_msg);
		}

		if (!empty($undelivered_msg))
		{
			$sql = 'DELETE FROM ' . PRIVMSGS_TO_TABLE . '
				WHERE ' . $db->sql_in_set('msg_id', $undelivered_msg);
			$db->sql_query($sql);

			$sql = 'DELETE FROM ' . PRIVMSGS_TABLE . '
				WHERE ' . $db->sql_in_set('msg_id', $undelivered_msg);
			$db->sql_query($sql);

			$phpbb_notifications->delete_notifications('notification.type.pm', $undelivered_msg);
		}
	}

	// Reset the user's pm count to 0
	$sql = 'UPDATE ' . USERS_TABLE . '
		SET user_new_privmsg = 0,
			user_unread_privmsg = 0
		WHERE ' . $user_id_sql;
	$db->sql_query($sql);

	// Delete private message data of the user
	$sql = 'DELETE FROM ' . PRIVMSGS_TO_TABLE . '
		WHERE ' . $user_id_sql;
	$db->sql_query($sql);

	if (!empty($delete_ids))
	{
		// Now we have to check which messages we can delete completely
		$sql = 'SELECT msg_id
			FROM ' . PRIVMSGS_TO_TABLE . '
			WHERE ' . $db->sql_in_set('msg_id', $delete_ids);
		$result = $db->sql_query($sql);

		while ($row = $db->sql_fetchrow($result))
		{
			unset($delete_ids[$row['msg_id']]);
		}
		$db->sql_freeresult($result);

		if (!empty($delete_ids))
		{
			// Check if there are any attachments we need to remove
			if (!function_exists('delete_attachments'))
			{
				include($phpbb_root_path . 'includes/functions_admin.' . $phpEx);
			}

			delete_attachments('message', $delete_ids, false);

			$sql = 'DELETE FROM ' . PRIVMSGS_TABLE . '
				WHERE ' . $db->sql_in_set('msg_id', $delete_ids);
			$db->sql_query($sql);

			$phpbb_notifications->delete_notifications('notification.type.pm', $delete_ids);
		}
	}

	// Set the remaining author id to anonymous
	// This way users are still able to read messages from users being removed
	$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
		SET author_id = ' . ANONYMOUS . '
		WHERE ' . $author_id_sql;
	$db->sql_query($sql);

	$sql = 'UPDATE ' . PRIVMSGS_TABLE . '
		SET author_id = ' . ANONYMOUS . '
		WHERE ' . $author_id_sql;
	$db->sql_query($sql);

	$db->sql_transaction('commit');

	return true;
}

/**
* Rebuild message header
*/
function rebuild_header($check_ary)
{
	global $db;

	$address = array();

	foreach ($check_ary as $check_type => $address_field)
	{
		// Split Addresses into users and groups
		preg_match_all('/:?(u|g)_([0-9]+):?/', $address_field, $match);

		$u = $g = array();
		foreach ($match[1] as $id => $type)
		{
			${$type}[] = (int) $match[2][$id];
		}

		$_types = array('u', 'g');
		foreach ($_types as $type)
		{
			if (sizeof(${$type}))
			{
				foreach (${$type} as $id)
				{
					$address[$type][$id] = $check_type;
				}
			}
		}
	}

	return $address;
}

/**
* Print out/assign recipient information
*/
function write_pm_addresses($check_ary, $author_id, $plaintext = false)
{
	global $db, $user, $template, $phpbb_root_path, $phpEx;

	$addresses = array();

	foreach ($check_ary as $check_type => $address_field)
	{
		if (!is_array($address_field))
		{
			// Split Addresses into users and groups
			preg_match_all('/:?(u|g)_([0-9]+):?/', $address_field, $match);

			$u = $g = array();
			foreach ($match[1] as $id => $type)
			{
				${$type}[] = (int) $match[2][$id];
			}
		}
		else
		{
			$u = $address_field['u'];
			$g = $address_field['g'];
		}

		$address = array();
		if (sizeof($u))
		{
			$sql = 'SELECT user_id, username, user_colour
				FROM ' . USERS_TABLE . '
				WHERE ' . $db->sql_in_set('user_id', $u);
			$result = $db->sql_query($sql);

			while ($row = $db->sql_fetchrow($result))
			{
				if ($check_type == 'to' || $author_id == $user->data['user_id'] || $row['user_id'] == $user->data['user_id'])
				{
					if ($plaintext)
					{
						$address[] = $row['username'];
					}
					else
					{
						$address['user'][$row['user_id']] = array('name' => $row['username'], 'colour' => $row['user_colour']);
					}
				}
			}
			$db->sql_freeresult($result);
		}

		if (sizeof($g))
		{
			if ($plaintext)
			{
				$sql = 'SELECT group_name, group_type
					FROM ' . GROUPS_TABLE . '
						WHERE ' . $db->sql_in_set('group_id', $g);
				$result = $db->sql_query($sql);

				while ($row = $db->sql_fetchrow($result))
				{
					if ($check_type == 'to' || $author_id == $user->data['user_id'] || $row['user_id'] == $user->data['user_id'])
					{
						$address[] = ($row['group_type'] == GROUP_SPECIAL) ? $user->lang['G_' . $row['group_name']] : $row['group_name'];
					}
				}
				$db->sql_freeresult($result);
			}
			else
			{
				$sql = 'SELECT g.group_id, g.group_name, g.group_colour, g.group_type, ug.user_id
					FROM ' . GROUPS_TABLE . ' g, ' . USER_GROUP_TABLE . ' ug
						WHERE ' . $db->sql_in_set('g.group_id', $g) . '
						AND g.group_id = ug.group_id
						AND ug.user_pending = 0';
				$result = $db->sql_query($sql);

				while ($row = $db->sql_fetchrow($result))
				{
					if (!isset($address['group'][$row['group_id']]))
					{
						if ($check_type == 'to' || $author_id == $user->data['user_id'] || $row['user_id'] == $user->data['user_id'])
						{
							$row['group_name'] = ($row['group_type'] == GROUP_SPECIAL) ? $user->lang['G_' . $row['group_name']] : $row['group_name'];
							$address['group'][$row['group_id']] = array('name' => $row['group_name'], 'colour' => $row['group_colour']);
						}
					}

					if (isset($address['user'][$row['user_id']]))
					{
						$address['user'][$row['user_id']]['in_group'] = $row['group_id'];
					}
				}
				$db->sql_freeresult($result);
			}
		}

		if (sizeof($address) && !$plaintext)
		{
			$template->assign_var('S_' . strtoupper($check_type) . '_RECIPIENT', true);

			foreach ($address as $type => $adr_ary)
			{
				foreach ($adr_ary as $id => $row)
				{
					$tpl_ary = array(
						'IS_GROUP'	=> ($type == 'group') ? true : false,
						'IS_USER'	=> ($type == 'user') ? true : false,
						'UG_ID'		=> $id,
						'NAME'		=> $row['name'],
						'COLOUR'	=> ($row['colour']) ? '#' . $row['colour'] : '',
						'TYPE'		=> $type,
					);

					if ($type == 'user')
					{
						$tpl_ary = array_merge($tpl_ary, array(
							'U_VIEW'		=> get_username_string('profile', $id, $row['name'], $row['colour']),
							'NAME_FULL'		=> get_username_string('full', $id, $row['name'], $row['colour']),
						));
					}
					else
					{
						$tpl_ary = array_merge($tpl_ary, array(
							'U_VIEW'		=> append_sid("{$phpbb_root_path}memberlist.$phpEx", 'mode=group&amp;g=' . $id),
						));
					}

					$template->assign_block_vars($check_type . '_recipient', $tpl_ary);
				}
			}
		}

		$addresses[$check_type] = $address;
	}

	return $addresses;
}

/**
* Get folder status
*/
function get_folder_status($folder_id, $folder)
{
	global $db, $user, $config;

	if (isset($folder[$folder_id]))
	{
		$folder = $folder[$folder_id];
	}
	else
	{
		return false;
	}

	$return = array(
		'folder_name'	=> $folder['folder_name'],
		'cur'			=> $folder['num_messages'],
		'remaining'		=> ($user->data['message_limit']) ? $user->data['message_limit'] - $folder['num_messages'] : 0,
		'max'			=> $user->data['message_limit'],
		'percent'		=> ($user->data['message_limit']) ? (($user->data['message_limit'] > 0) ? floor(($folder['num_messages'] / $user->data['message_limit']) * 100) : 100) : 0,
	);

	$return['message']	= $user->lang('FOLDER_STATUS_MSG', $user->lang('MESSAGES_COUNT', (int) $return['max']), (int) $return['cur'], $return['percent']);

	return $return;
}

//
// COMPOSE MESSAGES
//

/**
* Submit PM
*/
function submit_pm($mode, $subject, &$data, $put_in_outbox = true)
{
	global $db, $auth, $config, $phpEx, $template, $user, $phpbb_root_path, $phpbb_container, $phpbb_dispatcher;

	// We do not handle erasing pms here
	if ($mode == 'delete')
	{
		return false;
	}

	$current_time = time();

	/**
	* Get all parts of the PM that are to be submited to the DB.
	*
	* @event core.submit_pm_before
	* @var	string	mode	PM Post mode - post|reply|quote|quotepost|forward|edit
	* @var	string	subject	Subject of the private message
	* @var	array	data	The whole row data of the PM.
	* @since 3.1.0-b3
	*/
	$vars = array('mode', 'subject', 'data');
	extract($phpbb_dispatcher->trigger_event('core.submit_pm_before', compact($vars)));

	// Collect some basic information about which tables and which rows to update/insert
	$sql_data = array();
	$root_level = 0;

	// Recipient Information
	$recipients = $to = $bcc = array();

	if ($mode != 'edit')
	{
		// Build Recipient List
		// u|g => array($user_id => 'to'|'bcc')
		$_types = array('u', 'g');
		foreach ($_types as $ug_type)
		{
			if (isset($data['address_list'][$ug_type]) && sizeof($data['address_list'][$ug_type]))
			{
				foreach ($data['address_list'][$ug_type] as $id => $field)
				{
					$id = (int) $id;

					// Do not rely on the address list being "valid"
					if (!$id || ($ug_type == 'u' && $id == ANONYMOUS))
					{
						continue;
					}

					$field = ($field == 'to') ? 'to' : 'bcc';
					if ($ug_type == 'u')
					{
						$recipients[$id] = $field;
					}
					${$field}[] = $ug_type . '_' . $id;
				}
			}
		}

		if (isset($data['address_list']['g']) && sizeof($data['address_list']['g']))
		{
			// We need to check the PM status of group members (do they want to receive PM's?)
			// Only check if not a moderator or admin, since they are allowed to override this user setting
			$sql_allow_pm = (!$auth->acl_gets('a_', 'm_') && !$auth->acl_getf_global('m_')) ? ' AND u.user_allow_pm = 1' : '';

			$sql = 'SELECT u.user_type, ug.group_id, ug.user_id
				FROM ' . USERS_TABLE . ' u, ' . USER_GROUP_TABLE . ' ug
				WHERE ' . $db->sql_in_set('ug.group_id', array_keys($data['address_list']['g'])) . '
					AND ug.user_pending = 0
					AND u.user_id = ug.user_id
					AND u.user_type IN (' . USER_NORMAL . ', ' . USER_FOUNDER . ')' .
					$sql_allow_pm;
			$result = $db->sql_query($sql);

			while ($row = $db->sql_fetchrow($result))
			{
				$field = ($data['address_list']['g'][$row['group_id']] == 'to') ? 'to' : 'bcc';
				$recipients[$row['user_id']] = $field;
			}
			$db->sql_freeresult($result);
		}

		if (!sizeof($recipients))
		{
			trigger_error('NO_RECIPIENT');
		}
	}

	// First of all make sure the subject are having the correct length.
	$subject = truncate_string($subject);

	$db->sql_transaction('begin');

	$sql = '';

	switch ($mode)
	{
		case 'reply':
		case 'quote':
			$root_level = ($data['reply_from_root_level']) ? $data['reply_from_root_level'] : $data['reply_from_msg_id'];

			// Set message_replied switch for this user
			$sql = 'UPDATE ' . PRIVMSGS_TO_TABLE . '
				SET pm_replied = 1
				WHERE user_id = ' . $data['from_user_id'] . '
					AND msg_id = ' . $data['reply_from_msg_id'];

		// no break

		case 'forward':
		case 'post':
		case 'quotepost':
			$sql_data = array(
				'root_level'		=> $root_level,
				'author_id'			=> $data['from_user_id'],
				'icon_id'			=> $data['icon_id'],
				'author_ip'			=> $data['from_user_ip'],
				'message_time'		=> $current_time,
				'enable_bbcode'		=> $data['enable_bbcode'],
				'enable_smilies'	=> $data['enable_smilies'],
				'enable_magic_url'	=> $data['enable_urls'],
				'enable_sig'		=> $data['enable_sig'],
				'message_subject'	=> $subject,
				'message_text'		=> $data['message'],
				'message_attachment'=> (!empty($data['attachment_data'])) ? 1 : 0,
				'bbcode_bitfield'	=> $data['bbcode_bitfield'],
				'bbcode_uid'		=> $data['bbcode_uid'],
				'to_address'		=> implode(':', $to),
				'bcc_address'		=> implode(':', $bcc),
				'message_reported'	=> 0,
			);
		break;

		case 'edit':
			$sql_data = array(
				'icon_id'			=> $data['icon_id'],
				'message_edit_time'	=> $current_time,
				'enable_bbcode'		=> $data['enable_bbcode'],
				'enable_smilies'	=> $data['enable_smilies'],
				'enable_magic_url'	=> $data['enable_urls'],
				'enable_sig'		=> $data['enable_sig'],
				'message_subject'	=> $subject,
				'message_text'		=> $data['message'],
				'message_attachment'=> (!empty($data['attachment_data'])) ? 1 : 0,
				'bbcode_bitfield'	=> $data['bbcode_bitfield'],
				'bbcode_uid'		=> $data['bbcode_uid']
			);
		break;
	}

	if (sizeof($sql_data))
	{
		$query = '';

		if ($mode == 'post' || $mode == 'reply' || $mode == 'quote' || $mode == 'quotepost' || $mode == 'forward')
		{
			$db->sql_query('INSERT INTO ' . PRIVMSGS_TABLE . ' ' . $db->sql_build_array('INSERT', $sql_data));
			$data['msg_id'] = $db->sql_nextid();
		}
		else if ($mode == 'edit')
		{
			$sql = 'UPDATE ' . PRIVMSGS_TABLE . '
				SET message_edit_count = message_edit_count + 1, ' . $db->sql_build_array('UPDATE', $sql_data) . '
				WHERE msg_id = ' . $data['msg_id'];
			$db->sql_query($sql);
		}
	}

	if ($mode != 'edit')
	{
		if ($sql)
		{
			$db->sql_query($sql);
		}
		unset($sql);

		$sql_ary = array();
		foreach ($recipients as $user_id => $type)
		{
			$sql_ary[] = array(
				'msg_id'		=> (int) $data['msg_id'],
				'user_id'		=> (int) $user_id,
				'author_id'		=> (int) $data['from_user_id'],
				'folder_id'		=> PRIVMSGS_NO_BOX,
				'pm_new'		=> 1,
				'pm_unread'		=> 1,
				'pm_forwarded'	=> ($mode == 'forward') ? 1 : 0
			);
		}

		$db->sql_multi_insert(PRIVMSGS_TO_TABLE, $sql_ary);

		$sql = 'UPDATE ' . USERS_TABLE . '
			SET user_new_privmsg = user_new_privmsg + 1, user_unread_privmsg = user_unread_privmsg + 1, user_last_privmsg = ' . time() . '
			WHERE ' . $db->sql_in_set('user_id', array_keys($recipients));
		$db->sql_query($sql);

		// Put PM into outbox
		if ($put_in_outbox)
		{
			$db->sql_query('INSERT INTO ' . PRIVMSGS_TO_TABLE . ' ' . $db->sql_build_array('INSERT', array(
				'msg_id'		=> (int) $data['msg_id'],
				'user_id'		=> (int) $data['from_user_id'],
				'author_id'		=> (int) $data['from_user_id'],
				'folder_id'		=> PRIVMSGS_OUTBOX,
				'pm_new'		=> 0,
				'pm_unread'		=> 0,
				'pm_forwarded'	=> ($mode == 'forward') ? 1 : 0))
			);
		}
	}

	// Set user last post time
	if ($mode == 'reply' || $mode == 'quote' || $mode == 'quotepost' || $mode == 'forward' || $mode == 'post')
	{
		$sql = 'UPDATE ' . USERS_TABLE . "
			SET user_lastpost_time = $current_time
			WHERE user_id = " . $data['from_user_id'];
		$db->sql_query($sql);
	}

	// Submit Attachments
	if (!empty($data['attachment_data']) && $data['msg_id'] && in_array($mode, array('post', 'reply', 'quote', 'quotepost', 'edit', 'forward')))
	{
		$space_taken = $files_added = 0;
		$orphan_rows = array();

		foreach ($data['attachment_data'] as $pos => $attach_row)
		{
			$orphan_rows[(int) $attach_row['attach_id']] = array();
		}

		if (sizeof($orphan_rows))
		{
			$sql = 'SELECT attach_id, filesize, physical_filename
				FROM ' . ATTACHMENTS_TABLE . '
				WHERE ' . $db->sql_in_set('attach_id', array_keys($orphan_rows)) . '
					AND in_message = 1
					AND is_orphan = 1
					AND poster_id = ' . $user->data['user_id'];
			$result = $db->sql_query($sql);

			$orphan_rows = array();
			while ($row = $db->sql_fetchrow($result))
			{
				$orphan_rows[$row['attach_id']] = $row;
			}
			$db->sql_freeresult($result);
		}

		foreach ($data['attachment_data'] as $pos => $attach_row)
		{
			if ($attach_row['is_orphan'] && !isset($orphan_rows[$attach_row['attach_id']]))
			{
				continue;
			}

			if (!$attach_row['is_orphan'])
			{
				// update entry in db if attachment already stored in db and filespace
				$sql = 'UPDATE ' . ATTACHMENTS_TABLE . "
					SET attach_comment = '" . $db->sql_escape($attach_row['attach_comment']) . "'
					WHERE attach_id = " . (int) $attach_row['attach_id'] . '
						AND is_orphan = 0';
				$db->sql_query($sql);
			}
			else
			{
				// insert attachment into db
				if (!@file_exists($phpbb_root_path . $config['upload_path'] . '/' . utf8_basename($orphan_rows[$attach_row['attach_id']]['physical_filename'])))
				{
					continue;
				}

				$space_taken += $orphan_rows[$attach_row['attach_id']]['filesize'];
				$files_added++;

				$attach_sql = array(
					'post_msg_id'		=> $data['msg_id'],
					'topic_id'			=> 0,
					'is_orphan'			=> 0,
					'poster_id'			=> $data['from_user_id'],
					'attach_comment'	=> $attach_row['attach_comment'],
				);

				$sql = 'UPDATE ' . ATTACHMENTS_TABLE . ' SET ' . $db->sql_build_array('UPDATE', $attach_sql) . '
					WHERE attach_id = ' . $attach_row['attach_id'] . '
						AND is_orphan = 1
						AND poster_id = ' . $user->data['user_id'];
				$db->sql_query($sql);
			}
		}

		if ($space_taken && $files_added)
		{
			set_config_count('upload_dir_size', $space_taken, true);
			set_config_count('num_files', $files_added, true);
		}
	}

	// Delete draft if post was loaded...
	$draft_id = request_var('draft_loaded', 0);
	if ($draft_id)
	{
		$sql = 'DELETE FROM ' . DRAFTS_TABLE . "
			WHERE draft_id = $draft_id
				AND user_id = " . $data['from_user_id'];
		$db->sql_query($sql);
	}

	$db->sql_transaction('commit');

	// Send Notifications
	$pm_data = array_merge($data, array(
		'message_subject'		=> $subject,
		'recipients'			=> $recipients,
	));

	$phpbb_notifications = $phpbb_container->get('notification_manager');

	if ($mode == 'edit')
	{
		$phpbb_notifications->update_notifications('notification.type.pm', $pm_data);
	}
	else
	{
		$phpbb_notifications->add_notifications('notification.type.pm', $pm_data);
	}

	/**
	* Get PM message ID after submission to DB
	*
	* @event core.submit_pm_after
	* @var	string	mode	PM Post mode - post|reply|quote|quotepost|forward|edit
	* @var	string	subject	Subject of the private message
	* @var	array	data	The whole row data of the PM.
	* @var	array	pm_data	The data sent to notification class
	* @since 3.1.0-b5
	*/
	$vars = array('mode', 'subject', 'data', 'pm_data');
	extract($phpbb_dispatcher->trigger_event('core.submit_pm_after', compact($vars)));

	return $data['msg_id'];
}

/**
* Display Message History
*/
function message_history($msg_id, $user_id, $message_row, $folder, $in_post_mode = false)
{
	global $db, $user, $config, $template, $phpbb_root_path, $phpEx, $auth;

	// Select all receipts and the author from the pm we currently view, to only display their pm-history
	$sql = 'SELECT author_id, user_id
		FROM ' . PRIVMSGS_TO_TABLE . "
		WHERE msg_id = $msg_id
			AND folder_id <> " . PRIVMSGS_HOLD_BOX;
	$result = $db->sql_query($sql);

	$recipients = array();
	while ($row = $db->sql_fetchrow($result))
	{
		$recipients[] = (int) $row['user_id'];
		$recipients[] = (int) $row['author_id'];
	}
	$db->sql_freeresult($result);
	$recipients = array_unique($recipients);

	// Get History Messages (could be newer)
	$sql = 'SELECT t.*, p.*, u.*
		FROM ' . PRIVMSGS_TABLE . ' p, ' . PRIVMSGS_TO_TABLE . ' t, ' . USERS_TABLE . ' u
		WHERE t.msg_id = p.msg_id
			AND p.author_id = u.user_id
			AND t.folder_id NOT IN (' . PRIVMSGS_NO_BOX . ', ' . PRIVMSGS_HOLD_BOX . ')
			AND ' . $db->sql_in_set('t.author_id', $recipients, false, true) . "
			AND t.user_id = $user_id";

	// We no longer need those.
	unset($recipients);

	if (!$message_row['root_level'])
	{
		$sql .= " AND (p.root_level = $msg_id OR (p.root_level = 0 AND p.msg_id = $msg_id))";
	}
	else
	{
		$sql .= " AND (p.root_level = " . $message_row['root_level'] . ' OR p.msg_id = ' . $message_row['root_level'] . ')';
	}
	$sql .= ' ORDER BY p.message_time DESC';

	$result = $db->sql_query($sql);
	$row = $db->sql_fetchrow($result);

	if (!$row)
	{
		$db->sql_freeresult($result);
		return false;
	}

	$title = $row['message_subject'];

	$rowset = array();
	$folder_url = append_sid("{$phpbb_root_path}ucp.$phpEx", 'i=pm') . '&amp;folder=';

	do
	{
		$folder_id = (int) $row['folder_id'];

		$row['folder'][] = (isset($folder[$folder_id])) ? '<a href="' . $folder_url . $folder_id . '">' . $folder[$folder_id]['folder_name'] . '</a>' : $user->lang['UNKNOWN_FOLDER'];

		if (isset($rowset[$row['msg_id']]))
		{
			$rowset[$row['msg_id']]['folder'][] = (isset($folder[$folder_id])) ? '<a href="' . $folder_url . $folder_id . '">' . $folder[$folder_id]['folder_name'] . '</a>' : $user->lang['UNKNOWN_FOLDER'];
		}
		else
		{
			$rowset[$row['msg_id']] = $row;
		}
	}
	while ($row = $db->sql_fetchrow($result));
	$db->sql_freeresult($result);

	if (sizeof($rowset) == 1 && !$in_post_mode)
	{
		return false;
	}

	$title = censor_text($title);

	$url = append_sid("{$phpbb_root_path}ucp.$phpEx", 'i=pm');
	$next_history_pm = $previous_history_pm = $prev_id = 0;

	// Re-order rowset to be able to get the next/prev message rows...
	$rowset = array_values($rowset);

	for ($i = 0, $size = sizeof($rowset); $i < $size; $i++)
	{
		$row = &$rowset[$i];
		$id = (int) $row['msg_id'];

		$author_id	= $row['author_id'];
		$folder_id	= (int) $row['folder_id'];

		$subject	= $row['message_subject'];
		$message	= $row['message_text'];

		$message = censor_text($message);

		$decoded_message = false;

		if ($in_post_mode && $auth->acl_get('u_sendpm') && $author_id != ANONYMOUS)
		{
			$decoded_message = $message;
			decode_message($decoded_message, $row['bbcode_uid']);

			$decoded_message = bbcode_nl2br($decoded_message);
		}

		$parse_flags = ($row['bbcode_bitfield'] ? OPTION_FLAG_BBCODE : 0);
		$parse_flags |= ($row['enable_smilies'] ? OPTION_FLAG_SMILIES : 0);

		$message = generate_text_for_display($message, $row['bbcode_uid'], $row['bbcode_bitfield'], $parse_flags, false);

		$subject = censor_text($subject);

		if ($id == $msg_id)
		{
			$next_history_pm = (isset($rowset[$i + 1])) ? (int) $rowset[$i + 1]['msg_id'] : 0;
			$previous_history_pm = $prev_id;
		}

		$template->assign_block_vars('history_row', array(
			'MESSAGE_AUTHOR_QUOTE'		=> (($decoded_message) ? addslashes(get_username_string('username', $author_id, $row['username'], $row['user_colour'], $row['username'])) : ''),
			'MESSAGE_AUTHOR_FULL'		=> get_username_string('full', $author_id, $row['username'], $row['user_colour'], $row['username']),
			'MESSAGE_AUTHOR_COLOUR'		=> get_username_string('colour', $author_id, $row['username'], $row['user_colour'], $row['username']),
			'MESSAGE_AUTHOR'			=> get_username_string('username', $author_id, $row['username'], $row['user_colour'], $row['username']),
			'U_MESSAGE_AUTHOR'			=> get_username_string('profile', $author_id, $row['username'], $row['user_colour'], $row['username']),

			'SUBJECT'			=> $subject,
			'SENT_DATE'			=> $user->format_date($row['message_time']),
			'MESSAGE'			=> $message,
			'FOLDER'			=> implode($user->lang['COMMA_SEPARATOR'], $row['folder']),
			'DECODED_MESSAGE'	=> $decoded_message,

			'S_CURRENT_MSG'		=> ($row['msg_id'] == $msg_id),
			'S_AUTHOR_DELETED'	=> ($author_id == ANONYMOUS) ? true : false,
			'S_IN_POST_MODE'	=> $in_post_mode,

			'MSG_ID'			=> $row['msg_id'],
			'U_VIEW_MESSAGE'	=> "$url&amp;f=$folder_id&amp;p=" . $row['msg_id'],
			'U_QUOTE'			=> (!$in_post_mode && $auth->acl_get('u_sendpm') && $author_id != ANONYMOUS) ? "$url&amp;mode=compose&amp;action=quote&amp;f=" . $folder_id . "&amp;p=" . $row['msg_id'] : '',
			'U_POST_REPLY_PM'	=> ($author_id != $user->data['user_id'] && $author_id != ANONYMOUS && $auth->acl_get('u_sendpm')) ? "$url&amp;mode=compose&amp;action=reply&amp;f=$folder_id&amp;p=" . $row['msg_id'] : '')
		);
		unset($rowset[$i]);
		$prev_id = $id;
	}

	$template->assign_vars(array(
		'QUOTE_IMG'			=> $user->img('icon_post_quote', $user->lang['REPLY_WITH_QUOTE']),
		'HISTORY_TITLE'		=> $title,

		'U_VIEW_NEXT_HISTORY'		=> ($next_history_pm) ? "$url&amp;p=" . $next_history_pm : '',
		'U_VIEW_PREVIOUS_HISTORY'	=> ($previous_history_pm) ? "$url&amp;p=" . $previous_history_pm : '',
	));

	return true;
}

/**
* Set correct users max messages in PM folder.
* If several group memberships define different amount of messages, the highest will be chosen.
*/
function set_user_message_limit()
{
	global $user, $db, $config;

	// Get maximum about from user memberships - if it is 0, there is no limit set and we use the maximum value within the config.
	$sql = 'SELECT MAX(g.group_message_limit) as max_message_limit
		FROM ' . GROUPS_TABLE . ' g, ' . USER_GROUP_TABLE . ' ug
		WHERE ug.user_id = ' . $user->data['user_id'] . '
			AND ug.user_pending = 0
			AND ug.group_id = g.group_id';
	$result = $db->sql_query($sql);
	$message_limit = (int) $db->sql_fetchfield('max_message_limit');
	$db->sql_freeresult($result);

	$user->data['message_limit'] = (!$message_limit) ? $config['pm_max_msgs'] : $message_limit;
}

/**
* Generates an array of coloured recipient names from a list of PMs - (groups & users)
*
* @param	array	$pm_by_id	An array of rows from PRIVMSGS_TABLE, keys are the msg_ids.
*
* @return	array				2D Array: array(msg_id => array('username or group string', ...), ...)
*								Usernames are generated with {@link get_username_string get_username_string}
*								Groups are coloured and have a link to the membership page
*/
function get_recipient_strings($pm_by_id)
{
	global $db, $phpbb_root_path, $phpEx, $user;

	$address_list = $recipient_list = $address = array();

	$_types = array('u', 'g');

	foreach ($pm_by_id as $message_id => $row)
	{
		$address[$message_id] = rebuild_header(array('to' => $row['to_address'], 'bcc' => $row['bcc_address']));

		foreach ($_types as $ug_type)
		{
			if (isset($address[$message_id][$ug_type]) && sizeof($address[$message_id][$ug_type]))
			{
				foreach ($address[$message_id][$ug_type] as $ug_id => $in_to)
				{
					$recipient_list[$ug_type][$ug_id] = array('name' => $user->lang['NA'], 'colour' => '');
				}
			}
		}
	}

	foreach ($_types as $ug_type)
	{
		if (!empty($recipient_list[$ug_type]))
		{
			if ($ug_type == 'u')
			{
				$sql = 'SELECT user_id as id, username as name, user_colour as colour
					FROM ' . USERS_TABLE . '
					WHERE ';
			}
			else
			{
				$sql = 'SELECT group_id as id, group_name as name, group_colour as colour, group_type
					FROM ' . GROUPS_TABLE . '
					WHERE ';
			}
			$sql .= $db->sql_in_set(($ug_type == 'u') ? 'user_id' : 'group_id', array_map('intval', array_keys($recipient_list[$ug_type])));

			$result = $db->sql_query($sql);

			while ($row = $db->sql_fetchrow($result))
			{
				if ($ug_type == 'g')
				{
					$row['name'] = ($row['group_type'] == GROUP_SPECIAL) ? $user->lang['G_' . $row['name']] : $row['name'];
				}

				$recipient_list[$ug_type][$row['id']] = array('name' => $row['name'], 'colour' => $row['colour']);
			}
			$db->sql_freeresult($result);
		}
	}

	foreach ($address as $message_id => $adr_ary)
	{
		foreach ($adr_ary as $type => $id_ary)
		{
			foreach ($id_ary as $ug_id => $_id)
			{
				if ($type == 'u')
				{
					$address_list[$message_id][] = get_username_string('full', $ug_id, $recipient_list[$type][$ug_id]['name'], $recipient_list[$type][$ug_id]['colour']);
				}
				else
				{
					$user_colour = ($recipient_list[$type][$ug_id]['colour']) ? ' style="font-weight: bold; color:#' . $recipient_list[$type][$ug_id]['colour'] . '"' : '';
					$link = '<a href="' . append_sid("{$phpbb_root_path}memberlist.$phpEx", 'mode=group&amp;g=' . $ug_id) . '"' . $user_colour . '>';
					$address_list[$message_id][] = $link . $recipient_list[$type][$ug_id]['name'] . (($link) ? '</a>' : '');
				}
			}
		}
	}

	return $address_list;
}