wiki:howto/ZarafaMigrateConnectux

Context Navigation

Version 36 (modified by Edwin Eefting, 13 years ago) (diff)
--

Zarafa migratie van Connectux naar SYN-3

Algemeen

Update uw connectux server naar 6.20, oudere versies worden niet ondersteund door deze procedure. (op aanvraag wel mogelijk)

Zorg dat de users zijn aangemaakt op de SYN-3 server. (Als de installatie te groot is om dit handigmatig te doen, neem dan contact op met ons. Wij kunnen hiervoor scripts maken.)

Zet de mail ontvangst op beide servers tijdens de conversie uit. Anders kan nieuwe mail verloren gaan.

Deze procedure verwijderd alle bestaande Zarafa gegevens op de SYN-3 server.

Database kopieren van Connectux naar SYN-3

Zorg dat u mimimaal SYN-3 versie 4.4.1 hebt.

Zorg dat de Zarafa database op de SYN-3 server leeg is:

[Syn-3] root@syn.kado.local ~# svcstop /service/zarafa-server/                                                                                                                               
Stopping /service/zarafa-server/ ....OK
[Syn-3] root@syn.kado.local ~#  mysqladmin -f -p`cat /etc/my.passwd` drop zarafa && mysqladmin -p`cat /etc/my.passwd` create zarafa && rm -r /home/system/zarafa/*                
Database "zarafa" dropped
[Syn-3] root@syn.kado.local ~#

Achterhaal de mysql user en pass van de database op de Connectux server:

linux:/md0 # grep mysql /etc/araneus/zarafa/server.cfg
mysql_user              = root
mysql_password          = 12345
mysql_database          = zarafa

Om tijd te besparen, zorgen we dat de database rechtstreeks geïmporteerd word op de SYN-3 server, zonder tussenkomt van tijdelijke files. Voer uit op de Connectux server:

linux:/md0 # mysqldump --quick -uroot -p'12345' zarafa | ssh root@10.0.0.6 'mysql -p`cat /etc/my.passwd` zarafa'
root@10.0.0.6's password:

Afhankelijk van de database grote, kan dit soms een paar uur duren. Met dstat en top kunt u op de syn3 server zien wat er gaande is.

Mocht de conversie onderbroken worden met de melding 'ERROR 1153 (08S01) at line 24818: Got a packet bigger than 'max_allowed_packet' bytes', dan dient u deze instelling hoger te maken in beide mysql config files

User ID's en database conversie

Kopieer de passwd file van de oude server naar /root

Gebruik dit script voor de conversie:

#!/bin/bash
# Connectux2Zarafa Conversion script (C)2010 DatuX - Released under GPL 

svcstop /service/zarafa-server


SQLPW=`cat /etc/my.passwd`

echo "SYN-3: Reading user ID converions:"
echo "old user id -> username -> new user id"
#for OLDUID in `echo 'select externid from users where isgroup=0 and externid>0' |mysql -p$SQLPW zarafa`; do
for OLDUID in `echo 'select externid from users where object_type=1 and externid>0' |mysql -p$SQLPW zarafa`; do
    [ "$OLDUID" == "externid" ] && continue;

    USER=`cat ~/passwd |egrep ".+:.+:$OLDUID:"|cut -f1 -d:`
    NEWUID=`getent passwd |egrep "$USER:.+:.+:"|cut -f3 -d:`
    echo "$OLDUID -> $USER -> $NEWUID"
    if [ "$USER" == "" ];  then
        echo "Old user $OLDUID not found in old passwd file!"
        [ "$FORCE" ] ||  exit 1
        continue;
    fi
    if [ "$NEWUID" == "" ];  then
        echo "User $USER does not exist on this system yet!"
        [ "$FORCE" ] ||  exit 1
        continue;
    fi
    QUERY="$QUERY""update users set externid='$NEWUID' where (externid='$OLDUID');"
done

if [ "$QUERY" == "" ];then
    echo "Something went wrong.."
    [ "$FORCE" ] ||  exit 1
fi


echo "SYN-3: Updating database:"
echo "query: $QUERY"
echo -e "$QUERY" | mysql -p$SQLPW zarafa || exit 1

echo "SYN-3: Starting zarafa in conversion mode:"
rm /var/log/zarafa/server.log
zarafa-server --ignore-attachment-storage-conflict
sleep 10
tail -f /var/log/zarafa/server.log &

while ! grep 'Startup succeeded' /var/log/zarafa/server.log; do
    echo "SYN-3: Waiting for conversion to finish..."
    sleep 60
done

killall zarafa-server tail

echo "SYN-3: Converting attachments from database to files"
perl /usr/doc/zarafa/db-convert-attachments-to-files root $SQLPW zarafa /home/system/zarafa delete || exit 1
chown -R zarafa:zarafa /home/system/zarafa || exit 1


svcstart /service/zarafa-server || exit 1
echo "SYN-3: Conversion done, Zarafa should be available now."

Start met FORCE=1 ./convert als u wilt doorgaan zonder user-controle. (voor als er users ontbreken)\

Problemen tijdens conversie

Hier een voorbeeld van een standaard probleem:

[Syn-3] root@linux.ours.local ~# FORCE=1 ./convert
Stopping /service/zarafa-server ...OK
SYN-3: Reading user ID converions:
old user id -> username -> new user id
10001 -> osman -> 1002
10002 -> herman ->
User herman does not exist on this system yet!
10003 -> sunplan ->
User sunplan does not exist on this system yet!
SYN-3: Updating database:
query: update users set externid='1002' where (externid='10001');\n
SYN-3: Starting zarafa in conversion mode:
SYN-3: Waiting for conversion to finish...
Thu Jun  9 22:21:40 2011: Starting zarafa-server version 6,40,8,27223, pid 24485
Thu Jun  9 22:21:40 2011: Connection to database 'zarafa' succeeded
Thu Jun  9 22:21:40 2011: WARNING: zarafa-licensed not running, commercial features will not be available until it's started.
Thu Jun  9 22:21:40 2011: Start: Adding externid to 'object' table
Thu Jun  9 22:21:40 2011: Done: Adding externid to 'object' table
Thu Jun  9 22:21:40 2011: Start: Creating Single Instance Attachment table
SYN-3: Waiting for conversion to finish...
SYN-3: Waiting for conversion to finish...
SYN-3: Waiting for conversion to finish...
SYN-3: Waiting for conversion to finish...
SYN-3: Waiting for conversion to finish...
SYN-3: Waiting for conversion to finish...
SYN-3: Waiting for conversion to finish...
SYN-3: Waiting for conversion to finish...
Thu Jun  9 22:29:59 2011: Done: Creating Single Instance Attachment table
Thu Jun  9 22:29:59 2011: Start: Locking multiserver capability
Thu Jun  9 22:29:59 2011: Done: Locking multiserver capability
Thu Jun  9 22:29:59 2011: Start: Creating Addressbook Changes table
Thu Jun  9 22:29:59 2011: Done: Creating Addressbook Changes table
Thu Jun  9 22:29:59 2011: Start: Updating 'singleinstances' table to correct tag value
Thu Jun  9 22:29:59 2011: Done: Updating 'singleinstances' table to correct tag value
Thu Jun  9 22:29:59 2011: Start: Create table: synced messages
Thu Jun  9 22:29:59 2011: Done: Create table: synced messages
Thu Jun  9 22:29:59 2011: Start: Force Addressbook Resync
Thu Jun  9 22:29:59 2011: Skipped: Force Addressbook Resync
Thu Jun  9 22:29:59 2011: Start: Rename objecttype columns to objectclass
Thu Jun  9 22:29:59 2011: Done: Rename objecttype columns to objectclass
Thu Jun  9 22:29:59 2011: Start: Convert objecttype columns to objectclass values
Thu Jun  9 22:29:59 2011: Done: Convert objecttype columns to objectclass values
Thu Jun  9 22:29:59 2011: Start: Add object MV property table
Thu Jun  9 22:29:59 2011: Done: Add object MV property table
Thu Jun  9 22:29:59 2011: Start: Link objects in DB plugin through companyid
Thu Jun  9 22:29:59 2011: Done: Link objects in DB plugin through companyid
Thu Jun  9 22:29:59 2011: Start: Update outgoingqueue key
Thu Jun  9 22:29:59 2011: 0x00000009ea8568: SQL Failed: Can't DROP 'PRIMARY'; check that column/key exists, Query: "ALTER TABLE outgoingqueue DROP PRIMARY KEY, ADD PRIMARY KEY (`hierarchy_id`,`flags`,`store_id`)"
Thu Jun  9 22:29:59 2011: Failed: Rollback database
Thu Jun  9 22:30:00 2011: Can't update the database: Unable to upgrade zarafa from version 6.20.18389 to 6.40.8.27223
Thu Jun  9 22:30:00 2011: Server shutdown complete.
SYN-3: Waiting for conversion to finish...

Blijkbaar verwacht zarafa een index die niet bestaat, handmatige de query aanpassen lost het probleem op. (soms moet dit met meerdere indexen)

[Syn-3] root@linux.ours.local ~# mysql -p`cat /etc/my.passwd` zarafa
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 66
Server version: 5.0.91 Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> ALTER TABLE outgoingqueue DROP PRIMARY KEY, ADD PRIMARY KEY (`hierarchy_id`,`flags`,`store_id`);
ERROR 1091 (42000): Can't DROP 'PRIMARY'; check that column/key exists
mysql> ALTER TABLE outgoingqueue ADD PRIMARY KEY (`hierarchy_id`,`flags`,`store_id`);
Query OK, 0 rows affected (0.09 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> ALTER TABLE outgoingqueue DROP PRIMARY KEY, ADD PRIMARY KEY (`hierarchy_id`,`flags`,`store_id`);
Query OK, 0 rows affected (0.05 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql>

We starten het script nogmaals, in FORCE mode omdat de user conversie al geweest is:

[Syn-3] root@linux.ours.local ~# FORCE=1 ./convert 
Stopping /service/zarafa-server ...OK
SYN-3: Reading user ID converions:
old user id -> username -> new user id
ERROR 1054 (42S22) at line 1: Unknown column 'object_type' in 'where clause'
Something went wrong with user conversion..
SYN-3: Updating database:
query: 
SYN-3: Starting zarafa in conversion mode:
SYN-3: Waiting for conversion to finish...
Thu Jun  9 22:36:18 2011: Done: Update acl key
Thu Jun  9 22:36:18 2011: Start: Update externid in object table
Thu Jun  9 22:36:18 2011: Done: Update externid in object table
Thu Jun  9 22:36:18 2011: Start: Update keys in 'changes' table
Thu Jun  9 22:36:18 2011: Done: Update keys in 'changes' table
Thu Jun  9 22:36:18 2011: Start: Update mvproperties key
Thu Jun  9 22:36:18 2011: 0x0000000a356568: SQL Failed: Incorrect table definition; there can be only one auto column and it must be defined as a key, Query: "ALTER TABLE mvproperties DROP PRIMARY KEY, ADD PRIMARY KEY (`hierarchyid`,`tag`,`type`,`orderid`), DROP KEY `hi`"
Thu Jun  9 22:36:18 2011: Failed: Rollback database
Thu Jun  9 22:36:18 2011: Can't update the database: Unable to upgrade zarafa from version 6.40.27223 to 6.40.8.27223
Thu Jun  9 22:36:18 2011: Server shutdown complete.

Nu gaat het mis op een andere table, we doen dezelfde sql magic:

[Syn-3] root@linux.ours.local ~# mysql -p`cat /etc/my.passwd` zarafa
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 77
Server version: 5.0.91 Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> ALTER TABLE mvproperties DROP PRIMARY KEY, ADD PRIMARY KEY (`hierarchyid`,`tag`,`type`,`orderid`), DROP KEY `hi`;
ERROR 1075 (42000): Incorrect table definition; there can be only one auto column and it must be defined as a key
....
mysql> ALTER TABLE mvproperties  DROP KEY `hi`;
Query OK, 103 rows affected (0.07 sec)
Records: 103  Duplicates: 0  Warnings: 0

Na lang prutsen blijkt het nodig zijn deze key te droppen, hierna start zarafa correct en gaat de conversie verder:

[Syn-3] root@linux.lazytours.local ~# FORCE=1 ./convert 
Stopping /service/zarafa-server ...OK
SYN-3: Reading user ID converions:
old user id -> username -> new user id
ERROR 1054 (42S22) at line 1: Unknown column 'object_type' in 'where clause'
Something went wrong with user conversion..
SYN-3: Updating database:
query: 
SYN-3: Starting zarafa in conversion mode:
Thu Jun  9 22:43:11 2011: Start: Update mvproperties key
Thu Jun  9 22:43:12 2011: Startup succeeded on pid 21456
Thu Jun  9 22:43:12 2011: Done: Update mvproperties key
Thu Jun  9 22:43:12 2011: Start: Update DB plugin group to security groups
Thu Jun  9 22:43:12 2011: Done: Update DB plugin group to security groups
Thu Jun  9 22:43:12 2011: Start: Update DB/Unix plugin sendas settings
Thu Jun  9 22:43:12 2011: Done: Update DB/Unix plugin sendas settings
Thu Jun  9 22:43:12 2011: Ignoring attachment storing conflict as requested. Attachments are now stored with option 'files'
Thu Jun  9 22:43:12 2011: Listening for pipe connections on /var/run/zarafa
Thu Jun  9 22:43:12 2011: Listening for TCP connections on port 236
Thu Jun  9 22:43:12 2011: Startup succeeded on pid 21456
./convert: line 52: 21472 Terminated              tail -f /var/log/zarafa/server.log
SYN-3: Converting attachments from database to files
The size of all attachments in the database is: 5719539712 Bytes (5.33 GB)
Available space is: 466469113856 Bytes (434.43 GB)
Finding all attachments...
Processing 32922 attachments

Bovenstaande problemen heb ik nu 2x achter elkaar gehad, dus het lijk een standaard issue met 6.20->6.40.9 conversies. (de 6.20 database stamt wel af van een nog veel oudere zarafa versie die meerdere malen geupgrade is)

Gepost op zarafa forum: http://forums.zarafa.com/viewtopic.php?f=10&t=6783

Download in other formats:

Plain Text