[prev in list] [next in list] [prev in thread] [next in thread] 

List:       imp
Subject:    Re: [horde] Duration of database migration
From:       Michael Menge <michael.menge () zdv ! uni-tuebingen ! de>
Date:       2017-04-18 9:27:45
Message-ID: 20170418112745.Horde.vjd_s5I0UUlggBlPaRQBo4Z () webmail ! uni-tuebingen ! de
[Download RAW message or body]


Quoting Jan Schneider <jan@horde.org>:

> Zitat von Aras YORGANCI <yorgancia@itu.edu.tr>:
>
>> Hello dear list,
>>
>> I want to upgrade our Horde to latest version with fresh install. In the
>> test enviroment, when I run horde-db-migrate command on a dump of
>> production MySQL database, it takes too much time. Especially for
>> Horde_History and Turba modules, database migration lasts about 2 hours. I
>> know there is many facts that affect the process like hardware, network
>> etc. but I wonder that is there any solution shorten to database migration
>> on tables level.
>>
>> Regards.
>>
>> Aras
>
> No. You could parallelize some of the migration steps so they don't  
> add up, but the individual migrations have already been tuned as  
> much as possible.

here is the script we used to parallelize the migration steps.

----
horde-db-migrate imp &>> horde-db-migrate-imp.log &
horde-db-migrate ingo &>> horde-db-migrate-ingo.log &
horde-db-migrate kronolith 17 &>> horde-db-migrate-kronolith.log &
horde-db-migrate turba &>> horde-db-migrate-turba.log &
horde-db-migrate nag 9 &>> horde-db-migrate-nag.log &
horde-db-migrate mnemo 5 &>> horde-db-migrate-mnemo.log &
horde-db-migrate content &>> horde-db-migrate-content.log  &
horde-db-migrate Horde_ActiveSync &>> horde-db-migrate-ActiveSync.log &
horde-db-migrate Horde_Alarm &>> horde-db-migrate-Alarm.log &
horde-db-migrate Horde_Auth &>> horde-db-migrate-Auth.log &
horde-db-migrate Horde_Cache &>> horde-db-migrate-Cache.log &
horde-db-migrate Horde_Core &>> horde-db-migrate-Core.log &
horde-db-migrate Horde_Dav &>> horde-db-migrate-Dav.log &
horde-db-migrate Horde_Group &>> horde-db-migrate-Group.log &
horde-db-migrate Horde_History &>> horde-db-migrate-History.log &
horde-db-migrate Horde_Imap_Client &>> horde-db-migrate-Imap_Client.log &
horde-db-migrate Horde_Lock &>> horde-db-migrate-Lock.log &
horde-db-migrate Horde_Perms &>> horde-db-migrate-Perms.log &
horde-db-migrate Horde_Prefs &>> horde-db-migrate-Prefs.log &
horde-db-migrate Horde_Queue &>> horde-db-migrate-Queue.log &
horde-db-migrate Horde_SessionHandler &>>  
horde-db-migrate-SessionHandler.log &
horde-db-migrate Horde_SyncMl &>> horde-db-migrate-SyncML.log &
horde-db-migrate Horde_Token &>> horde-db-migrate-Token.log &
horde-db-migrate Horde_Vfs &>> horde-db-migrate-Vfs.log &
wait
horde-db-migrate imp &>> horde-db-migrate-imp.log &
horde-db-migrate ingo &>> horde-db-migrate-ingo.log &
horde-db-migrate content &>> horde-db-migrate-content.log  &
horde-db-migrate kronolith &>> horde-db-migrate-kronolith.log
horde-db-migrate turba &>> horde-db-migrate-turba.log
horde-db-migrate nag &>> horde-db-migrate-nag.log
horde-db-migrate mnemo &>> horde-db-migrate-mnemo.log
wait
horde-db-migrate &>> horde-db-migrate-all.log
horde-db-migrate &>> horde-db-migrate-all.log
----

> Suggestions or ideas for further improvements are always welcome though.
>

On our migration the script imp/migration/3_imp_maillog_update.php was  
running very long.
We could speed it up by using the following SQL statement, but we are  
using PostgreSQL and the
statement uses information about our UserIDs, so it is not viable as  
general improvement for horde.
But you might be able to adapt the statement to mysql and your environment.

UPDATE horde_histories SET object_uid=regexp_replace(object_uid,  
'^imp\.([a-z0-9]{7})\.', 'imp:\1:')  WHERE object_uid like 'imp.%';

Regards

   Michael

> -- 
> Jan Schneider
> The Horde Project
> https://www.horde.org/
>
> -- 
> Horde mailing list
> Frequently Asked Questions: http://horde.org/faq/
> To unsubscribe, mail: horde-unsubscribe@lists.horde.org



--------------------------------------------------------------------------------
M.Menge                                Tel.: (49) 7071/29-70316
Universität Tübingen                   Fax.: (49) 7071/29-5912
Zentrum für Datenverarbeitung          mail:  
michael.menge@zdv.uni-tuebingen.de
Wächterstraße 76
72074 Tübingen

-- 
Horde mailing list
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: horde-unsubscribe@lists.horde.org

[prev in list] [next in list] [prev in thread] [next in thread]