Exchangetoox: Difference between revisions

From Open-Xchange
 
(25 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Introduction ==
== Introduction ==


The document describes a way how mailboxes can be migrated from MS Exchange to a cyrus IMAP server using imapsync without the need to know the users passwords.
The document describes a way how mailboxes can be migrated from MS Exchange™ to a cyrus IMAP server using imapsync without the need to know the user passwords.


== Pre requirements ==
== Requirements ==


# a cyrus IMAP server and a admin account
# A cyrus IMAP server and an admin account
#: the admin accounts for cyrus are defined in the parameter '''admins: '''of the configuration file '''/etc/imapd.conf '''. In this examples, the admin user name is '''cyrus''' with password '''cpass'''.
#: the admin accounts for cyrus are defined in the '''admins: ''' parameter of the '''/etc/imapd.conf ''' configuration file. In the examples, the admin user name is '''cyrus''' with password '''cpass'''.
# Microsoft Exchange 2003
# Microsoft Exchange 2003™
#:others Exchange versions might work, too. 2003 was the version this howto was written with.
#:other Microsoft Exchange™ versions might work, too. This howto was written for the 2003 version.
# postmaster account for MS Exchange
# IMAP connector installed and activated for Microsoft Exchange™
#:the Exchange account for migration needs to have the rights to access all mailboxes on the Exchange server. In the examples the user name is '''migration''' with password '''mpass'''. The user can be created following these steps:
# Postmaster account for MS Exchange™
## create a new '''DWORD''' in '''HKEY_Current_User\Software\ Microsoft\Exchange\ExAdmin''' with the name '''ShowSecurityPage''' and set this to '''1'''.
#:The Exchange™ account for migration needs to have the permission to access all mailboxes on the MS Exchange™ server. In the examples the user name is '''migration''' with password '''mpass'''. The user can be created following these steps:
## open the Active Directory User and Computer console and create a new user with the name '''migration''', password '''mpass''' and Exchange attributes.
## Create a new '''DWORD''' in '''HKEY_Current_User\Software\ Microsoft\Exchange\ExAdmin''' with the name '''ShowSecurityPage''' and set this to '''1'''.
## reopen the Exchange System Manger and give that new user full access to the entire mail organization (right click on the most upper entry naming your organization -> properties -> security -> add)
## Open the Active Directory User and Computer console and create a new user with the name '''migration''', password '''mpass''' and MS Exchange™ attributes.
## Reopen the MS Exchange™ System Manager and give that new user full access to the entire mail organization (right click on the uppermost entry naming your organization -> properties -> security -> add)


== Testing the admin users ==
== Testing the admin users ==


Those example IMAP logins descibe a telnet session where a succesfull IMAP connections gets created and closed (cyrus is running on localhost and the Exchange server on 10.20.30.170)
Those example IMAP logins describe a telnet session where a succesful IMAP connection is created and closed (cyrus is running on localhost and the Exchange server on 10.20.30.170)


=== Test if the cyrus admin can access the server ===
=== Test if the cyrus admin can access the server ===
Line 34: Line 35:
  []#
  []#


=== Test if the migration user can login to its own mailbox ===
=== Test if the migration user can log in to own mailbox ===
   []# telnet 10.20.30.170 imap
   []# telnet 10.20.30.170 imap
  Trying 10.20.30.170...
  Trying 10.20.30.170...
Line 48: Line 49:
  []#
  []#


=== Test if the migration user can login for user test in domain ===
=== Test if the migration user can log in for user test in domain ===
 
This tests shows if it is possible to access the mail box "test" with the credentials of the migration user:


  []# telnet 10.20.30.170 imap
  []# telnet 10.20.30.170 imap
Line 65: Line 68:
== Migrating a single user ==
== Migrating a single user ==


Access to the Exchange mailbox is done via '''<domain>/migration/<user to migrate>''' and the password of the migration account. In cyrus the Data is stored as user with the credentials of the admin account. The imapsync command is as follows:
Access to the MS Exchange™ mailbox is done via '''<domain>/migration/<user to migrate>''' and the password of the migration account. In cyrus the Data is stored as user with the credentials of the admin account. The imapsync command is as follows:


  imapsync --host1 <ms exchange> --user1 <domain>/migration/<user to migrtate1> --password1 \
  imapsync --host1 <ms exchange> --user1 <domain>/migration/<user to migrtate1> --password1 \
Line 93: Line 96:
|-
|-
| --exclude
| --exclude
| a list of standart folders not to migrate
| a list of system folders not to migrate
|-
|-
|}
|}


The above command does actually do a try run without migrating any data. This is because of the '''--try''' option. It is needed to see what special folders are available that do not need to be migrated. This are the folders calendar,contacts,tasks,notices and shared folder. They are different in various languages. It is also save to check if the standard folders are the same on both sides (if not, that can be tuned with --regextrans2 and a regular expression that fixes that).
The above command does actually do a try run without migrating any data. This is because of the '''--dry''' option. It is needed to see what special folders are available that do not need to be migrated. These are the folders calendar,contacts,tasks,notices and shared folder. They differ from each other in the single languages. It is also save to check if the standard folders are the same on both sides (if not, that can be tuned with (multiple are possible) --regextrans2 and a regular expression that fixes that).
 
Removing the '''--dry''' option will actually migrate the user.
 
Then, the sync can run multiple times e. g., a comprehensive job can run to migrate the majority of the data and once again just before the user changes to the new server.
 
== Bulk import of multiple users ==
 
=== Sample for importing OX users out of the OX database ===
 
This script basically lists all user from a Open-Xchange context and uses the user id as username for the imap servers on both sides to export and to import all.
 
#!/bin/sh
# Target OX context id
CID=1
# cyrus settings
CYRHOST=localhost
CYRUSADM=cadm
CYRPWD=cpass
# exchange settings
MSHOST=10.20.30.170
MSDOM=domain
MSADM=madm
MSPASS=secret
SYSFOLD="&ANY-ffentliche Ordner|&ANY-ffentliche Ordner/Internet Newsgroups|Aufgaben|Journal|Kalender|Kontakte|Notizen|Calendar|Contacts|Notes|Tasks"
export PATH=$PATH:/opt/open-xchange/sbin/
for user in $(listuser -c $CID | grep -v ^Id | awk '{ print $2 }'); do
        imapsync --host1 "$MSHOST" \
                --user1 "${MSDOM}/${MSADM}/${user}" \
                --password1 "$MSPASS" \
                --host2 "$CYRHOST" \
                --user2 "$user" \
                --authuser2 "$CYRUSADM" \
                --password2 "$CYRPWD" \
                --authmech2 PLAIN \
                --prefix1 "INBOX." \
                --regextrans2 's/INBOX.INBOX/INBOX/' \
                --noauthmd5 \
                --subscribe \
                --exclude \""$SYSFOLD"\" \
                --dry
done
 
remove '''--dry''' so the script actually takes action.
 
=== Sample for importing LDAP users out of Active Directory ===
 
This script queries a Active Directory server for all "sAMAccountName" attributes in the users tree to use it as IMAP usernames for export and import them all.
 
#!:/bin/sh
# cyrus settings
CYRHOST=localhost
CYRUSADM=test1111
CYRPWD=S3cr3t3
# exchange settings
MSHOST=10.20.30.170
MSDOM=perf
MSADM=migration
MSPASS=secret
SYSFOLD="&ANY-ffentliche Ordner|&ANY-ffentliche Ordner/Internet Newsgroups|Aufgaben|Journal|Kalender|Kontakte|Notizen|Calendar|Contacts|Notes|Tasks"
MSDN="CN=Administrator,CN=users,dc=perf,dc=open-xchange,dc=local"
MSDNPW="netline"
MSIGNORE="^#|SUPPORT|WIN2003|krbtgt|Administrator|Gast"
export PATH=$PATH:/opt/open-xchange/sbin/
for user in $(ldapsearch -x -D${MSDN} -w "$MSDNPW" '(objectclass=user)' sAMAccountName | grep sAMAccountName | grep -v -E "($MSIGNORE)" | awk '{ print $2 }'); do
        imapsync --host1 "$MSHOST" \
                --user1 "${MSDOM}/${MSADM}/${user}" \
                --password1 "$MSPASS" \
                --host2 "$CYRHOST" \
                --user2 "$user" \
                --authuser2 "$CYRUSADM" \
                --password2 "$CYRPWD" \
                --authmech2 PLAIN \
                --prefix1 "INBOX." \
                --regextrans2 's/INBOX.INBOX/INBOX/' \
                --noauthmd5 \
                --subscribe \
                --exclude \""$SYSFOLD"\" \
                --dry
done
 
remove '''--dry''' so the script actually takes action.
 
== Known issues ==


Removing the '''--try''' option will make the user really migrated.
=== Synced mail folders are not seen on Open-Xchange ===


The sync then can run multiple times so for example a big job can run to migrate majority of the data and and once again just before the go live for later changes.
#Verify if the missing mail folders show up in the output of imapsync and if that is not the case tune the '''--exclude''' option
#Verify if the missing folders are there but not subscibed ( right click on '''INBOX''' -> '''subscribe folders''' in the Open-Xchange web gui). It is possible to set '''com.openexchange.mail.ignoreSubscription=true''' in '''"/opt/open-xchange/etc/groupware/mail.properties"''' to let all folders show up regardless of their subscription status.

Latest revision as of 07:23, 1 September 2009

Introduction

The document describes a way how mailboxes can be migrated from MS Exchange™ to a cyrus IMAP server using imapsync without the need to know the user passwords.

Requirements

  1. A cyrus IMAP server and an admin account
    the admin accounts for cyrus are defined in the admins: parameter of the /etc/imapd.conf configuration file. In the examples, the admin user name is cyrus with password cpass.
  2. Microsoft Exchange 2003™
    other Microsoft Exchange™ versions might work, too. This howto was written for the 2003 version.
  3. IMAP connector installed and activated for Microsoft Exchange™
  4. Postmaster account for MS Exchange™
    The Exchange™ account for migration needs to have the permission to access all mailboxes on the MS Exchange™ server. In the examples the user name is migration with password mpass. The user can be created following these steps:
    1. Create a new DWORD in HKEY_Current_User\Software\ Microsoft\Exchange\ExAdmin with the name ShowSecurityPage and set this to 1.
    2. Open the Active Directory User and Computer console and create a new user with the name migration, password mpass and MS Exchange™ attributes.
    3. Reopen the MS Exchange™ System Manager and give that new user full access to the entire mail organization (right click on the uppermost entry naming your organization -> properties -> security -> add)

Testing the admin users

Those example IMAP logins describe a telnet session where a succesful IMAP connection is created and closed (cyrus is running on localhost and the Exchange server on 10.20.30.170)

Test if the cyrus admin can access the server

[]# telnet localhost imap
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape character is '^]'.
* OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS AUTH=LOGIN AUTH=PLAIN SASL-IR] servera Cyrus IMAP4 v2.3.7-Invoca-RPM-OX-2.3.7-3.OX server ready
. login cyrus cpass
. OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID LOGINDISABLED ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT SORT=MODSEQ THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE CATENATE CONDSTORE IDLE LISTEXT LIST-SUBSCRIBED X-NETSCAPE URLAUTH] User logged in
. logout
* BYE LOGOUT received
. OK Completed
Connection closed by foreign host.
[]#

Test if the migration user can log in to own mailbox

 []# telnet 10.20.30.170 imap
Trying 10.20.30.170...
Connected to mastercane.open-xchange.com (10.20.30.170).
Escape character is '^]'.
* OK Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0 (win2003.perf.open-xchange.local), steht zur Verf�gung.
. login migration mpass
. OK LOGIN completed.
. logout
* BYE Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0, meldet sich ab.
. OK LOGOUT completed.
Connection closed by foreign host.
[]#

Test if the migration user can log in for user test in domain

This tests shows if it is possible to access the mail box "test" with the credentials of the migration user: 

[]# telnet 10.20.30.170 imap
Trying 10.20.30.170...
Connected to mastercane.open-xchange.com (10.20.30.170).
Escape character is '^]'.
* OK Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0 (win2003.perf.open-xchange.local), steht zur Verf�gung.
. login domain/migration/test mpass
. OK LOGIN completed.
. logout
* BYE Der Microsoft Exchange Server 2003 IMAP4rev1-Server, Version 6.5.6944.0, meldet sich ab.
. OK LOGOUT completed.
Connection closed by foreign host.
[]#

Migrating a single user

Access to the MS Exchange™ mailbox is done via <domain>/migration/<user to migrate> and the password of the migration account. In cyrus the Data is stored as user with the credentials of the admin account. The imapsync command is as follows: 

imapsync --host1 <ms exchange> --user1 <domain>/migration/<user to migrtate1> --password1 \
         mpass --host2 <cyrus server> --user2 <user to migrate2> --authuser2 cyrus \
         --password2 cpass --authmech2 PLAIN --prefix1 "INBOX." --regextrans2 \
         's/INBOX.INBOX/INBOX/'  --noauthmd5 --subscribe \
         --exclude "&ANY-ffentliche Ordner|Aufgaben|Journal|Kalender|Kontakte|Notizen" --dry
variable value
<ms exchange> hostname or ip of the Exchange server
<domain> Active directory domain name
<user to migrtate1> Exchange user to be migrated
<cyrus server> hostname or ip of the cyrus imap server
user to migrate2 target user name in the cyrus imap server
--exclude a list of system folders not to migrate

The above command does actually do a try run without migrating any data. This is because of the --dry option. It is needed to see what special folders are available that do not need to be migrated. These are the folders calendar,contacts,tasks,notices and shared folder. They differ from each other in the single languages. It is also save to check if the standard folders are the same on both sides (if not, that can be tuned with (multiple are possible) --regextrans2 and a regular expression that fixes that).

Removing the --dry option will actually migrate the user.

Then, the sync can run multiple times e. g., a comprehensive job can run to migrate the majority of the data and once again just before the user changes to the new server.

Bulk import of multiple users

Sample for importing OX users out of the OX database

This script basically lists all user from a Open-Xchange context and uses the user id as username for the imap servers on both sides to export and to import all. 

#!/bin/sh

# Target OX context id
CID=1

# cyrus settings
CYRHOST=localhost
CYRUSADM=cadm
CYRPWD=cpass

# exchange settings
MSHOST=10.20.30.170
MSDOM=domain
MSADM=madm
MSPASS=secret
SYSFOLD="&ANY-ffentliche Ordner|&ANY-ffentliche Ordner/Internet Newsgroups|Aufgaben|Journal|Kalender|Kontakte|Notizen|Calendar|Contacts|Notes|Tasks"


export PATH=$PATH:/opt/open-xchange/sbin/


for user in $(listuser -c $CID | grep -v ^Id | awk '{ print $2 }'); do

        imapsync --host1 "$MSHOST" \
                --user1 "${MSDOM}/${MSADM}/${user}" \
                --password1 "$MSPASS" \
                --host2 "$CYRHOST" \
                --user2 "$user" \
                --authuser2 "$CYRUSADM" \
                --password2 "$CYRPWD" \
                --authmech2 PLAIN \
                --prefix1 "INBOX." \
                --regextrans2 's/INBOX.INBOX/INBOX/' \
                --noauthmd5 \
                --subscribe \
                --exclude \""$SYSFOLD"\" \
                --dry

done

remove --dry so the script actually takes action.

Sample for importing LDAP users out of Active Directory

This script queries a Active Directory server for all "sAMAccountName" attributes in the users tree to use it as IMAP usernames for export and import them all. 

#!:/bin/sh

# cyrus settings
CYRHOST=localhost
CYRUSADM=test1111
CYRPWD=S3cr3t3

# exchange settings
MSHOST=10.20.30.170
MSDOM=perf
MSADM=migration
MSPASS=secret
SYSFOLD="&ANY-ffentliche Ordner|&ANY-ffentliche Ordner/Internet Newsgroups|Aufgaben|Journal|Kalender|Kontakte|Notizen|Calendar|Contacts|Notes|Tasks"
MSDN="CN=Administrator,CN=users,dc=perf,dc=open-xchange,dc=local"
MSDNPW="netline"
MSIGNORE="^#|SUPPORT|WIN2003|krbtgt|Administrator|Gast"

export PATH=$PATH:/opt/open-xchange/sbin/

for user in $(ldapsearch -x -D${MSDN} -w "$MSDNPW" '(objectclass=user)' sAMAccountName | grep sAMAccountName | grep -v -E "($MSIGNORE)" | awk '{ print $2 }'); do

       imapsync --host1 "$MSHOST" \
               --user1 "${MSDOM}/${MSADM}/${user}" \
               --password1 "$MSPASS" \
               --host2 "$CYRHOST" \
               --user2 "$user" \
               --authuser2 "$CYRUSADM" \
               --password2 "$CYRPWD" \
               --authmech2 PLAIN \
               --prefix1 "INBOX." \
               --regextrans2 's/INBOX.INBOX/INBOX/' \
               --noauthmd5 \
               --subscribe \
               --exclude \""$SYSFOLD"\" \
               --dry

done

remove --dry so the script actually takes action.

Known issues

Synced mail folders are not seen on Open-Xchange

  1. Verify if the missing mail folders show up in the output of imapsync and if that is not the case tune the --exclude option
  2. Verify if the missing folders are there but not subscibed ( right click on INBOX -> subscribe folders in the Open-Xchange web gui). It is possible to set com.openexchange.mail.ignoreSubscription=true in "/opt/open-xchange/etc/groupware/mail.properties" to let all folders show up regardless of their subscription status.
Retrieved from "https://wiki.open-xchange.com/wiki/index.php?title=Exchangetoox&oldid=4713"