OX Outlook Uploader

Open-Xchange Microsoft Outlook® Uploader (Beta)

Later this year, Open-Xchange will release Open-Xchange Microsoft Outlook® Uploader (short: OXUploader), a migration tool to export data from Microsoft Outlook® or from a Microsoft Exchange Server® to the Open-Xchange Server.

Features

• Migration of E-Mails, appointments, contacts, distribution lists, tasks and sticky notes
• Folder structure is preserved, subfolders are created as necessary
• Attachments of E-Mails and other objects are migrated
• Support for recurring appointments and participants
• Migration of .pst-files
• Migration of data files from installed mail profiles
• Migration of Microsoft Exchange mailboxes
• Easy to use 3-step interactive wizard to configure the migration
• Unattended mode to perform migrations without user interaction
• Commandline interface to automate batch migrations

Prerequisites

The OXUploader migration tool is installed on a windows system having access to the data that should be migrated. The data is then migrated from there to the Open-Xchange server. Please ensure that the following requirements are met before installing the tool:

Windows System

• Windows XP and newer or Windows Server 2003 and newer
• Microsoft® Office® Outlook (2002/XP, 2003, 2007 or 2010) or Microsoft® Exchange Server or standalone versions of Extended MAPI (Exchange 5.5 through 2010)
• Microsoft® .NET Framework 2.0

Open-Xchange Server

• Open-Xchange server, version 6.16.1 or later
• The user account(s) for which the data should be migrated should already have been created

Download and installation

Please note that everything from this page and beyond is in Beta: That means that nothing is final, changes will be made before final release, stability is not guaranteed and nothing from this site should be used in production.

To download the Open-Xchange Microsoft Outlook® Uploader (Beta) installation packages and Release Notes, follow this link

The access-data are:

• User name: oxuploaderbeta
• Password: first$cut

After the download of the package, you can start the installation by double-clicking on the package. Please follow the installation steps in the wizard. The installation does not require administrative rights.

Usage (interactive mode)

	1. Choose the migration source First, either choose to migrate a personal storage file (.pst file), a data store from the local mail profiles, or an Exchange server mailbox. Depending on your choice, you can specify the migration source in the next step.
	2a. Select a personal storage file to migrate When clicking on Browse the standard folder for saving .pst files opens. As some .pst files are password protected the password can be entered in the second field. When having chosen a file, click on Next.
	2b. Select a data file from a profile to migrate Select one of the existing profiles from the drop-down menu. Select a store from the profile. Finally, enter a password if needed. To proceed, click Next.
	2c. Select an Exchange Server mailbox to migrate Enter the URL of the Microsoft Exchange server. Select the username of the the account on the Exchange server. Click on "Logon..." to connect to the Exchange server and fetch the available mailboxes. When being asked, enter your password to logon to the Exchange server Select one of the existing mailboxes from the drop-down menu. To proceed, click Next.
	3. Please enter information about the server In the Server URL field, enter the Open-Xchange Server address. Enter your username in the second field. Enter your password in the third field. In order to make further settings, activate the "Configure advanced settings" checkbox. To proceed, click on Next.
	4. Configure advanced options If the folders should be migrated into a subfolder, enter it's name in the first field. In case not all mail folders are to be exported, the folders can be selected by clicking on the "Select..." button. Select the type of items to be migrated by marking the apporpiate checkboxes, and optionally define a minimum date for the items. In order to ignore the "Trash", "Junk" or "Sync Issues" folders for the migration, activate the respective checkbox. If empty folders should be skipped, activate the respective checkbox. If the target folders should be emptied before items are migrated into them, activate the respective checkbox.
	5. Migration in process This window displays the status and a log protocol of the migration. When the process is finished, click on Next.
	6. Migration completed This window displays an overview of the migration and you can view the log file by clickin on the respective button. To close the wizard, click on Finish.

Usage (unattended mode)

The migration tool can be started in a special "batch mode", where no user interaction is necessary. The options for the migration can be set in the configuration files, or passed via the commandline when starting the tool, see (3). When the parameter "batchmode" is "true", the migration starts automatically. In this case, at least the following settings have to be available, either supplied via the commandline or defined in the configuration file:

• Username
• ServerURL
• PSTFile or ProfileName and StoreName or ExchangeServerURL
• Password or AdminUser and AdminPassword

For example, to migrate the .pst-file "c:\test.pst", a command could look like:

OXUploaderC.exe -batchmode true -pstfile "c:\test.pst" -serverurl "http://ox.example.invalid" -username test -password secret

Multiple instances of the tool can be run simultaneously from one or multiple client machines, more information can be found in (6).

Configuration

Configuration files

The application's configuration settings can be adjusted in config files, both for the OXUploader.exe main application for interactive mode and for the commandline version of the tool OXUploaderC.exe (see below). The files are named as the corresponding executable, with the file extension ".config": OXUploader.exe.config and OXUploaderC.exe.config respectively. A description of each parameter can be found in Configuration parameters

Commandline parameters

Furthermore, the application can be started with commandline parameters. All supplied options overwrite the default settings from the configuration file, which are assumed if an option is missing. The following parameters are recognized:

-adminuser <Admin username on the OX server (for admin migration)>
-adminpassword <Admin password on the OX server (for admin migration)>
-appointmentsmindate <Minimum date of appointments to be migrated>
-batchmode <true/false>
-clearfolders <true/false>
-emailssmindate <Minimum date of e-mails to be migrated>
-exchangemailboxname <Name of the Exchange mailbox to migrate>
-exchangeserverurl <URL of the Exchange server>
-exchangeserverusername <Username on the Exchange server>
-exportappointmentparticipants <true/false>
-exportappointmentuids <true/false>
-ignorejunkfolder <true/false>
-ignoresyncissuesfolder <true/false>
-ignoretrashfolder <true/false>
-importfoldername <Name of the migration target subfolder>
-logautoflush <true/false>
-logconsole <true/false>
-loglevel <off/error/warning/info/verbose>
-logonexchangemailbox <true/false>
-logpostdata <true/false>
-logtextfile <Path to logfile>
-migrateappointments <true/false>
-migratecontacts <true/false>
-migrateemails <true/false>
-migratenotes <true/false>
-migratetasks <true/false>
-password <Password on the OX server>
-profilename <Name of the profile>
-pstfile <Path to PST file>
-pstpassword <Password for the PST file>
-serverurl <URL of the OX server>
-skipemptyfolders <true/false>
-storename <Name of the store inside the profile>
-tasksmindate <Minimum date of tasks to be migrated>
-uploadchunksize <Number of uploaded mails per request>
-uploadthresholdbytes <Threshold of bytes for sending forced>
-username <Username on the OX server>
-recoverableexceptionmaxretries <max. retries after recoverable error>

Misc

The following chapters contain some advanced information and configration options for the OXUploader migration tool.

Performance considerations

The time needed to perform a migration correspondends with the size of the .pst file, of course. So, for large mailboxes, this could be a quite time intensive task, and may be improved in knowledge of the following observations, especially when performing multiple simultaneous batch migrations. The I/O-ratio is about the following: Per each byte read from disk, about 0.65 bytes are written to disk, and 0.12 bytes are sent to the server. This is caused by the fact that the messages read out from the .pst-file are exported to RFC822 .eml files first, then those files are loaded again before finally sending them to the server. Therefore, the throughput of the local file system should be as fast as possible. Since the temporary created RFC822 e-mail files are created in the user's temp folder, it would be best to move the location to a separate location as the .pst-file - another hdd, maybe even a ramdisk for the temp directory would make sense for heavy batch migrations. It's also recommended to temporarily disable virus protection and indexing services during the migration, and Outlook 2007 should be preferred against Outlook 2003.

Limitations when running multiple instances

When executing multiple migrations simulataneously from one machine, each instance of the tool must have exclusive access to the .pst-file it is migrating. Furthermore, each instance must use a different logfile. The tool relies on Outlook, and so it also can't work around Outlooks own limitations, especially as Outlook is designed as client application. More details can be found at the knowledge base article here: http://support.microsoft.com/kb/257757/en-us .

Admin migration

When the server is configured appropiate, the migration tool can also be used to import .pst-files into the OX account of the specified user, without the need to supply the user's password. Therefore, the username and password of a special admin user is needed for authentication against the OX server. For example, to migrate the .pst-file "c:\test.pst" into the account "test", a command could look like:

OXUploaderC.exe -batchmode true -pstfile "c:\test.pst" -serverurl "http://ox.example.invalid" -username test -adminuser admin -adminpassword secret

Personal storage files

Sometimes .pst-files are broken our corrupted, and the migration tool may not be able to open them correctly, or opening them takes a very long time when the internal repair operations take place. To reduce the impact of broken .pst-files, it's recommended to check the integrity of the files using Microsofts SCANPST.exe tool, see http://support.microsoft.com/kb/287497 . Sadly this utility is designed to run interactively, but there are some utilities out there that automate the SCANPST.exe tool to be run from the commandline, pointing to the .pst-file to scan, for example cmdscan.exe from http://www.olfolders.de/Lang/English/OLfix/download.htm . Doing so, one would be able to incorporate a SCANPST.exe launch prior a batch call to the migration tool, e.g. the following command could be invoked to scan and repair the .pst file "c:\test.pst", passing the path to the SCANPST.exe location:

cmdscan.exe -rename "c:\Program Files (x86)\Microsoft Office\Office12\SCANPST.EXE" "c:\test.pst"

Duplicated items

Migrating the same items into the same target folder multiple times usually results in the items getting duplicated on the server. An exception to this rule are appointments with unique identifiers (UIDs), see http://www.ietf.org/rfc/rfc2445.txt, chapter 4.8.4.7, for details. To avoid importing an appointment with the same identifier multiple times, the server rejects an appointment when the UID already exists on the server. To force the migration of appointments regardless of their UIDs, the tool optionally removes the UIDs from the appointment. The relevant setting is named "ExportAppointmentUIDs", when set to "False", existing UIDs are removed before sending them to the server. Furthermore, the migration tool has an option to clear the folder contents before triggering the export. When "ClearFolders" is set to "True", the target folder is emptied prior to the items being exported. Note that any existing data in these folders will be permanently lost.

Exchange mailboxes

Instead of using a local mail profile or a .pst file for migration, it's also possible to logon to and migrate an Exchange mailbox directly. It's recommeded that the migration tool is executed inside the exchange domain by the owner of the mailbox. To perform an administrative migration for other user's mailboxes, the following preconditions must be met:

• Ensure the target account already has been created on the OX server
• Grant the administrating user "Receive As" permissions to the mailbox store of all accounts that should be migrated, details can be found at http://support.microsoft.com/default.aspx?scid=kb;EN-US;821897 and http://msexchangeteam.com/archive/2006/01/25/418099.aspx
• The migration tool is ececuted by the administrative user, who already should be logged on to the exchange domain

The administrative migration can only be executed from the commandline in "batch mode", see (5) for details. The mailbox or the name of the user that is going to be migrated needs to be specified inside the commandline parameter "exchangemailboxname". For example, to migrate the mailbox of exchange user "test" to the Open-Xchange account with username "test", combined with admin migration at the OX server (see (6c) for details), one could execute the migration tool in the following way:

OXUploaderC.exe -batchmode true -exchangeserverurl 192.168.0.4 -exchangemailboxname test -serverurl "http://ox.example.invalid" -username test -adminuser admin -adminpassword secret

Or, if the password of the OX user is available:

OXUploaderC.exe -batchmode true -exchangeserverurl 192.168.0.4 -exchangemailboxname test -serverurl "http://ox.example.invalid" -username test -password secret

Administrative installation

The OXUploader migration tool itself can be installed without the need for elevated access rights by the end user, as the installer only accesses the local non-roaming application data directory and the HKCU hive in the registry. It can't be installed 'per machine', which means that it is not allowed to set the value of the ALLUSERS msi property to "1". The tool can be pre-configured for the end-users by setting public properties at the commandline for the installation process. These are processed by the installer and then written into the tool's config-file. The following parameters can be adjusted (the values in square brackets that are listed below indicate the default values of the properties when not overridden by the msi commandline - so that would be the default pre-configuration):

ADMINUSER []
ADMINPASSWORD []
BATCHMODE [False]
CLEARFOLDERS [False]
EXPORTAPPOINTMENTPARTICIPANTS [True]
EXPORTAPPOINTMENTUIDS [True]
IGNOREJUNKFOLDER [True]
IGNORETRASHFOLDER [True]
IMPORTFOLDERNAME []
LOGAUTOFLUSH [True]
LOGCONSOLE [False]
LOGLEVEL [Info]
LOGTEXTFILE [.\pst2ox.log]
MIGRATEAPPOINTMENTS [True]
MIGRATECONTACTS [True]
MIGRATEEMAILS [True]
MIGRATENOTES [True]
MIGRATETASK [True]
PASSWORD []
PROFILENAME []
PSTFILE []
PSTPASSWORD []
SERVERURL [http://]
SKIPEMPTYFOLDERS [True]
STORENAME []
UPLOADCHUNKSIZE [25]
UPLOADTHRESHOLDBYTES [2097152]
OXUSERNAME []
RECOVERABLEEXCEPTIONMAXRETRIES [3]

For example, to predefine a Server URL that should be used for the migration afterwards, one would execute e.g.:

 "Open-Xchange Outlook Uploader.msi" SERVERURL="http://ox.example.invalid"

Address Resolution

While the Open-Xchange server uses SMTP as the only recipient address type, data from Microsoft Exchange might contain several additional address types. To migrate such Exchange data to the Open-Xchange server, each e-mail recipient or particpiant of an appointment or task needs a valid SMTP address. While the migration tool tries to retrieve the SMTP address of message recipients in different ways, sometimes there might be recipients that can't be resolved without a connection to the Microsoft Exchange server / Active Directory and it's global address lists. Especially, this may happen with .pst-files that were created using Microsofts EXMERGE tool. In order to resolve that issue, the OXUploader migration tool can establish a connection to the exchange server and to it's global address list prior migrating a .pst-file to the Open-Xchange server, so that the recipient's SMTP addresses can be resolved. To do so, there are two parameters that can be set to logon to the exchange server prior migrating the .pst-file: ExchangeServerURL and ExchangeServerUsername. The username parameter is optional (if not defined, the identity of the current windows user is used) and normally should not need to be set in batch-mode to prevent a windows logon prompt. For example, to logon to the exchange server at "192.168.0.4" and perform an admin migration for the username "test" in unattended mode, a command could look like:

OXUploaderC.exe -batchmode true -exchangeserverurl 192.168.0.4 -serverurl "http://ox.example.invalid" -username test -adminuser admin -adminpassword secret -pstfile ".\test.pst" -logtextfile ".\test.log"

Configuration parameters

This chapter describes all possible configuration parameters that can be found in the OXUploader.exe.config / OXUploaderC.exe.config configuration files or can be passed to the migration tool from the commandline.

AdminPassword

The password associated with the administrator user (AdminUser) on the Open-Xchange server. Only used when performing admin migration.

AdminUser

The username of the administrator user on the Open-Xchange server. Only used when performing admin migration.

AppointmentsMinDate

If defined, only appointments that end after this date are migrated. Recurring appointments are migrated, if the end date of the series has not been set, or if the end date of the series is not after this defined value. Date format should be yyy-MM-dd.

BatchMode

Specifies whether the tool should be run in a special batchmode without user interaction or not, see unattended mode for details.

ClearFolders

When set to 'True', the migration tool clears the target folder contents before exporting items to that folders. Existing subfolders are not affected by this setting, only the folder's contents. Also, the folder clear-command is only executed for folders that are actually migrated, not for ignored or skipped folders.

EMailsMinDate

If defined, only e-mails that were sent or received after this date are migrated. Date format should be yyyy-MM-dd.

ExchangeMailboxName

Specifies the mailbox name on the Microsoft Exchange server that should be used for migration. Used in combination with the ExchangeServerURL and ExchangeServerUsername parameters. If not defined, the default mailbox for the Exchange user is used. This parameter is also used to define a shared Exchange mailbox, see Exchange mailboxes for details.

ExchangeServerURL

The URL of the Microsoft Exchange server that hosts the mailbox to be migrated. Only used, when PSTFile and ProfileName are not defined.

ExchangeServerUsername

The username of the Exchange account to migrate. Used in combination with the ExchangeServerURL and ExchangeMailboxName parameters. If not defined, the current windows identity is used.

ExportAppointmentParticipants

Specifies whether the participants (recipients) of appointments should be migrated or not. If set to 'False', only the Open-Xchange user (Username) will participate the exported appointments, otherwise all original participants will be added to the migrated appointment, too.

ExportAppointmentUIDs

Specifies whether the unique identifiers (UIDs) of appointments should be migrated or not. Details can be found in the chapter Duplicated items.

HistoryFile

The path to the file where the migration tool stores the migration results internally to keep track of which folders already have been migrated successfully.

IgnoreJunkFolder

Whether to ignore the Junk folder during the migration or not.

IgnoreSyncIssuesFolder

Whether to ignore the Sync Issues folder during the migration or not.

IgnoreTrashFolder

Whether to ignore the Trash folder during the migration or not.

ImportFolderName

Specifies the name of an additional subfolder below the OX default folders that is used as target for the migrated folders. If left blank, the OX default folders are used as target directly.

LogAutoFlush

Specifies whether the log should be flushed to disk automatically after each entry.

LogConsole

Whether to output the log to the console or not.

LogLevel

Specifies the verbosity of the generated log, possible values are Error, Warning, Info and Verbose.

LogonExchangeMailbox

It's possible to establish a connection to the an MS Exchange server and to it's global address list prior migrating .pst-files, so that the recipient's SMTP addresses can be resolved. When set to True, the URL of the Exchange server has to be defined in ExchangeServerURL.

LogPOSTData

For debugging reasons, this option enables or disables logging of the complete POST data that is sent to the Open-Xchange server during migration.

LogTextFile

Specifies the path to the logfile.

MigrateAppointments

Whether to migrate appointments or not.

MigrateContacts

Whether to migrate contacts and distribution lists or not.

MigrateEMails

Whether to migrate e-mail or not.

MigrateNotes

Whether to migrate sticky notes or not.

MigrateTasks

Whether to migrate tasks or not.

Password

The password for the target account (UserName) on the Open-Xchange server.

ProfileName

The name of the local mail profile to be migrated. Used in combination with the StoreName parameter.

PSTFile

The path to the .pst-file to be migrated.

PSTPassword

The password to access the .pst-file, only needed if the .pst has been protected with a password.

RecoverableExceptionMaxRetries

Specifies how often a request is repeated in case of possibly recoverable errors.

ServerURL

The URL of the Open-Xchange server.

SkipEmptyFolders

Whether to include folders that don't contain any items or not. Subfolders in empty folders are not affected by this setting.

StoreName

The name of the message store inside the local mail profile. Used in combination with the ProfileName parameter.

TasksMinDate

If defined, only tasks with a due date after this date are migrated. Date format should be yyyy-MM-dd.

UploadChunkSize

Specifies the number of e-mail items that are sent to the Open-Xchange server during the migration per each request.

UploadThresholdBytes

Specifies a value in bytes after which an e-mail message chunk is sent to the server, even if when the UploadChunkSize is not yet exceeded.

Username

The user name of the target account on the Open-Xchange server.

Functional limitations

• Due to differences in handling recurring tasks between the Open-Xchange server and Microdoft Exchange, recurring tasks are not supported.
• Folder permissions are not transferred to the Open-Xchange server; each new subfolder that is created during the migration gets a set of default permissions (target user has admin permissions).

Reporting of Bugs

Please report bugs and missing features via Open-Xchange Bugzilla. Many thanks in advance for your support.

Product: Open-Xchange MS Outlook Uploader

Open-Xchange Microsoft Outlook® Uploader (Open Source)

Portions of the Software may use, include third party software, other copyrighted material or Open Source Software. Acknowledgements, licensing terms and disclaimers for such material are contained in separate agreements. Licensee’s use of such material is governed by the terms of the applicable agreements and can be found on the OX web site, are listed under: http://www.open-xchange.com/node/1109

Compile from sources

Download and install Outlook Redemption

The Open-Xchange Microsoft Outlook Uploader requires the Outlook Redemption library® by Dmitry Streblechenko. As the distributable version of Redemption cannot be used in open source projects, it is not included in the repository. However, for development purposes only, you can download a copy of the library at http://www.dimastr.com/redemption/ and install it on your development PC.

Checkout the sources from Subversion

The source code for the Open-Xchange Microsoft Outlook® Uploader is available in a public Subversion repository. To checkout the latest sources, use the following command:

svn checkout https://svn.open-xchange.com/migration/OXUploader

Build using Visual Studio®

The Open-Xchange Microsoft Outlook® Uploader can be compiled easily using Visual Studio 2008®. For both the interactive and the commandline version of the migration tool, there are C# project files (.csproj) in the PST2OX and the PST2OXc subdirectories (PST2OX is the 'codename' of the migration tool). Just open the project file in Visual Studio and build the project.