AppSuite:CustomDBMigration

This information is valid from 7.6.1 on.

How to write custom bundles to execute database migration statements.

Summary: With release 7.6.1 it is possible to change the database schema based on the open source tool Liquibase. Currently only updating the configdb is supported. This article gives a short introduction (based on an existing sample bundle) how to write custom database migration bundles and how to attach your custom statements to those statements given by Open-Xchange.

Please have a look at Liquibase, its features and documentation before writing custom bundles: http://www.liquibase.org/documentation/ .

Additional information about Liquibase within the Open-Xchange server (restrictions in usage, developer hints, available tools, ...) are available on http://oxpedia.org/wiki/index.php?title=AppSuite:DBMigration

Prerequisite

This article is based on an existing sample bundle located in the public git repository backend-samples. Clone the repository by executing

git clone https://git.open-xchange.com/git/examples/backend-samples

The required bundle is named 'com.openexchange.sample.database.migration'.

Bundle dependencies

To execute custom database migration for the configdb you have to track the service com.openexchange.database.migration.DBMigrationExecutorService (default provided within bundle com.openexchange.database.migration).

Using database migration

First of all you have to create the ChangeLog XML file that contains the ChangeSets. The sample bundle shows two different approaches: A declarative one via liquibase migration directives and a programmatic one that executes plain SQL. Programmatic changes must also be referenced in the ChangeLog file.

Declarative changes

Liquibase executes statements based on declarative descriptions. Creating a table for instance will look like

<changeSet id="7.6.1:com.openexchange.sample.database.migration:sampleTable" author="steffen.templin@open-xchange.com">
  <preConditions onFail="MARK_RAN">
    <not>
      <tableExists tableName="sample" />
    </not>
  </preConditions>
  <comment>Creates the 'sample' table</comment>
  <createTable tableName="sample">
    <column name="id" type="int" autoIncrement="true">
      <constraints primaryKey="true" nullable="false" />
    </column>
    <column name="value" type="varchar(255)" />
  </createTable>
</changeSet>

A list of available changes (e. g. addAutoIncrement, addColumn, addForeignKeyConstraint, createIndex, createProcedure, dropUniqueConstraint, insert, update and many more) can be found at http://www.liquibase.org/documentation/changes/index.html

SQL changes

It is possible to reference Java classes from within the ChangeLog file (see below). The referenced Java class has to implement CustomSqlChange as shown in the bundle.

<changeSet id="7.6.1:com.openexchange.sample.database.migration:sampleTable2" author="martin.schneider@open-xchange.com">
  <preConditions onFail="MARK_RAN">
    <not>
      <tableExists tableName="sample_2" />
    </not>
  </preConditions>
  <comment>
    This ChangeSet executes custom sql based on the implementation of CustomSqlChange. The execution depends on the precondions result.
  </comment>
  <customChange class="com.openexchange.sample.database.migration.custom.ExampleCustomSqlChange" />
</changeSet>

Running the migrations

After you retrieved the DBMigrationExecutorService from the OSGi service registry, you can schedule your migration ChangeLog via the scheduleConfigDBMigration(String fileLocation, ResourceAccessor accessor) method.

The first argument is the path to your ChangeLog file, starting at your bundles root (i.e. the highest level within your JAR file). If your structure looks like below, this would be /liquibase/configdbChangeLog.xml:

com.example.ox.extension.jar
|-com
| |-example
| | |-ox
| | | |-extension
| | | | |-Activator.class
|-META-INF
| |-MANIFEST.MF
|-liquibase
| |-configdbChangeLog.xml

The second argument must be of type liquibase.resource.ResourceAccessor and must be able to load the ChangeLog file and all referenced resources therein. You most likely want to use an instance of com.openexchange.database.migration.resource.accessor.BundleResourceAccessor here, initialized with your custom bundles context.

Results

After using Liquibase the first time you will have two additional tables for managing its state. Furthermore the following output shows the two customtables:

<code>
mysql> show tables;
+------------------------+
| Tables_in_configdb     |
+------------------------+
| DATABASECHANGELOG      |
| DATABASECHANGELOGLOCK  |
| customtable1           |
| customtable2           |
+------------------------+
</code>

customtable1

The createTable statement from above created the following table:

<code>
mysql> show create table customtable1;
+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table        | Create Table                                                                                                                                                                    |
+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| customtable1 | CREATE TABLE `customtable1` (
  `id` int(11) NOT NULL,
  `name` varchar(50) NOT NULL,
  `active` bit(1) DEFAULT b'1',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0,00 sec)
</code>

customtable2

The CustomSqlChange implementation referenced created the following table:

<code>
mysql> show create table customtable2;
+--------------+----------------------------------------------------------------------------------------------------------+
| Table        | Create Table                                                                                             |
+--------------+----------------------------------------------------------------------------------------------------------+
| customtable2 | CREATE TABLE `customtable2` (
  `customColumn` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
+--------------+----------------------------------------------------------------------------------------------------------+
1 row in set (0,00 sec)
</code>

Important hints

• Execute the statements by using the ResourceAccessor 'ClassLoaderResourceAccessor' as shown in the bundle. Call com.openexchange.database.migration.DBMigrationExecutorService.execute(String, List<ResourceAccessor>) and provide the ResourceAccessor. If you do not use this accessor custom Java classes won't be found.
• A good solution to distinguish between ChangeSets that have to be executed is to use Liquibase 'Contexts'. For further information have a look at the Liquibase documentation: http://www.liquibase.org/documentation/contexts.html