Restrictions on importing data: Difference between revisions

From Open-Xchange
No edit summary
No edit summary
 
Line 1: Line 1:
{{UnderConstruction|Needs review by current developer}}
There are [[:Category:Import|several ways to import data into Open-Xchange]], but all are restricted by the fact that OX uses a SQL database to store data about its core elements (contacts, appointments and tasks). That means that the field size is limited.
There are [[:Category:Import|several ways to import data into Open-Xchange]], but all are restricted by the fact that OX uses a SQL database to store data about its core elements (contacts, appointments and tasks). That means that the field size is limited.


Note: The fields have been sized generously while still having efficient row sizes for fast SQL queries. If you encounter a problem, it usually lies with generic fields like "description" or "comment", which some other programs use to store vast amounts of data (for example a VCard representation of the same contact you just imported).
If there is such a problem, the data will be shortened/truncated to field the field. The technical details vary a little:
* If your database is set to automatically truncate data, the content will be cut off at that part.
* If your database is not to for automatic truncation, the server will try to do this. But the database only reports one oversized field at a time, so the server might need several attempts to trim the whole record in case more than one field is too big. This is slower.


If there is such a problem, one out of several possibilities might occur:
The fields have been sized generously while still having efficient row sizes for fast SQL queries. If you encounter a problem, it usually lies with generic fields like "description" or "comment", which some other programs use to store vast amounts of data (for example a VCard representation of the same contact you just imported).
* If your database is set to automatically truncate data, the content will be cut off at that part.
* If your database is not set for automatic truncation, the JDBC driver will throw a TruncationException.
** If your OX is set to automatically truncate, it will do so, though a bit less elegant than the database (because there is less information to go with): All problematic fields will be truncated one after the other, each change triggering a new attempt to store it, until either the data fits the database or the attempts are used up.
** If your OX is not set to truncate, it will simply fail.


To determine whether data fits the database, the size of the database fields can be used. But two things should be kept in mind: Depending on the character set used, a character can take up more than one byte. The default character set used it a 3-byte UTF8 set. Chinese installations, for example, might use a set that takes up more bytes per character or not even UTF8 at all. Also, some installations have used different field sizes for reasons ranging from performance to missing updates (though updates to database tables are really rare).
To determine whether data fits the database, the size of the database fields can be used. But two things should be kept in mind: Depending on the character set used, a character can take up more than one byte. The default character set used it a 3-byte UTF8 set. Chinese installations, for example, might use a set that takes up more bytes per character or not even UTF8 at all. Also, some installations have used different field sizes for reasons ranging from performance to missing updates (though updates to database tables are really rare).

Latest revision as of 09:53, 28 May 2013

There are several ways to import data into Open-Xchange, but all are restricted by the fact that OX uses a SQL database to store data about its core elements (contacts, appointments and tasks). That means that the field size is limited.

If there is such a problem, the data will be shortened/truncated to field the field. The technical details vary a little:

  • If your database is set to automatically truncate data, the content will be cut off at that part.
  • If your database is not to for automatic truncation, the server will try to do this. But the database only reports one oversized field at a time, so the server might need several attempts to trim the whole record in case more than one field is too big. This is slower.

The fields have been sized generously while still having efficient row sizes for fast SQL queries. If you encounter a problem, it usually lies with generic fields like "description" or "comment", which some other programs use to store vast amounts of data (for example a VCard representation of the same contact you just imported).

To determine whether data fits the database, the size of the database fields can be used. But two things should be kept in mind: Depending on the character set used, a character can take up more than one byte. The default character set used it a 3-byte UTF8 set. Chinese installations, for example, might use a set that takes up more bytes per character or not even UTF8 at all. Also, some installations have used different field sizes for reasons ranging from performance to missing updates (though updates to database tables are really rare).

Now you have been sufficiently warned: What follows is the data of a typical default installation. Yours might vary.

This part is currently under construction.
Reason: Currently collecting data from fresh installation