Csv import - Revision history

217.6.212.138: /* Speed */

2011-09-20T10:58:34Z

Speed

← Older revision		Revision as of 10:58, 20 September 2011
Line 67:		Line 67:
	The import speed heavily depends on your hardware and infrastructure. A test on a virtual machine (Laptop, 2.5Ghz, 2GB Ram, all services and OX on the same system) imported 1000 contexts in 37 seconds and additional 900 users in 20 seconds. Your mileage my vary depending on the amount of contexts/users.		The import speed heavily depends on your hardware and infrastructure. A test on a virtual machine (Laptop, 2.5Ghz, 2GB Ram, all services and OX on the same system) imported 1000 contexts in 37 seconds and additional 900 users in 20 seconds. Your mileage my vary depending on the amount of contexts/users.

	Other installations have imported 560.000 user in 11 hours, which included the creation of 77.000 contexts. This was done by running 1 import shell for contexts and after that 1 import shell for creating users inside the contexts. After testing optimizations, the import of the 560.000 users was done in 2 hours and 20 minutes, by running 3 shells, each on a different node and reading a dedicated csv file. So, the user CSV list was splitted in 3 parts and each part was imported by a seperate shell at the same time.		Other installations have imported 560.000 user in 11 hours, which included the creation of 77.000 contexts. This was done by running 1 import shell for contexts and after that 1 import shell for creating users inside the contexts. After testing optimizations, the import of the 560.000 users was done in 2 hours and 20 minutes, by running 3 shells, each on a different node and reading a dedicated csv file. So, the user CSV list was splitted in 3 parts and each part was imported by a seperate shell at the same time. One import node was running with 32GB of RAM, the other two with 14GB of RAM. Each having a quad core CPU.

	=== Definitions ===		=== Definitions ===

217.6.212.138: /* Speed */

2011-09-20T10:55:47Z

Speed

← Older revision		Revision as of 10:55, 20 September 2011
Line 65:		Line 65:

	=== Speed ===		=== Speed ===
	The import speed depends on your hardware and infrastructure. A test on a virtual machine (Laptop, 2.5Ghz, 2GB Ram, all services and OX on the same system) imported 1000 contexts in 37 seconds and additional 900 users in 20 seconds. Your mileage my vary depending on the amount of contexts/users.		The import speed heavily depends on your hardware and infrastructure. A test on a virtual machine (Laptop, 2.5Ghz, 2GB Ram, all services and OX on the same system) imported 1000 contexts in 37 seconds and additional 900 users in 20 seconds. Your mileage my vary depending on the amount of contexts/users.

	Other installations have imported 560.000 user in 11 hours, which included the creation of 77.000 contexts.		Other installations have imported 560.000 user in 11 hours, which included the creation of 77.000 contexts. This was done by running 1 import shell for contexts and after that 1 import shell for creating users inside the contexts. After testing optimizations, the import of the 560.000 users was done in 2 hours and 20 minutes, by running 3 shells, each on a different node and reading a dedicated csv file. So, the user CSV list was splitted in 3 parts and each part was imported by a seperate shell at the same time.

	=== Definitions ===		=== Definitions ===

Cutmasta: /* Speed */

2011-09-19T11:17:03Z

Speed

← Older revision		Revision as of 11:17, 19 September 2011
Line 65:		Line 65:

	=== Speed ===		=== Speed ===
	The import speed depends on your hardware and infrastructure. A test on a virtual machine (Laptop, 2.5Ghz, 2GB Ram, all services and OX on the same system) imported 1000 contexts in 37 seconds and additional 900 users in 20 seconds. Your mileage my vary.		The import speed depends on your hardware and infrastructure. A test on a virtual machine (Laptop, 2.5Ghz, 2GB Ram, all services and OX on the same system) imported 1000 contexts in 37 seconds and additional 900 users in 20 seconds. Your mileage my vary depending on the amount of contexts/users.

			Other installations have imported 560.000 user in 11 hours, which included the creation of 77.000 contexts.

	=== Definitions ===		=== Definitions ===

Malasa at 15:19, 8 September 2011

2011-09-08T15:19:54Z

← Older revision		Revision as of 15:19, 8 September 2011
Line 17:		Line 17:

	<pre>		<pre>
	contextid,addmapping,username,password,displayname,givenname,surname,email,quota,access-combination-name,language,timezone		contextid,addmapping,username,password,displayname,givenname,surname,email,quota,access-combination-name,language,timezone,contextname
	"100","loginmapping_100","contextadmin","contextadminpassword","Context Admin 100","Context 100","Admin","admin_100@mymailserver.org","1024","all","en_GB","Europe/London"		"100","loginmapping_100","contextadmin","contextadminpassword","Context Admin 100","Context 100","Admin","admin_100@mymailserver.org","1024","all","en_GB","Europe/London","ctx100"
	"101","loginmapping_101","contextadmin","contextadminpassword","Context Admin 101","Context 101","Admin","admin_101@mymailserver.org","1024","all","en_GB","Europe/London"		"101","loginmapping_101","contextadmin","contextadminpassword","Context Admin 101","Context 101","Admin","admin_101@mymailserver.org","1024","all","en_GB","Europe/London","ctx101"
	"102","loginmapping_102","contextadmin","contextadminpassword","Context Admin 102","Context 102","Admin","admin_102@mymailserver.org","1024","all","en_GB","Europe/London"		"102","loginmapping_102","contextadmin","contextadminpassword","Context Admin 102","Context 102","Admin","admin_102@mymailserver.org","1024","all","en_GB","Europe/London","ctx102"
	"103","loginmapping_103","contextadmin","contextadminpassword","Context Admin 103","Context 103","Admin","admin_103@mymailserver.org","1024","all","en_GB","Europe/London"		"103","loginmapping_103","contextadmin","contextadminpassword","Context Admin 103","Context 103","Admin","admin_103@mymailserver.org","1024","all","en_GB","Europe/London","ctx103"
	"104","loginmapping_104","contextadmin","contextadminpassword","Context Admin 104","Context 104","Admin","admin_104@mymailserver.org","1024","all","en_GB","Europe/London"		"104","loginmapping_104","contextadmin","contextadminpassword","Context Admin 104","Context 104","Admin","admin_104@mymailserver.org","1024","all","en_GB","Europe/London","ctx104"
	"105","loginmapping_105","contextadmin","contextadminpassword","Context Admin 105","Context 105","Admin","admin_105@mymailserver.org","1024","all","en_GB","Europe/London"		"105","loginmapping_105","contextadmin","contextadminpassword","Context Admin 105","Context 105","Admin","admin_105@mymailserver.org","1024","all","en_GB","Europe/London","ctx105"
	</pre>		</pre>

Line 58:		Line 58:

	=== Additional attributes ===		=== Additional attributes ===
	The ~~example covers~~ only the most common attributes. If you want to import other information check the --extendedoptions for a list of possible options:		The examples cover only the most common attributes. If you want to import other information check the --extendedoptions for a list of possible options (this work both for createuser and createcontext):

	createuser --extendedoptions		createuser --extendedoptions

			You can use all double dash options directly in your field definition. All short options like -L have also a long optionname, in this example --addmappings.

			=== Speed ===
			The import speed depends on your hardware and infrastructure. A test on a virtual machine (Laptop, 2.5Ghz, 2GB Ram, all services and OX on the same system) imported 1000 contexts in 37 seconds and additional 900 users in 20 seconds. Your mileage my vary.

	=== Definitions ===		=== Definitions ===

217.6.212.138: /* Create OX Contexts and OX Users from CSV Data */

2011-08-31T08:24:24Z

Create OX Contexts and OX Users from CSV Data

← Older revision		Revision as of 08:24, 31 August 2011
Line 11:		Line 11:
	In fact you have to know/provide the same data in the CSV file as you would need to create an OX user account via the Commandline tools or RMI/SOAP API.		In fact you have to know/provide the same data in the CSV file as you would need to create an OX user account via the Commandline tools or RMI/SOAP API.

	Note: The first line in each of the 2 data files, defines the column names and has to be in every file you want use in the batch creation process. ~~So you~~ can either create 1 big files for all contexts and one big file for your user accounts, or you can create 1 file for each context and one file for all of the users inside this context. It´s totally up you how you want to process the files.		Note: The first line in each of the 2 data files, defines the column names and has to be in every file you want use in the batch creation process. The column names must be exactly like "long options" of the commandline tools. You can check for all possible "columns/long options" via "createuser --extendedoptions" and "createcontext --extendedoptions". You can either create 1 big files for all contexts and one big file for your user accounts, or you can create 1 file for each context and one file for all of the users inside this context. It´s totally up you how you want to process the files.

	=== Creating new contexts ===		=== Creating new contexts ===

Choeger: /* Definitions */

2011-08-31T06:22:53Z

Definitions

← Older revision		Revision as of 06:22, 31 August 2011
Line 63:		Line 63:

	=== Definitions ===		=== Definitions ===
	Language codes: you can use the same you have in the language menue for the user, like de_DE, en_GB, en_US, jp_JP ...		Language codes: you can use the same you have in the language menue for the user, like de_DE, en_GB, en_US, jp_JP ..., check [[Available_Translations]] for a list of what's supported.

	A list of timezones:		A list of timezones:
	[http://en.wikipedia.org/wiki/Zone.tab Wikipedia:Zone.tab]		[http://en.wikipedia.org/wiki/Zone.tab Wikipedia:Zone.tab]

217.6.212.138: /* Creating users inside contexts */

2011-08-30T11:38:07Z

Creating users inside contexts

← Older revision		Revision as of 11:38, 30 August 2011
Line 51:		Line 51:
	"104","contextadmin","contextadminpassword","sandra_linux","Sandra Linux","Sandra","Linux","secret","sandra_linux@mymailserver.org","sandra_linux","imap://imapserver:143","smtp://mailserver:25","1024","all","en_GB","Europe/London"</pre>		"104","contextadmin","contextadminpassword","sandra_linux","Sandra Linux","Sandra","Linux","secret","sandra_linux@mymailserver.org","sandra_linux","imap://imapserver:143","smtp://mailserver:25","1024","all","en_GB","Europe/London"</pre>

	This example has two users, pete_test and sandra_linux, which will be created in the previously created context 105 and in context 104, ~~if you use~~ it ~~as an~~ input file ~~for the~~ create user ~~CSV import~~.		This example has two users, pete_test and sandra_linux, which will be created in the previously created context 105 and in context 104. As you can see, it is possible to merge users from different contexts in a single CSV input file. Of course, it is also possible to create a CSV user file per context and use it after the context was created. It is now totally up to you, how you handle the workflow.

	To import the users:		To import the users:

217.6.212.138: /* Creating users inside contexts */

2011-08-30T11:36:33Z

Creating users inside contexts

← Older revision		Revision as of 11:36, 30 August 2011
Line 49:		Line 49:
	<pre>contextid,adminuser,adminpass,username,displayname,givenname,surname,password,email,imaplogin,imapserver,smtpserver,quota,access-combination-name,language,timezone		<pre>contextid,adminuser,adminpass,username,displayname,givenname,surname,password,email,imaplogin,imapserver,smtpserver,quota,access-combination-name,language,timezone
	"105","contextadmin","contextadminpassword","pete_test","Pete Test","Pete","Test","secret","pete_test@mymailserver.org","pete_test","imap://imapserver:143","smtp://mailserver:25","1024","all","de_DE","Europe/Berlin"		"105","contextadmin","contextadminpassword","pete_test","Pete Test","Pete","Test","secret","pete_test@mymailserver.org","pete_test","imap://imapserver:143","smtp://mailserver:25","1024","all","de_DE","Europe/Berlin"
	"~~105~~","contextadmin","contextadminpassword","sandra_linux","Sandra Linux","Sandra","Linux","secret","sandra_linux@mymailserver.org","sandra_linux","imap://imapserver:143","smtp://mailserver:25","1024","all","en_GB","Europe/London"</pre>		"104","contextadmin","contextadminpassword","sandra_linux","Sandra Linux","Sandra","Linux","secret","sandra_linux@mymailserver.org","sandra_linux","imap://imapserver:143","smtp://mailserver:25","1024","all","en_GB","Europe/London"</pre>

	This example has two users, pete_test and sandra_linux, which will be created in the previously created context 105, if you use it as an input file for the create user CSV import.		This example has two users, pete_test and sandra_linux, which will be created in the previously created context 105 and in context 104, if you use it as an input file for the create user CSV import.

	To import the users:		To import the users:

217.6.212.138: /* Creating users inside contexts */

2011-08-30T11:36:04Z

Creating users inside contexts

← Older revision		Revision as of 11:36, 30 August 2011
Line 51:		Line 51:
	"105","contextadmin","contextadminpassword","sandra_linux","Sandra Linux","Sandra","Linux","secret","sandra_linux@mymailserver.org","sandra_linux","imap://imapserver:143","smtp://mailserver:25","1024","all","en_GB","Europe/London"</pre>		"105","contextadmin","contextadminpassword","sandra_linux","Sandra Linux","Sandra","Linux","secret","sandra_linux@mymailserver.org","sandra_linux","imap://imapserver:143","smtp://mailserver:25","1024","all","en_GB","Europe/London"</pre>

	This example has two users, pete_test and sandra_linux, in the previously created context 105.		This example has two users, pete_test and sandra_linux, which will be created in the previously created context 105, if you use it as an input file for the create user CSV import.

	To import the users:		To import the users:

Malasa: /* Creating users inside contexts */

2011-08-30T11:11:14Z

Creating users inside contexts

← Older revision		Revision as of 11:11, 30 August 2011
Line 47:		Line 47:
	===Creating users inside contexts ===		===Creating users inside contexts ===

	<pre>contextid,adminuser,adminpass~~,contextid~~,username,displayname,givenname,surname,password,email,imaplogin,imapserver,smtpserver,quota,access-combination-name,language,timezone		<pre>contextid,adminuser,adminpass,username,displayname,givenname,surname,password,email,imaplogin,imapserver,smtpserver,quota,access-combination-name,language,timezone
	"105","contextadmin","contextadminpassword","pete_test","Pete Test","Pete","Test","secret","pete_test@mymailserver.org","pete_test","imap://imapserver:143","smtp://mailserver:25","1024","all","de_DE","Europe/Berlin"		"105","contextadmin","contextadminpassword","pete_test","Pete Test","Pete","Test","secret","pete_test@mymailserver.org","pete_test","imap://imapserver:143","smtp://mailserver:25","1024","all","de_DE","Europe/Berlin"
	"105","contextadmin","contextadminpassword","sandra_linux","Sandra Linux","Sandra","Linux","secret","sandra_linux@mymailserver.org","sandra_linux","imap://imapserver:143","smtp://mailserver:25","1024","all","en_GB","Europe/London"		"105","contextadmin","contextadminpassword","sandra_linux","Sandra Linux","Sandra","Linux","secret","sandra_linux@mymailserver.org","sandra_linux","imap://imapserver:143","smtp://mailserver:25","1024","all","en_GB","Europe/London"</pre>
	</pre>

	This example has two users, pete_test and sandra_linux, in the previously created context 105.		This example has two users, pete_test and sandra_linux, in the previously created context 105.