After a longer period of time, I have decided to install Kolab and use it as a personal information manager. The installation process went as expected until the setup process tried to install the Roundcube database and failed miserably.
Source of the problem
The problem can be easily identified by the error messages returned by the setup process.
Follow the example below to see MySQL errors at the very end.
# setup-kolab Please supply a password for the LDAP administrator user 'admin', used to login to the graphical console of 389 Directory server. Administrator password [Sii79iQyU3gkHN3]: Please supply a password for the LDAP Directory Manager user, which is the administrator user you will be using to at least initially log in to the Web Admin, and that Kolab uses to perform administrative tasks. Directory Manager password [nNQAwQVnSDRzSUq]: Please choose the system user and group the service should use to run under. These should be existing, unprivileged, local system POSIX accounts with no shell. User [dirsrv]: Group [dirsrv]: This setup procedure plans to set up Kolab Groupware for the following domain name space. This domain name is obtained from the reverse DNS entry on your network interface. Please confirm this is the appropriate domain name space. sleeplessbeastie.eu [Y/n]: The standard root dn we composed for you follows. Please confirm this is the root dn you wish to use. dc=sleeplessbeastie,dc=eu [Y/n]: Setup is now going to set up the 389 Directory Server. This may take a little while (during which period there is no output and no progress indication). [..] Please supply a Cyrus Administrator password. This password is used by Kolab to execute administrative tasks in Cyrus IMAP. You may also need the password yourself to troubleshoot Cyrus IMAP and/or perform other administrative tasks against Cyrus IMAP directly. Cyrus Administrator password [qolmP-8qsQAiPRe]: Please supply a Kolab Service account password. This account is used by various services such as Postfix, and Roundcube, as anonymous binds to the LDAP server will not be allowed. Kolab Service password [yp48IyJvn8rd12H]: [..] What MySQL server are we setting up? - 1: Existing MySQL server (with root password already set). - 2: New MySQL server (needs to be initialized). Choice: 1 Please supply the root password for MySQL, so we can set up user accounts for other components that use MySQL. MySQL root password: Please supply a password for the MySQL user 'kolab'. This password will be used by Kolab services, such as the Web Administration Panel. MySQL kolab password [LAOv8Gmn2l1Gs-G]: Please supply the timezone PHP should be using. You have to use a Continent or Country / City locality name like 'Europe/Berlin', but not just 'CEST'. Timezone ID [UTC]: Please supply a password for the MySQL user 'roundcube'. This password will be used by the Roundcube webmail interface. MySQL roundcube password [yP7R_m4wJVcNiDW]: ERROR 1005 (HY000) at line 9: Can't create table 'roundcube.kolab_alarms' (errno: 150) ERROR 1146 (42S02) at line 179: Table 'roundcube.system' doesn't exist [..]
These errors are generated due to the following code inside /usr/share/pyshared/pykolab/setup/setup_roundcube.py file.
schema_files = []
for root, directories, filenames in os.walk('/usr/share/doc/'):
for directory in directories:
if directory.startswith("roundcubemail"):
for root, directories, filenames in os.walk(os.path.join(root, directory)):
for filename in filenames:
if filename.startswith('mysql.initial') and filename.endswith('.sql'):
schema_filepath = os.path.join(root,filename)
if not schema_filepath in schema_files:
schema_files.append(schema_filepath)
break
breakI assume that its goal is to recursively find every mysql.initial*.sql file inside /usr/share/doc/roundcubemail*/ directories.
Finding the solution
The first approach – Problem identification
Copy the earlier mentioned code to the distinct script and slightly modify it to output additional information.
$ cat first_code_check.py
import os
schema_files = []
for root, directories, filenames in os.walk('/usr/share/doc/'):
for directory in directories:
if directory.startswith("roundcubemail"):
print "-> " + directory + " (" + root + ")"
for root, directories, filenames in os.walk(os.path.join(root, directory)):
for filename in filenames:
print " \ " + filename
if filename.startswith('mysql.initial') and filename.endswith('.sql'):
schema_filepath = os.path.join(root,filename)
if not schema_filepath in schema_files:
schema_files.append(schema_filepath)
It is easy to notice the problem after the script execution – the root variable inside third for loop is messing with future iterations.
$ python first_code_check.py
-> roundcubemail-plugins-kolab (/usr/share/doc/) \ changelog.Debian.gz \ copyright -> roundcubemail-plugin-threadingasdefault (/usr/share/doc/roundcubemail-plugins-kolab) -> roundcubemail (/usr/share/doc/roundcubemail-plugins-kolab)
The second approach – Python script
The most obvious fix is to rename the second occurrence of the root variable.
$ cat second_code_check.py
import os
schema_files = []
for root, directories, filenames in os.walk('/usr/share/doc/'):
for directory in directories:
if directory.startswith("roundcubemail"):
print "-> " + directory + "(" + root + ")"
for nested_root, directories, filenames in os.walk(os.path.join(root, directory)):
for filename in filenames:
print " \ " + filename
if filename.startswith('mysql.initial') and filename.endswith('.sql'):
schema_filepath = os.path.join(nested_root,filename)
if not schema_filepath in schema_files:
schema_files.append(schema_filepath)
print " ! added schema file"A simple test reveals that it works as expected.
$ python second_code_check.py
-> roundcubemail-plugins-kolab(/usr/share/doc/)
\ changelog.Debian.gz
\ copyright
-> roundcubemail-plugin-threadingasdefault(/usr/share/doc/)
\ changelog.Debian.gz
\ copyright
-> roundcubemail(/usr/share/doc/)
\ mysql.initial.sql
! added schema file
\ mssql.initial.sql
\ README.md
\ sqlite.initial.sql
\ changelog.Debian.gz
\ copyright
\ postgres.initial.sql
-> roundcubemail-plugin-contextmenu(/usr/share/doc/)
\ changelog.gz
\ changelog.Debian.gz
\ copyrightThe second approach – Path
--- /usr/share/pyshared/pykolab/setup/setup_roundcube.py.orig 2014-06-21 18:43:40.975058719 +0200
+++ /usr/share/pyshared/pykolab/setup/setup_roundcube.py 2014-06-21 19:12:46.957149746 +0200
@@ -139,16 +139,13 @@
for root, directories, filenames in os.walk('/usr/share/doc/'):
for directory in directories:
if directory.startswith("roundcubemail"):
- for root, directories, filenames in os.walk(os.path.join(root, directory)):
+ for nested_root, directories, filenames in os.walk(os.path.join(root, directory)):
for filename in filenames:
if filename.startswith('mysql.initial') and filename.endswith('.sql'):
- schema_filepath = os.path.join(root,filename)
+ schema_filepath = os.path.join(nested_root,filename)
if not schema_filepath in schema_files:
schema_files.append(schema_filepath)
- break
- break
-
if os.path.isdir('/usr/share/roundcubemail'):
rcpath = '/usr/share/roundcubemail/'
elif os.path.isdir('/usr/share/roundcube'):The third approach – Python script
This could be simplified a bit by replacing problematic os.walk call.
$ cat third_code_check.py
import os
import fnmatch
schema_files = []
for root, directories, filenames in os.walk('/usr/share/doc/'):
for directory in directories:
if directory.startswith("roundcubemail"):
print "-> " + directory + "(" + root + ")"
for filename in os.listdir(os.path.join(root, directory)):
if fnmatch.fnmatch(filename, 'mysql.initial*.sql'):
schema_filepath = os.path.join(root,directory,filename)
if not schema_filepath in schema_files:
schema_files.append(schema_filepath)
print " \ " + filename
$ python third_code_check.py
-> roundcubemail-plugins-kolab (/usr/share/doc/) -> roundcubemail-plugin-threadingasdefault (/usr/share/doc/) -> roundcubemail (/usr/share/doc/) \ mysql.initial.sql -> roundcubemail-plugin-contextmenu (/usr/share/doc/)
The third approach – Patch
--- /usr/share/pyshared/pykolab/setup/setup_roundcube.py.orig 2014-06-21 18:43:40.975058719 +0200
+++ /usr/share/pyshared/pykolab/setup/setup_roundcube.py 2014-06-21 19:43:14.235162418 +0200
@@ -20,6 +20,7 @@
from Cheetah.Template import Template
import hashlib
import os
+import fnmatch
import random
import re
import subprocess
@@ -139,15 +140,11 @@
for root, directories, filenames in os.walk('/usr/share/doc/'):
for directory in directories:
if directory.startswith("roundcubemail"):
- for root, directories, filenames in os.walk(os.path.join(root, directory)):
- for filename in filenames:
- if filename.startswith('mysql.initial') and filename.endswith('.sql'):
- schema_filepath = os.path.join(root,filename)
- if not schema_filepath in schema_files:
- schema_files.append(schema_filepath)
-
- break
- break
+ for filename in os.listdir(os.path.join(root, directory)):
+ if fnmatch.fnmatch(filename, 'mysql.initial*.sql'):
+ schema_filepath = os.path.join(root,directory,filename)
+ if not schema_filepath in schema_files:
+ schema_files.append(schema_filepath)
if os.path.isdir('/usr/share/roundcubemail'):
rcpath = '/usr/share/roundcubemail/'Fourth approach – Python script
This code can be simplified further by not reading directories recursively.
$ cat fourth_code_check.py
import os
import fnmatch
import glob
schema_files = []
for directory in glob.glob('/usr/share/doc/roundcubemail*'):
print "-> " + directory
for filename in os.listdir(directory):
if fnmatch.fnmatch(filename, 'mysql.initial*.sql'):
schema_filepath = os.path.join(directory,filename)
if not schema_filepath in schema_files:
schema_files.append(schema_filepath)
print " \ " + filename
$ python fourth_code_check.py
-> roundcubemail-plugins-kolab (/usr/share/doc/) -> roundcubemail-plugin-threadingasdefault (/usr/share/doc/) -> roundcubemail (/usr/share/doc/) \ mysql.initial.sql -> roundcubemail-plugin-contextmenu (/usr/share/doc/)
Fourth approach – Patch
--- /usr/share/pyshared/pykolab/setup/setup_roundcube.py.orig 2014-06-21 18:43:40.975058719 +0200
+++ /usr/share/pyshared/pykolab/setup/setup_roundcube.py 2014-06-21 19:59:42.380040035 +0200
@@ -20,6 +20,8 @@
from Cheetah.Template import Template
import hashlib
import os
+import glob
+import fnmatch
import random
import re
import subprocess
@@ -136,18 +138,12 @@
fp.close()
schema_files = []
- for root, directories, filenames in os.walk('/usr/share/doc/'):
- for directory in directories:
- if directory.startswith("roundcubemail"):
- for root, directories, filenames in os.walk(os.path.join(root, directory)):
- for filename in filenames:
- if filename.startswith('mysql.initial') and filename.endswith('.sql'):
- schema_filepath = os.path.join(root,filename)
- if not schema_filepath in schema_files:
- schema_files.append(schema_filepath)
-
- break
- break
+ for directory in glob.glob('/usr/share/doc/roundcubemail*'):
+ for filename in os.listdir(directory):
+ if fnmatch.fnmatch(filename, 'mysql.initial*.sql'):
+ schema_filepath = os.path.join(directory,filename)
+ if not schema_filepath in schema_files:
+ schema_files.append(schema_filepath)
if os.path.isdir('/usr/share/roundcubemail'):
rcpath = '/usr/share/roundcubemail/'Ending notes
I am not a Python developer. I wrote about this issue as it was an interesting way to learn about directory traversal and file filtering implemented in Python language.
It is up to you to decide which solution is appropriate for you – for more recent information, please read the official bug report.