Page 1 of 1

once again, language / umlauts / LC_* / UTF-8 probs [solved]

Posted: Sun Aug 20, 2006 10:17 am
by TCWardrobe_
Hi community, hi Scalix staff!

I have an annoying issue with scalix related to german umlauts. First, what I did that this problem occured:

Setting up scalix
dl'ed SUSE-10.0-EvalDVD-i386-GM.iso, burned it, installed from that me as debianer must say, SuSE netinst sucks! :P )
dl'ed scalix-core-intel-suse-10.0.1.tar.gz, untared that, tried to install, failing due to "wrong" distro and some missing packages
changed /etc/SuSE-release so it shows (apparently just added "OSS", I absolutely don't get the SuSE naming scheme)

Code: Select all

SUSE LINUX 10.0 (i586) OSS
VERSION = 10.0

installed the missing packages and finally installed scalix without major problems

Adding users with SAC
Added about 5 users all went ok, number 6 had umlauts in his name and adding failed... somehow (don't remember the exact errormessage). If I try to delete that user through SAC, or just let me show the contents of this account I get

Code: Select all

Die Benutzerinformationen für den ausgewählten Benutzer konnten nicht geladen werden.

Code: Select all

Konnte nicht löschen: Anja Gr?terich.

If I try to delete the user with omdelu

Code: Select all

thebe:/opt/scalix/bin # omshowu -n Anja.Grueterich@tcw.local
Authentication ID: Anja.Grueterich@tcw.local
User Name : Anja Grueterich /CN=Anja Grueterich/S-TX=Gr?terich/CN-TX=Anja Gr?terich
MailNode : thebe,tcw
Internet Address : \=?UTF-8?Q?Anja_Gr\=C3\=BCterich?\= <Anja.Grueterich@tcw.local>
System Login : 60541
Password : set
Admin Capabilities : NO
Mailbox Admin Capabilities : NO
Language : GERMAN
Virtual Vault : Enabled (default)
Mail Account: Unlocked
Last Signon : Never.
Receipt of mail : ENABLED
Service level : 0
Excluded from Tidying : NO
User Class : Full

thebe:/opt/scalix/bin # omdelu -n Anja.Grueterich@tcw.local
[OM 8013] The name/mailnode contains invalid characters

omdelu: No users were deleted


Trying to fix it
I added some variables to setclasspath.sh

Code: Select all

thebe:/opt/scalix/bin # head -n 5 /opt/scalix-tomcat/bin/setclasspath.sh|tail -2export LANG=de_DE.UTF-8
export LC_ALL=de_DE.UTF-8
thebe:/opt/scalix/bin #


made sure that my locales settings are correct anyway

Code: Select all

thebe:/opt/scalix/bin # locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=


and if I look in what environment tomcat is actually running

Code: Select all

thebe:/opt/scalix/bin # cat /proc/`lsof -i :8005|tail -1|cut -d " " -f5`/environ | tr "\0" "\n"|egrep "LANG|LC_ALL"
LC_ALL=de_DE.UTF-8
LANG=de_DE.UTF-8
thebe:/opt/scalix/bin #


and still can't delete or get to the account (SAC) of that user

I got all the steps I did trying to fix that from various posts from this forum, did I miss a post? Any suggestions? It may be that now I could add users with umlauts but I am not in the mood of testing that without further help, I don't wan't too much "dead" users in this initial test-setup and of course I try to understand what went wrong and maybe if this issue could be addressed in the installer in the first place.

If all fails, for me it would be ok for now if I could just delete that user somehow and I would not use umlauts for the testusers so I can go on testing the other things.

TIA
Michael

Posted: Sun Aug 20, 2006 11:56 am
by florian
Hi Michael,

I believe the issue is what you're describing - as you were running tomcat in the "wrong" environment when you created the user, some of the characters in her name were encoded incorrectly. Therefore now SAC can't get ahold of the user properly anymore and can't delete it.

Please try to delete the user with the omdelu command, e.g.

Code: Select all

omdelu -n grueterich


(this is the lastname with the u-umlaut resolved into ue, which will work for as long as it is unique).

Can you then try to recreate a user and see if that one can then be managed using SAC?

Maybe one further check, just to be sure - in your script testing the environment for Tomcat, you only check for LC_ALL. However, if LC_CTYPE is set I believe this overrides LC_ALL for the character set, so you should actually be looking it at just LC_ in your grep.

Thanks for your very complete post,
Florian.

Posted: Sun Aug 20, 2006 12:32 pm
by TCWardrobe_
Hola! That one was easy! Thanks Florian!

Code: Select all

thebe:/opt/scalix/bin # omdelu -n grueterich
omdelu: One user successfully deleted

I never thought one can use anything like that as identifier as long as it is unique...

So, let's try if i can Add this user now... and it works! Fetching account info works too, so I assume deletion wouldn't be a problem anymore ;)

At first I did not grep for anything, just used |less and as I saw what environment variables were there I formed the grep command to make the pasting not bigger as needed. So, LC_ALL was the only LC_* variable shown.... but as I realise now, even if I'd had greped for LC_ it wouldn't got bigger anyway ;)

So, let's call it closed then, but can you see why this problem arose? I think if I had the problem, many other german (probably all that use non ascii characters in names) users may have it too. Is it _exactly_ the same problem discrobed one or two times here in the forum? AFAIS SuSE 10 is an utf-8 distro itself at default... so... what happend?

Posted: Sun Aug 20, 2006 12:38 pm
by florian
Well,

honestly - I can't tell you why this and that distro sets these variables in any specific way - we found this initially with all distros when the LANG variable was set to a non-UTF-8 value; then we had the installer add that designation to the Tomcat start scripts; obviously this didn't catch the SuSE case as the LC_ variables take precedence.

This was tracked as Scalix Bug 11539 (we'll open up our Bugzilla soon), fixed for Scalix 11, which now also makes sure the LC_ variables are setup correctly when you start Tomcat.

Glad that it works for you,
-- F.