[SYS 22] Invalid argument

[mabadjiev]

Hello,
I am getting the followin error:

ERROR Local Delivery(Local Delivery) Mon Jul 24 23:50:14 2006
[SYS 22] Invalid argument
Pid of logging process: 21510
Current errno value: 22
Last Msg Id: 10140335.1153809682
Last Msg DirectRef: 0012e574b5c00203

---
It's not very clear what that means. Anyone?

[stephan.klein]

The same thing here with inbound messages (latest Scalix version on Debian):

ERROR Local Delivery(Local Delivery) Thu Feb 22 18:37:32 2007
[SYS 22] Invalid argument
Pid of logging process: 23355
Current errno value: 22
Last Msg Id: 45DDC74A.40002(a)DOMAIN_REMOVED
Last Msg DirectRef: 00012000ac0914b7

Despite this error, the message gets delivered.

What is the reason for this error? Any hints how to solve it?

Thank you & kind regards
Stephan

[Richard Hall]

Some more detail would be helpful:

Does this occur for all messages delivered?
Does it occur only for delivery to certain users' mailboxes?
If it is only certain messages do they have anything in common?
Is there a stack trace in the error log for these errors?

Thanks - Richard

[stephan.klein]

Hi Richard,

thank you for your reply!

To answer your questions:

As far as I tested by now, it hits all messages delivered to local accounts. There is no difference, if the mail comes from an external user or via webmail from an internal user.

I have checked two users who are the only users in my testing environment and it happens to both of them. I tried sending using different mail clients, too - no difference.

catalina.out does not log any error.

Thank you, kind regards
Stephan

Update:
The following details I found in "Lokale Mailzustellung" in the /sac:

ERROR Local Delivery(Local Delivery) 02.23.07 13:52:30
[SYS 22] Invalid argument
Current errno value: 22
Last Msg Id: 33C7DC0BB932C6478F85CAF52E45072B7F5195(a)DOMAIN_REMOVED
Last Msg DirectRef: 00011c079042341d
-> ct_FindDITEntry
<- ct_FindDITEntry
-> nf_BroadcastNotif
-> ct_FindDITEntry
<- ct_FindDITEntry
-> ct_FindDITEntry
<- ct_FindDITEntry
-> nfda_SendMessage
-> openInFiles
<- openInFiles
<- nfda_SendMessage
<- nf_BroadcastNotif
<- ct_NotifyCtner
-> ct_UpdateMessageInfo
<- /build/11.0.1/src/lib/ombase/os/os_shrmem.c:37[1,22]
<- /build/11.0.1/src/lib/nsl/nsl_lib.c:200[1,22]


ERROR Local Delivery(Local Delivery) 02.23.07 13:55:20
[SYS 22] Invalid argument
Current errno value: 22
Last Msg Id: 33C7DC0BB932C6478F85CAF52E45072B7F5197(a)DOMAIN_REOVED
Last Msg DirectRef: 00011c082e3185d5
<- ct_FindDITEntry
-> ct_MapDITEntryToRef
<- ct_MapDITEntryToRef
-> ct_NotifyCtner
-> ct_FindDITEntry
<- ct_FindDITEntry
-> nf_BroadcastNotif
-> ct_FindDITEntry
<- ct_FindDITEntry
-> ct_FindDITEntry
<- ct_FindDITEntry
<- nf_BroadcastNotif
<- ct_NotifyCtner
-> ct_UpdateMessageInfo
<- /build/11.0.1/src/lib/ombase/os/os_shrmem.c:37[1,22]
<- /build/11.0.1/src/lib/nsl/nsl_lib.c:200[1,22]


ERROR Local Delivery(Local Delivery) 02.23.07 13:57:39
[SYS 22] Invalid argument
Current errno value: 22
Last Msg Id: 8106699.41172235458484.JavaMail.root(a)server06
Last Msg DirectRef: 00011c0d7d652b34
<- ct_ExtractDIData
-> ct_MapDITEntryToRef
<- ct_MapDITEntryToRef
-> ct_NotifyCtner
-> ct_FindDITEntry
<- ct_FindDITEntry
-> nf_BroadcastNotif
-> ct_FindDITEntry
<- ct_FindDITEntry
-> ct_FindDITEntry
<- ct_FindDITEntry
<- nf_BroadcastNotif
<- ct_NotifyCtner
-> ct_UpdateMessageInfo
<- /build/11.0.1/src/lib/ombase/os/os_shrmem.c:37[1,22]
<- /build/11.0.1/src/lib/nsl/nsl_lib.c:200[1,22]

[Richard Hall]

Hi Stephan,

From the error stack trace I can tell that local.delivery is having a problem in accessing shared memory. Can you tell me if the problem persists after you restart local.delivery:

Code: Select all

omoff -d 0 ld
omon ld

If it does persist can you try restarting Scalix completely:

Code: Select all

omshut
omrc

Does it still happen after restating Scalix?
Regards - Richard

[stephan.klein]

Hi Richard,

I just checked your two suggestions, neither of them solves the problem, the error remains.

I have attached the Stop/Start output.

Thank you for your help,

kind regards from Germany
Stephan

---
server06:/var/opt/scalix/s6/s/logs# omshut
Disabling 14 subsystem(s).
omoff : [OM 4835] Subsystem Item Structure Server is already off.
Disabling 8 subsystem(s).
server06:/var/opt/scalix/s6/s/logs# omrc
Scalix 11.0.1.11

Copyright (C) 2002-2007 Scalix Corporation. All rights reserved.
Copyright (C) 2002 Hewlett-Packard Development Company, L.P. All rights reserved.
(c) Copyright Hewlett-Packard Company 1989-2001. All rights reserved.
Portions (c) Copyright Raima(TM) Corporation 1984,1990. All rights reserved.
Portions (c) Copyright 1991-1995 Regents of the University of Michigan.
All rights reserved.
Portions (c) Copyright 1989-1994 Regents of the University of California.
All rights reserved.
Portions (c) Copyright 1999 The MITRE Corporation, Keith Visco.
All rights reserved.
Portions (c) Copyright 1998-1999 James Clark. All rights reserved.
Portions (c) Copyright 1998-1999 Zveno Pty Ltd, Steve Ball. All rights reserved.
Portions (c) Copyright 1994-1999 Australian National University, CSIRO.
All rights reserved.
Portions (c) Copyright 1996-1999 Andrew G. Morgan. All rights reserved.
Portions (c) Copyright 1996,1998 Cristian Gafton, CRYPTOCard Inc.
Portions (c) Copyright 1994-1999 Sun Microsystems Inc, Scriptics Corporation.

The Logictran R2Net Converter is copyright (C) 2000, Logictran, Inc. Portions
of the R2Net Converter are based in part on the work of the University of
Toronto.

RESTRICTED RIGHTS LEGEND
Use, duplication, or disclosure by the U.S. Government is subject to
restrictions as set forth in sub-paragraph (c)(1)(ii) of the Rights in
Technical Data and Computer Software clause in DFARS 252.227-7013.

Scalix Corporation
1400 Fashion Island Blvd.
Suite 602
San Mateo, CA 94404-2061
USA

Rights for non-DOD U.S. Government Departments and Agencies are as set
forth in FAR 52.227-19(c)(1,2).

Enabling 8 subsystem(s).
Database Monitor Started
Directory Relay Server Started
Notification Server Started
LDAP Daemon Started
IMAP Server Daemon Started
SMTP Relay Started
Mime Browser Controller Started
Event Server Started
Enabling 14 subsystem(s).
Internet Mail Gateway Started
Sendmail Interface Stopped
Local Client Interface Enabled
Remote Client Interface Enabled
Test Server Started
Directory Synchronization Stopped
Dump Server Stopped
Service Router Started
Local Delivery Started
Request Server Started
Print Server Started
Bulletin Board Server Started
Background Search Service Started
CDA Server Started
POP3 interface Started
Omscan Server Started
Archiver Started
omrc : Scalix started

[Richard Hall]

OK, so it seems to be a persistent problem with shared memory.
What do get when you run the ipcs -m command?

For a Scalix 11 system (RHEL3) I get:

Code: Select all

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status     
0x00000000 7208960    scalix    666        9220       75                     
0x00000000 7241729    scalix    666        21352      70                     
0x00000000 7274498    scalix    666        62924      20                     
0x00000000 7307267    scalix    666        566152     36                     
0x00000000 7340036    scalix    666        3747144    40                     
0x00000000 7372805    scalix    666        185548     1                       
0x00000000 7405574    scalix    666        62892      9                       
0x00000000 7438343    scalix    666        6352       65                     
0x00000000 7471112    scalix    666        2359548    14                     
0x00000000 7503881    scalix    660        65536      5                       
0x00577a89 7176202    postgres  600        10461184   7

Cheers - Richard

[stephan.klein]

Hi Richard,

I see

Code: Select all

server06:/var/opt/scalix/s6/s/logs# ipcs -m

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x0052e2c1 0          postgres  600        10436608   1
0x00577a89 32769      postgres  600        10436608   6
0x00000000 720898     scalix    666        9220       44
0x00000000 753667     scalix    666        21352      39
0x00000000 786436     scalix    666        62924      16
0x00000000 819205     scalix    666        565792     10
0x00000000 851974     scalix    666        3747144    16
0x00000000 884743     scalix    666        185548     1
0x00000000 917512     scalix    666        62892      2
0x00000000 950281     scalix    666        6352       34
0x00000000 983050     scalix    666        2359548    25


CU
Stephan

[jch]

I think the EINVAL (errno = 22) might be a red herring because the code looks as though it might be attempting to detach an invalid pointer at the point of failure. Which is a pity -- it would be nice to know the real reason for failure -- it looks as though a call to shmat() is failing. There might be a way to find the real error ...

While I'm thinking about that though can you tell us what's in each of the /proc/sys/kernel/shm* files -- the usual cause of shared memory failures is lack of resource somewhere. What version of the kernel (uname -r) are you running? And which Debian release?

We might be able to find the actual error. Please bear with the instructions as I tried this out on a 64 bit machine (running 32 bit Scalix!). We want to put the local delivery process into an aborted state so we can start it "by hand" under strace so try this:

Code: Select all

kill $(omsetsvc -p ld)
omstat -s  # check that local delivery is "aborted"
strace -f -o /tmp/ld local.delivery -i

Check that the error is reported by omshowlog and then do the "kill" command again. If you grep for "shm" in /etc/ld you should get all the sys v shared memory calls including, vitally, the value of errno for any that failed. Just showing us the output of "grep shm /etc/ld" would be useful then we can might be able to tell what's going on.

jch

[stephan.klein]

Hi,

sorry for delay.

I did some further tests on my system and installed a second machine running scalix.

The problem seems to be related to the grsecurity extensions I compiled in the kernel. On the second system I tested, the licensing daemon did not work (segfault). After disabeling grsecurity, everything worked fine, so is disabled it on the first machine and now everything is fine here,too.

If you want to track this issue and need further information, please do not hesitate to contact me.

Thank you for your excellent support in this forum!

Stephan

[florian]

Stephan,

not sure and I don't think we're planning on supporting this environment anytime soon, but out of personal interest, did grsecurity add an entry to it's audit logs about what our processes were trying to do and was blocked as a result of them being there?

Florian.