Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Dovecot mailing list
Hello,

I am trying to get rspam to learn what is spam and what is ham.

After following the IMAPSieve guide in the dovecot wiki
(https://wiki2.dovecot.org/HowTo/AntispamWithSieve), the mail.log shows
the following, whenever I move a mail from the Spam folder to my inbox
and vice versa:

sieve: Opening script 1 of 1 from `/usr/lib/dovecot/sieve/report-ham.sieve'
sieve: Loading script /usr/lib/dovecot/sieve/report-ham.sieve
sieve: Script binary /usr/lib/dovecot/sieve/report-ham.svbin
successfully loaded
sieve: binary save: not saving binary
/usr/lib/dovecot/sieve/report-ham.svbin, because it is already stored
sieve: Executing script from `/usr/lib/dovecot/sieve/report-ham.svbin'
sieve: action pipe: running program: sa-learn-ham.sh
Mailbox INBOX: Opened mail UID=2996 because: mail stream
Debug: waiting for program `/usr/lib/dovecot/sieve/sa-learn-ham.sh' to
finish after 0 msecs
Error: program `/usr/lib/dovecot/sieve/sa-learn-ham.sh' terminated
abnormally, signal 11
Error: sieve: pipe action: failed to pipe message to program
`sa-learn-ham.sh': refer to server log for more information. [2019-11-17
00:39:33]
Error: sieve: Execution of script
/usr/lib/dovecot/sieve/report-ham.sieve failed


My relevant dovecot configuration:

plugin {
   imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve
   imapsieve_mailbox1_causes = COPY
   imapsieve_mailbox1_name = Spam
   imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
   imapsieve_mailbox2_causes = COPY
   imapsieve_mailbox2_from = Spam
   imapsieve_mailbox2_name = *

   sieve =
file:/var/vmail/sieve/%d/%n/scripts;active=/var/vmail/sieve/%d/%n/active-script.sieve
   sieve_before = /var/vmail/sieve/global/spam-global.sieve
   sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
   sieve_pipe_bin_dir = /usr/lib/dovecot/sieve
   sieve_plugins = sieve_imapsieve sieve_extprograms
}

$ dovecot --version
2.2.27 (c0f36b0)

$ ls -l /usr/lib/dovecot/sieve/
total 24
-rw-r--r-- 1 root root 314 Nov 17 00:25 report-ham.sieve
-rw-r--r-- 1 root root 448 Nov 17 00:30 report-ham.svbin
-rw-r--r-- 1 root root 199 Nov 17 00:25 report-spam.sieve
-rw-r--r-- 1 root root 354 Nov 17 00:30 report-spam.svbin
-rwxr-xr-x 1 root root  41 Nov 17 00:32 sa-learn-ham.sh
-rwxr-xr-x 1 root root  42 Nov 17 00:32 sa-learn-spam.sh

$ cat sa-learn-ham.sh
#!/bin/sh
exec /usr/bin/rspamc learn_ham


When I execute the script manually, it seems to work:
$ cat [some-mail] | /usr/lib/dovecot/sieve/sa-learn-ham.sh
Results for file: stdin (0.011 seconds)
success = true;
filename = "stdin";
scan_time = 0.012000;

Again, with the same mail:
$ cat [some-mail] | /usr/lib/dovecot/sieve/sa-learn-ham.sh
Results for file: stdin (0.000 seconds)
error = "<some-mail> has been already learned as ham, ignore it";
filename = "stdin";
scan_time = 0.0;

Any ideas what causes the abnormal temrination of the script?

Regards,
Jens
Reply | Threaded
Open this post in threaded view
|

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Dovecot mailing list
Jens Rey via dovecot skrev den 2019-11-17 00:51:

> $ cat sa-learn-ham.sh
> #!/bin/sh
> exec /usr/bin/rspamc learn_ham

rspamc is not spamc

is rspamd installed ?, is spamassassn installed ?

make sa-laern-ham.sh change name to rspamd-learn-ham.sh and addopt sieve
to that name change

i cant se other error on what you have
Reply | Threaded
Open this post in threaded view
|

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Dovecot mailing list
> make sa-laern-ham.sh change name to rspamd-learn-ham.sh and addopt sieve
> to that name change

I thought about the same thing while following the wiki guide. But in
the end it's just the name of a bash script. Now I changed it as you
proposed, since it just makes more sense, but the error remains:

Error: program `/usr/lib/dovecot/sieve/rspamd-learn-ham.sh' terminated
abnormally, signal 11
Error: sieve: pipe action: failed to pipe message to program
`rspamd-learn-ham.sh': refer to server log for more information.
[2019-11-17 08:35:58]
Error: sieve: Execution of script
/usr/lib/dovecot/sieve/report-ham.sieve failed

Reply | Threaded
Open this post in threaded view
|

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Dovecot mailing list
Since I want to avoid debugging old and already fixed bugs, I updated to

# dovecot --version
2.3.8 (9df20d2db)

Now the debug log is a bit more chatty, but the error remains:

Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Created
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass
environment: USER=[hidden email]
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass
environment: HOME=/var/vmail/mailboxes/domain.tld/user
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass
environment: HOST=mail.domain.tld
Debug: Mailbox INBOX: UID 22039: Opened mail because: mail stream
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh:
Establishing connection
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked
child process
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
Connected to program
Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
write(program stdin) failed: Broken pipe
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
Failed to run program
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
Disconnected
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
Child process ended
Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
Terminated abnormally with signal 11
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
Destroy
Error: sieve: pipe action: failed to pipe message to program
`rspamd-learn-ham.sh': refer to server log for more information.
[2019-11-17 19:32:40]
sieve: left message in mailbox 'INBOX'
Error: sieve: Execution of script
/usr/lib/dovecot/sieve/report-ham.sieve failed
--

E-Mail:    [hidden email]
Tel.:      +49 211 43636543
Mobil:     +49 162 9706249
Anschrift: Adolf-Klarenbach-Straße 16, 40589 Düsseldorf
Reply | Threaded
Open this post in threaded view
|

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Dovecot mailing list


On 17/11/2019 19:40, Jens Rey via dovecot wrote:

> Since I want to avoid debugging old and already fixed bugs, I updated to
>
> # dovecot --version
> 2.3.8 (9df20d2db)
>
> Now the debug log is a bit more chatty, but the error remains:
>
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Created
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass
> environment: USER=[hidden email]
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass
> environment: HOME=/var/vmail/mailboxes/domain.tld/user
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass
> environment: HOST=mail.domain.tld
> Debug: Mailbox INBOX: UID 22039: Opened mail because: mail stream
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh:
> Establishing connection
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked
> child process
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
> Connected to program
> Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
> write(program stdin) failed: Broken pipe
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
> Failed to run program
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
> Disconnected
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
> Child process ended
> Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
> Terminated abnormally with signal 11
> Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891):
> Destroy
> Error: sieve: pipe action: failed to pipe message to program
> `rspamd-learn-ham.sh': refer to server log for more information.
> [2019-11-17 19:32:40]
> sieve: left message in mailbox 'INBOX'
> Error: sieve: Execution of script
> /usr/lib/dovecot/sieve/report-ham.sieve failed

Dovecot forks a child process to execute the program. Subsequently, at
some point, the child process fails with a segmentation fault. Granted,
that could still be happening in Dovecot code, but since the child
process quickly executes the actual program (the shell in this case), it
is more likely that this problem emerges outside Dovecot.

You can attach GDB to imap with follow-fork-mode set to 'child', so you
can maybe see what causes the segfault.

Regards,

Stephan.


Reply | Threaded
Open this post in threaded view
|

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Dovecot mailing list
Hi,

> You can attach GDB to imap with follow-fork-mode set to 'child', so you
> can maybe see what causes the segfault.

now I am confused. Hopefully, I used GDB correctly, but as far as I can
see, everything seemed to work.

The only difference I can see is that I executed the whole thing as
root, but I would hope doveadm uses the correct user (vmail). At least
on the file system all emails still belong to vmail, so I guess that works.

For a quick reference, here is what I executed:

root@mail:~# gdb --args doveadm exec imap -u user@domain
...
(gdb) set follow-fork-mode child
(gdb) r
. SELECT Spam
. MOVE 1 Inbox

And the (imho) relevant output:

imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Establishing connection
[New process 13335]
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked child process
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Connected to
program
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
process 13335 is executing new program: /bin/dash
process 13335 is executing new program: /usr/bin/rspamc
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffee7ff700 (LWP 13338)]
[New Thread 0x7fffebffe700 (LWP 13339)]
[New Thread 0x7fffe97fd700 (LWP 13340)]
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished
streaming payload to program
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished input
to program
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Disconnected
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Waiting for
program to finish after 886 msecs
[Thread 0x7fffe97fd700 (LWP 13340) exited]
[Thread 0x7fffebffe700 (LWP 13339) exited]
[Thread 0x7ffff7fd1140 (LWP 13335) exited]
[Inferior 2 (process 13335) exited normally]
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Child process ended
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Destroy
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Info: sieve: pipe
action: piped message to program `rspamd-learn-ham.sh'


There is more output, lots more, but we are talking about spam after
all... If it is needed, please say so and I will provide it.

Regards,
Jens
Reply | Threaded
Open this post in threaded view
|

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Dovecot mailing list


Op 20-11-2019 om 8:50 schreef Jens Rey via dovecot:

> Hi,
>
>> You can attach GDB to imap with follow-fork-mode set to 'child', so
>> you can maybe see what causes the segfault.
>
> now I am confused. Hopefully, I used GDB correctly, but as far as I
> can see, everything seemed to work.
>
> The only difference I can see is that I executed the whole thing as
> root, but I would hope doveadm uses the correct user (vmail). At least
> on the file system all emails still belong to vmail, so I guess that
> works.
>
> For a quick reference, here is what I executed:
>
> root@mail:~# gdb --args doveadm exec imap -u user@domain
> ...
> (gdb) set follow-fork-mode child
> (gdb) r
> . SELECT Spam
> . MOVE 1 Inbox
>
> And the (imho) relevant output:
>
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Establishing connection
> [New process 13335]
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked child process
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Connected to
> program
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library
> "/lib/x86_64-linux-gnu/libthread_db.so.1".
> process 13335 is executing new program: /bin/dash
> process 13335 is executing new program: /usr/bin/rspamc
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library
> "/lib/x86_64-linux-gnu/libthread_db.so.1".
> [New Thread 0x7fffee7ff700 (LWP 13338)]
> [New Thread 0x7fffebffe700 (LWP 13339)]
> [New Thread 0x7fffe97fd700 (LWP 13340)]
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished
> streaming payload to program
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished
> input to program
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Disconnected
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Waiting for
> program to finish after 886 msecs
> [Thread 0x7fffe97fd700 (LWP 13340) exited]
> [Thread 0x7fffebffe700 (LWP 13339) exited]
> [Thread 0x7ffff7fd1140 (LWP 13335) exited]
> [Inferior 2 (process 13335) exited normally]
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Child process
> ended
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program
> exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Destroy
> imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Info: sieve: pipe
> action: piped message to program `rspamd-learn-ham.sh'
>
>
> There is more output, lots more, but we are talking about spam after
> all... If it is needed, please say so and I will provide it.

Hmm, so now it suddenly works. So, why would it not fail when run in GDB?

>
> Regards,
> Jens

Reply | Threaded
Open this post in threaded view
|

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

Dovecot mailing list

> Hmm, so now it suddenly works. So, why would it not fail when run in GDB?

That would be my question as well. I have no idea except from having
executed gdb as root and dovecot running as user vmail.