zlib plugin producing errors on 2.3.0

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

zlib plugin producing errors on 2.3.0

Adam Weinberger
Hello,

I use the zlib and imap_zlib plugins on FreeBSD. As of 2.3.0, my logs
are producing these errors every so often, but AFAICT the messages
themselves aren't getting corrupted.

Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0)
Fatal: master: service(imap): child 80128 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes })
Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0)
Fatal: master: service(imap): child 80266 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes })

They always come in pairs like that. Following is my doveconf. Let
me know what else I can provide here. Thanks!

# Adam


# 2.3.0 (c8b89eb): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.0 (d68c23a1)
# OS: FreeBSD 11.1-RELEASE-p6 amd64  nullfs
auth_mechanisms = plain login
first_valid_gid = 1021
first_valid_uid = 1021
last_valid_gid = 1022
last_valid_uid = 1022
listen = imap.jail.apnoea.adamw.org
mail_location = mdbox:/mail/%u/mail
mail_plugins = " zlib virtual fts fts_lucene"
mail_prefetch_count = 5
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox FreeBSD {
    autoexpunge = 17 weeks
  }
  mailbox FreeBSD/TodaysCommits {
    autoexpunge = 2 days
  }
  mailbox FreeBSD/automation {
    autoexpunge = 1 days
  }
  mailbox FreeBSD/portmgr {
    autoexpunge = 26 weeks
  }
  mailbox FreeBSD/ports {
    autoexpunge = 12 weeks
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    autoexpunge = 30 days
    special_use = \Trash
  }
  mailbox spam/probably {
    autoexpunge = 30 days
  }
  mailbox spam/totally {
    autoexpunge = 5 days
  }
  prefix =
  separator = /
}
passdb {
  args = scheme=BLF-CRYPT username_format=%u /path/to/userdb.passwd
  driver = passwd-file
}
plugin {
  fts = lucene
  fts_autoindex = yes
  fts_autoindex_max_recent_msgs = 25
  fts_lucene = whitespace_chars=@
  sieve = file:/scripts/sieve/%u.sieve;bindir=/mail/%u/sieve/
  sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.filter +editheader
  sieve_filter_bin_dir = /scripts/sieve/filter
  sieve_pipe_bin_dir = /scripts/sieve/pipe
  sieve_plugins = sieve_extprograms
}
postmaster_address = postmaster@...
protocols = imap lmtp
service imap-login {
  inet_listener imaps {
    port = 0
  }
}
service lmtp {
  inet_listener lmtp {
    port = 24
  }
}
ssl = required
ssl_cert = </path/to/fullchain.pem
ssl_cipher_list = TLSv1.2+AEAD:!aNULL
ssl_dh =  # hidden, use -P to show it
ssl_key =  # hidden, use -P to show it
ssl_min_protocol = TLSv1.2
ssl_prefer_server_ciphers = yes
userdb {
  args = username_format=%u /path/to/userdb.passwd
  driver = passwd-file
}
protocol lmtp {
  mail_plugins = " zlib virtual fts fts_lucene sieve"
}
protocol imap {
  mail_max_userip_connections = 20
  mail_plugins = " zlib virtual fts fts_lucene imap_zlib"
}
protocol lda {
  mail_plugins = " zlib virtual fts fts_lucene sieve"
}


--
Adam Weinberger
[hidden email]
http://www.adamw.org
Reply | Threaded
Open this post in threaded view
|

Re: zlib plugin producing errors on 2.3.0

Florian Pritz
On 24.12.2017 15:58, Adam Weinberger wrote:

> Hello,
>
> I use the zlib and imap_zlib plugins on FreeBSD. As of 2.3.0, my logs
> are producing these errors every so often, but AFAICT the messages
> themselves aren't getting corrupted.
>
> Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0)
> Fatal: master: service(imap): child 80128 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes })
> Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0)
> Fatal: master: service(imap): child 80266 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes })
I'm also seeing these errors, but I actually use gz compression. I'm
running 2.3.0 on Arch Linux. Config and logs are below.

Do you need any more information or can you reproduce this?

Florian


log:

> Jan  5 17:37:38 karif dovecot[28075]: imap(xxx)<29587><u2BR4gdi6d6kJ9hk>: Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0)
> Jan  5 17:37:38 karif dovecot[28075]: imap(xxx)<29587><u2BR4gdi6d6kJ9hk>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xc4567) [0x7fe8af7be567] -> /usr/lib/dovecot/libdovecot.so.0(+0xc462a) [0x7fe8af7be62a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7
> fe8af72ddf9] -> /usr/lib/dovecot/modules/lib20_zlib_plugin.so(+0x757b) [0x7fe8ae91a57b] -> /usr/lib/dovecot/libdovecot.so.0(+0xeb686) [0x7fe8af7e5686] -> dovecot/imap(+0x18bcd) [0x56168ab2fbcd] -> dovecot/imap(+0x11755) [0x56168ab28755] -> /usr/lib/dovecot/libdovecot.so.0(
> io_loop_call_io+0x6b) [0x7fe8af7d691b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x137) [0x7fe8af7d8247] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4d) [0x7fe8af7d6a1d] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x39) [0x7fe8af7d6
> c39] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x14) [0x7fe8af7515a4] -> dovecot/imap(main+0x357) [0x56168ab23cc7] -> /usr/lib/libc.so.6(__libc_start_main+0xea) [0x7fe8af362f4a] -> dovecot/imap(_start+0x2a) [0x56168ab23eba]
> Jan  5 17:37:38 karif dovecot[28075]: imap(xxx)<29587><u2BR4gdi6d6kJ9hk>: Fatal: master: service(imap): child 29587 killed with signal 6 (core not dumped)

config:

> # 2.3.0 (c8b89eb): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.0 (d68c23a1)
> # OS: Linux 4.14.6-1-ARCH x86_64 Arch Linux
> auth_mechanisms = plain login
> auth_username_format = %Ln
> imap_id_log = *
> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k
> mail_location = mdbox:~/.mdbox
> mail_plugins = " zlib notify mail_log quota"
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext
> mdbox_rotate_size = 20 M
> namespace inbox {
>   hidden = no
>   inbox = yes
>   list = yes
>   location =
>   mailbox Drafts {
>     auto = subscribe
>     special_use = \Drafts
>   }
>   mailbox Junk {
>     auto = subscribe
>     special_use = \Junk
>   }
>   mailbox Sent {
>     auto = subscribe
>     special_use = \Sent
>   }
>   mailbox Trash {
>     auto = subscribe
>     special_use = \Trash
>   }
>   prefix =
>   separator = .
>   type = private
> }
> passdb {
>   driver = pam
> }
> plugin {
>   mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
>   mail_log_fields = uid box msgid size
>   quota = count:User quota
>   quota_status_nouser = DUNNO
>   quota_status_overquota = 552 5.2.2 Mailbox is full
>   quota_status_success = DUNNO
>   quota_vsizes = yes
>   sieve = ~/.dovecot.sieve
>   sieve_dir = ~/.sieve
>   sieve_global_dir = /etc/dovecot/sieve/global/
>   sieve_global_path = /etc/dovecot/sieve/default.sieve
>   zlib_save = gz
>   zlib_save_level = 9
> }
> protocols = imap pop3 sieve lmtp
> service auth {
>   unix_listener auth-client {
>     group = postfix
>     mode = 0660
>     user = postfix
>   }
>   user = root
> }
> service imap-login {
>   process_limit = 400
>   process_min_avail = 5
> }
> service lmtp {
>   drop_priv_before_exec = yes
>   unix_listener /var/spool/postfix/private/dovecot-lmtp {
>     group = postfix
>     mode = 0660
>     user = postfix
>   }
> }
> service managesieve-login {
>   inet_listener sieve-obsolete {
>     port = 2000
>   }
> }
> ssl_cert = </etc/letsencrypt/live/mail.server-speed.net/fullchain.pem
> ssl_cipher_list = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
> ssl_dh =  # hidden, use -P to show it
> ssl_key =  # hidden, use -P to show it
> ssl_options = no_compression
> ssl_prefer_server_ciphers = yes
> userdb {
>   driver = passwd
> }
> protocol imap {
>   imap_client_workarounds = tb-extra-mailbox-sep
>   mail_max_userip_connections = 30
>   mail_plugins = " zlib notify mail_log quota imap_zlib"
> }
> protocol lda {
>   mail_plugins = " zlib notify mail_log quota sieve"
>   postmaster_address = [hidden email]
> }
> protocol lmtp {
>   mail_plugins = " zlib notify mail_log quota sieve"
>   postmaster_address = [hidden email]
> }
> protocol sieve {
>   managesieve_logout_format = bytes ( in=%i : out=%o )
> }


signature.asc (875 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: zlib plugin producing errors on 2.3.0

Uppenbrink
In reply to this post by Adam Weinberger
On 24.12.2017 15:58, Adam Weinberger wrote:

> Hello,
>
> I use the zlib and imap_zlib plugins on FreeBSD. As of 2.3.0, my logs
> are producing these errors every so often, but AFAICT the messages
> themselves aren't getting corrupted.
>
> Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion
> failed: (zstream->ostream.finished ||
> zstream->ostream.ostream.stream_errno != 0)
> Fatal: master: service(imap): child 80128 killed with signal 6 (core
> not dumped - set service imap { drop_priv_before_exec=yes })
> Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion
> failed: (zstream->ostream.finished ||
> zstream->ostream.ostream.stream_errno != 0)
> Fatal: master: service(imap): child 80266 killed with signal 6 (core
> not dumped - set service imap { drop_priv_before_exec=yes })
>
> They always come in pairs like that. Following is my doveconf. Let
> me know what else I can provide here. Thanks!

I had this errors in my logs, too.

It happens only to users who using K-9 Mail on Android. They all had a
setting enabled called "Use compression on network: Mobile, Wi-Fi,
Other" in "Incoming server settings". I didn't looked for other imap
clients, because without this setting enabled the errors vanished.

But it is likely that there are other imap clients who try to compress
the transfer.
No stored mails are corrupted, it is only the connection somehow.

Carsten
Reply | Threaded
Open this post in threaded view
|

Re: zlib plugin producing errors on 2.3.0

Timo Sirainen
On 5 Jan 2018, at 18.33, Carsten Uppenbrink <[hidden email]> wrote:

>
> On 24.12.2017 15:58, Adam Weinberger wrote:
>> Hello,
>> I use the zlib and imap_zlib plugins on FreeBSD. As of 2.3.0, my logs
>> are producing these errors every so often, but AFAICT the messages
>> themselves aren't getting corrupted.
>> Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion
>> failed: (zstream->ostream.finished ||
>> zstream->ostream.ostream.stream_errno != 0)
>> Fatal: master: service(imap): child 80128 killed with signal 6 (core
>> not dumped - set service imap { drop_priv_before_exec=yes })
>> Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion
>> failed: (zstream->ostream.finished ||
>> zstream->ostream.ostream.stream_errno != 0)
>> Fatal: master: service(imap): child 80266 killed with signal 6 (core
>> not dumped - set service imap { drop_priv_before_exec=yes })
>> They always come in pairs like that. Following is my doveconf. Let
>> me know what else I can provide here. Thanks!
>
> I had this errors in my logs, too.
>
> It happens only to users who using K-9 Mail on Android. They all had a setting enabled called "Use compression on network: Mobile, Wi-Fi, Other" in "Incoming server settings". I didn't looked for other imap clients, because without this setting enabled the errors vanished.
>
> But it is likely that there are other imap clients who try to compress the transfer.
> No stored mails are corrupted, it is only the connection somehow.

Oh, it's the imap_zlib plugin / IMAP COMPRESS extension that is crashing. Looks like it happens every time when COMPRESS-enabled client disconnects, so it's probably not visible to clients.

Reply | Threaded
Open this post in threaded view
|

Re: zlib plugin producing errors on 2.3.0

Aki Tuomi-2


On 06.01.2018 20:54, Timo Sirainen wrote:

> On 5 Jan 2018, at 18.33, Carsten Uppenbrink <[hidden email]> wrote:
>> On 24.12.2017 15:58, Adam Weinberger wrote:
>>> Hello,
>>> I use the zlib and imap_zlib plugins on FreeBSD. As of 2.3.0, my logs
>>> are producing these errors every so often, but AFAICT the messages
>>> themselves aren't getting corrupted.
>>> Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion
>>> failed: (zstream->ostream.finished ||
>>> zstream->ostream.ostream.stream_errno != 0)
>>> Fatal: master: service(imap): child 80128 killed with signal 6 (core
>>> not dumped - set service imap { drop_priv_before_exec=yes })
>>> Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion
>>> failed: (zstream->ostream.finished ||
>>> zstream->ostream.ostream.stream_errno != 0)
>>> Fatal: master: service(imap): child 80266 killed with signal 6 (core
>>> not dumped - set service imap { drop_priv_before_exec=yes })
>>> They always come in pairs like that. Following is my doveconf. Let
>>> me know what else I can provide here. Thanks!
>> I had this errors in my logs, too.
>>
>> It happens only to users who using K-9 Mail on Android. They all had a setting enabled called "Use compression on network: Mobile, Wi-Fi, Other" in "Incoming server settings". I didn't looked for other imap clients, because without this setting enabled the errors vanished.
>>
>> But it is likely that there are other imap clients who try to compress the transfer.
>> No stored mails are corrupted, it is only the connection somehow.
> Oh, it's the imap_zlib plugin / IMAP COMPRESS extension that is crashing. Looks like it happens every time when COMPRESS-enabled client disconnects, so it's probably not visible to clients.
>

Can you try if this
https://github.com/dovecot/core/commit/23da0fa1b30cc11bcc1d467674a0950c527e9ff1.patch
fixes your issue?

Aki
Reply | Threaded
Open this post in threaded view
|

Re: zlib plugin producing errors on 2.3.0

Uppenbrink
On 09.01.2018 11:22, Aki Tuomi wrote:
> Can you try if this
> https://github.com/dovecot/core/commit/23da0fa1b30cc11bcc1d467674a0950c527e9ff1.patch
> fixes your issue?

The panics are gone, it seems this patch fixes the issue. I see in my
logs normal login and connection closed messages.
I applied this patch to dovecot v2.3.0 and tested it with the Android
K-9 Mail App. I used the option "Use compression on network", which
seems to be a default for a new connection.

Carsten