v2.3.0 release candidate released

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

v2.3.0 release candidate released

Timo Sirainen
https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig 

It's finally time for v2.3 release branch! There are several new and exciting features in it. I'm especially happy about the new logging and statistics code, which will allow us to generate statistics for just about everything. We didn't have time to implement everything we wanted for them yet, and there especially aren't all that many logging events yet that can be used for statistics. We'll implement those to v2.3.1, which might also mean that some of the APIs might still change in v2.3.1 if that's required.

We also have new lib-smtp server code, which was used to implement SMTP submission server and do a partial rewrite for LMTP server. Please test these before v2.3.0 to make sure we don't have any bad bugs left!

BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1.

Some of the larger changes:

 * Various setting changes, see https://wiki2.dovecot.org/Upgrading/2.3
 * Logging rewrite started: Logging is now based on hierarchical events.
   This makes it possible to do various things, like: 1) giving
   consistent log prefixes, 2) enabling debug logging with finer
   granularity, 3) provide logs in more machine readable formats
   (e.g. json). Everything isn't finished yet, especially a lot of the
   old logging code still needs to be translated to the new way.
 * Statistics rewrite started: Stats are now based on (log) events.
   It's possible to gather statistics about any event that is logged.
   See http://wiki2.dovecot.org/Statistics for details
 * ssl_dh setting replaces the old generated ssl-parameters.dat
 * IMAP: When BINARY FETCH finds a broken mails, send [PARSE] error
   instead of [UNKNOWNCTE]
 * Linux: core dumping via PR_SET_DUMPABLE is no longer enabled by
   default due to potential security reasons (found by cPanel Security
   Team).

 + Added support for SMTP submission proxy server, which includes
   support for BURL and CHUNKING extension.
 + LMTP rewrite. Supports now CHUNKING extension and mixing of
   local/proxy recipients.
 + auth: Support libsodium to add support for ARGON2I and ARGON2ID
   password schemes.
 + auth: Support BLF-CRYPT password scheme in all platforms
 + auth: Added LUA scripting support for passdb/userdb.
   See https://wiki2.dovecot.org/AuthDatabase/Lua
 - Input streams are more reliable now when there are errors or when
   the maximum buffer size is reached. Previously in some situations
   this could have caused Dovecot to try to read already freed memory.
 - Output streams weren't previously handling failures when writing a
   trailer at the end of the stream. This mainly affected encrypt and
   zlib compress ostreams, which could have silently written truncated
   files if the last write happened to fail (which shouldn't normally
   have ever happened).
 - virtual plugin: Fixed panic when fetching mails from virtual
   mailboxes with IMAP BINARY extension.
 - Many other smaller fixes

Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

James-4
On 18/12/2017 15:23, Timo Sirainen wrote:

> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz

"smtp-server-cmd-helo.c", line 140: void function cannot return value
"smtp-server-cmd-helo.c", line 148: void function cannot return value


--- ../original/src/lib-smtp/smtp-server-cmd-helo.c 2017-12-18
15:04:05.000000000 +0000
+++ src/lib-smtp/smtp-server-cmd-helo.c 2017-12-18 16:39:09.643332687 +0000
@@ -137,7 +137,7 @@
  {
  /* ehlo = "EHLO" SP ( Domain / address-literal ) CRLF */

- return smtp_server_cmd_helo_run(cmd, params, FALSE);
+ smtp_server_cmd_helo_run(cmd, params, FALSE);
  }

  void smtp_server_cmd_helo(struct smtp_server_cmd_ctx *cmd,
@@ -145,5 +145,5 @@
  {
  /* helo = "HELO" SP Domain CRLF */

- return smtp_server_cmd_helo_run(cmd, params, TRUE);
+ smtp_server_cmd_helo_run(cmd, params, TRUE);
  }





pigeonhole compile complained so I stopped at just compiling dovecot.  I
assume an updated pigeonhole is coming.

"edit-mail.c", line 235: prototype mismatch: 2 args passed, 3 expected
"edit-mail.c", line 1662: prototype mismatch: 2 args passed, 3 expected
"edit-mail.c", line 1671: undefined struct/union member:
set_cache_corrupted_reason
"edit-mail.c", line 1671: function designator is not of function type
"edit-mail.c", line 1673: cannot recover from previous errors


Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Stephan Bosch-2
Op 12/18/2017 om 6:16 PM schreef James:
> On 18/12/2017 15:23, Timo Sirainen wrote:
>
>> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
>
> "smtp-server-cmd-helo.c", line 140: void function cannot return value
> "smtp-server-cmd-helo.c", line 148: void function cannot return value

Just wondering: which system/compiler is giving these errors?

> pigeonhole compile complained so I stopped at just compiling dovecot. 
> I assume an updated pigeonhole is coming.
>
> "edit-mail.c", line 235: prototype mismatch: 2 args passed, 3 expected
> "edit-mail.c", line 1662: prototype mismatch: 2 args passed, 3 expected
> "edit-mail.c", line 1671: undefined struct/union member:
> set_cache_corrupted_reason
> "edit-mail.c", line 1671: function designator is not of function type
> "edit-mail.c", line 1673: cannot recover from previous errors

Indeed.

Regards,

Stephan.


Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

James-4
On 18/12/2017 18:15, Stephan Bosch wrote:
> Op 12/18/2017 om 6:16 PM schreef James:

>> "smtp-server-cmd-helo.c", line 140: void function cannot return value
>> "smtp-server-cmd-helo.c", line 148: void function cannot return value
>
> Just wondering: which system/compiler is giving these errors?

Sun/Oracle Studio compiler on Solaris.

$ cc -V
cc: Studio 12.5 Sun C 5.14 SunOS_i386 2016/05/31


James.

Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Odhiambo Washington-4
In reply to this post by Timo Sirainen
On 18 December 2017 at 18:23, Timo Sirainen <[hidden email]> wrote:

> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig
>
>
I decided to test the compile on my servers which are all
running dovecot-2.2.33.2:

(1) It was successfully compiled on FreeBSD 11.0-STABLE amd64.

(2) I will try tomorrow on FreeBSD 9-3-STABLE and report too.

(3) However, I tested a compile on FreeBSD 8.4-STABLE i386 and it failed
like this:


mv -f .deps/test_auth_cache-test-auth-cache.Tpo
.deps/test_auth_cache-test-auth-cache.Po
gcc -DHAVE_CONFIG_H -I. -I../..  -I../../src/lib -I../../src/lib-auth
-I../../src/lib-test -I../../src/lib-dict -I../../src/lib-dns
-I../../src/lib-http -I../../src/lib-sql -I../../src/lib-settings
-I../../src/lib-old-stats -I../../src/lib-ntlm -I../../src/lib-otp
-I../../src/lib-master -I../../src/lib-oauth2 -I../../src/lib-ssl-iostream
-I../../src/lib-lua
-DAUTH_MODULE_DIR=\""/opt/dovecot2.2/lib/dovecot/auth"\"
-DPKG_LIBEXECDIR=\""/opt/dovecot2.2/libexec/dovecot"\"
-DPKG_RUNDIR=\""/opt/dovecot2.2/var/run/dovecot"\"   -fPIE -DPIE
 -std=gnu99 -g -O2 -fstack-protector -D_FORTIFY_SOURCE=2 -Wall -W
-Wmissing-prototypes -Wmissing-declarations -Wpointer-arith
-Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime
-Wstrict-aliasing=2 -I/usr/local/include   -MT auth-main.o -MD -MP -MF
.deps/auth-main.Tpo -c -o auth-main.o `test -f 'main.c' || echo './'`main.c
mv -f .deps/auth-main.Tpo .deps/auth-main.Po
gmake[3]: *** No rule to make target '-L/usr/local/lib', needed by 'auth'.
Stop.
gmake[3]: Leaving directory
'/usr/home/wash/Tools/Dovecot/2.3/dovecot-2.3.0.rc1/src/auth'
Makefile:562: recipe for target 'all-recursive' failed
gmake[2]: *** [all-recursive] Error 1
gmake[2]: Leaving directory
'/usr/home/wash/Tools/Dovecot/2.3/dovecot-2.3.0.rc1/src'
Makefile:680: recipe for target 'all-recursive' failed
gmake[1]: *** [all-recursive] Error 1
gmake[1]: Leaving directory
'/usr/home/wash/Tools/Dovecot/2.3/dovecot-2.3.0.rc1'
Makefile:524: recipe for target 'all' failed
gmake: *** [all] Error 2
(21:56:56 <~/Tools/Dovecot/2.3/dovecot-2.3.0.rc1>


PS: On all the servers, I compile using the same options I have used
with dovecot-2.2.33.2.


--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Michael Marley
In reply to this post by Timo Sirainen
On 2017-12-18 10:23, Timo Sirainen wrote:

> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig 
>
> It's finally time for v2.3 release branch! There are several new and exciting features in it. I'm especially happy about the new logging and statistics code, which will allow us to generate statistics for just about everything. We didn't have time to implement everything we wanted for them yet, and there especially aren't all that many logging events yet that can be used for statistics. We'll implement those to v2.3.1, which might also mean that some of the APIs might still change in v2.3.1 if that's required.
>
> We also have new lib-smtp server code, which was used to implement SMTP submission server and do a partial rewrite for LMTP server. Please test these before v2.3.0 to make sure we don't have any bad bugs left!
>
> BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1.
>
> Some of the larger changes:
>
> * Various setting changes, see https://wiki2.dovecot.org/Upgrading/2.3
> * Logging rewrite started: Logging is now based on hierarchical events.
> This makes it possible to do various things, like: 1) giving
> consistent log prefixes, 2) enabling debug logging with finer
> granularity, 3) provide logs in more machine readable formats
> (e.g. json). Everything isn't finished yet, especially a lot of the
> old logging code still needs to be translated to the new way.
> * Statistics rewrite started: Stats are now based on (log) events.
> It's possible to gather statistics about any event that is logged.
> See http://wiki2.dovecot.org/Statistics for details
> * ssl_dh setting replaces the old generated ssl-parameters.dat
> * IMAP: When BINARY FETCH finds a broken mails, send [PARSE] error
> instead of [UNKNOWNCTE]
> * Linux: core dumping via PR_SET_DUMPABLE is no longer enabled by
> default due to potential security reasons (found by cPanel Security
> Team).
>
> + Added support for SMTP submission proxy server, which includes
> support for BURL and CHUNKING extension.
> + LMTP rewrite. Supports now CHUNKING extension and mixing of
> local/proxy recipients.
> + auth: Support libsodium to add support for ARGON2I and ARGON2ID
> password schemes.
> + auth: Support BLF-CRYPT password scheme in all platforms
> + auth: Added LUA scripting support for passdb/userdb.
> See https://wiki2.dovecot.org/AuthDatabase/Lua
> - Input streams are more reliable now when there are errors or when
> the maximum buffer size is reached. Previously in some situations
> this could have caused Dovecot to try to read already freed memory.
> - Output streams weren't previously handling failures when writing a
> trailer at the end of the stream. This mainly affected encrypt and
> zlib compress ostreams, which could have silently written truncated
> files if the last write happened to fail (which shouldn't normally
> have ever happened).
> - virtual plugin: Fixed panic when fetching mails from virtual
> mailboxes with IMAP BINARY extension.
> - Many other smaller fixes
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Goetz Schultz
In reply to this post by Odhiambo Washington-4
Hi

From reeBSD 9.3-RELEASE amd64:

Compiles ok
Installs ok
On startup 1: Error about new config settings (adjusted)
On startup 2: doveadm(root): Error:
net_connect_unix(/var/run/dovecot//stats-writer) failed: Connection refused

It seems to work so far. I get messages from fetchmail being delivered
to the mailbox, yet that stale error is there. Quick search did not
showed any results.

Not looked yet, but is there a guide for the ssl_min_version setting?
Seems only to wok when I set "SSLv3" which I really don't want.


Thanks and regards

  Goetz R. Schultz

On 18/12/17 19:04, Odhiambo Washington wrote:

> On 18 December 2017 at 18:23, Timo Sirainen <[hidden email]> wrote:
>
>> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
>> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig
>>
>>
> I decided to test the compile on my servers which are all
> running dovecot-2.2.33.2:
>
> (1) It was successfully compiled on FreeBSD 11.0-STABLE amd64.
>
> (2) I will try tomorrow on FreeBSD 9-3-STABLE and report too.
>
> (3) However, I tested a compile on FreeBSD 8.4-STABLE i386 and it failed
> like this:
>


signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Goetz Schultz
Found the guide .... Can you remove last entry please ... works with TLSv1.


Thanks and regards

  Goetz R. Schultz

On 18/12/17 20:05, Goetz Schultz wrote:

> Hi
>
> From reeBSD 9.3-RELEASE amd64:
>
> Compiles ok
> Installs ok
> On startup 1: Error about new config settings (adjusted)
> On startup 2: doveadm(root): Error:
> net_connect_unix(/var/run/dovecot//stats-writer) failed: Connection refused
>
> It seems to work so far. I get messages from fetchmail being delivered
> to the mailbox, yet that stale error is there. Quick search did not
> showed any results.
>
> Not looked yet, but is there a guide for the ssl_min_version setting?
> Seems only to wok when I set "SSLv3" which I really don't want.
>
>
> Thanks and regards
>
>   Goetz R. Schultz
>
> On 18/12/17 19:04, Odhiambo Washington wrote:
>> On 18 December 2017 at 18:23, Timo Sirainen <[hidden email]> wrote:
>>
>>> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
>>> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig
>>>
>>>
>> I decided to test the compile on my servers which are all
>> running dovecot-2.2.33.2:
>>
>> (1) It was successfully compiled on FreeBSD 11.0-STABLE amd64.
>>
>> (2) I will try tomorrow on FreeBSD 9-3-STABLE and report too.
>>
>> (3) However, I tested a compile on FreeBSD 8.4-STABLE i386 and it failed
>> like this:
>>
>


signature.asc (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Mark Moseley
In reply to this post by Timo Sirainen
On Mon, Dec 18, 2017 at 7:23 AM, Timo Sirainen <[hidden email]> wrote:

> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig
>
> It's finally time for v2.3 release branch! There are several new and
> exciting features in it. I'm especially happy about the new logging and
> statistics code, which will allow us to generate statistics for just about
> everything. We didn't have time to implement everything we wanted for them
> yet, and there especially aren't all that many logging events yet that can
> be used for statistics. We'll implement those to v2.3.1, which might also
> mean that some of the APIs might still change in v2.3.1 if that's required.
>
> We also have new lib-smtp server code, which was used to implement SMTP
> submission server and do a partial rewrite for LMTP server. Please test
> these before v2.3.0 to make sure we don't have any bad bugs left!
>
> BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1.
>
> Some of the larger changes:
>
>  * Various setting changes, see https://wiki2.dovecot.org/Upgrading/2.3
>  * Logging rewrite started: Logging is now based on hierarchical events.
>    This makes it possible to do various things, like: 1) giving
>    consistent log prefixes, 2) enabling debug logging with finer
>    granularity, 3) provide logs in more machine readable formats
>    (e.g. json). Everything isn't finished yet, especially a lot of the
>    old logging code still needs to be translated to the new way.
>  * Statistics rewrite started: Stats are now based on (log) events.
>    It's possible to gather statistics about any event that is logged.
>    See http://wiki2.dovecot.org/Statistics for details
>  * ssl_dh setting replaces the old generated ssl-parameters.dat
>  * IMAP: When BINARY FETCH finds a broken mails, send [PARSE] error
>    instead of [UNKNOWNCTE]
>  * Linux: core dumping via PR_SET_DUMPABLE is no longer enabled by
>    default due to potential security reasons (found by cPanel Security
>    Team).
>
>  + Added support for SMTP submission proxy server, which includes
>    support for BURL and CHUNKING extension.
>  + LMTP rewrite. Supports now CHUNKING extension and mixing of
>    local/proxy recipients.
>  + auth: Support libsodium to add support for ARGON2I and ARGON2ID
>    password schemes.
>  + auth: Support BLF-CRYPT password scheme in all platforms
>  + auth: Added LUA scripting support for passdb/userdb.
>    See https://wiki2.dovecot.org/AuthDatabase/Lua
>  - Input streams are more reliable now when there are errors or when
>    the maximum buffer size is reached. Previously in some situations
>    this could have caused Dovecot to try to read already freed memory.
>  - Output streams weren't previously handling failures when writing a
>    trailer at the end of the stream. This mainly affected encrypt and
>    zlib compress ostreams, which could have silently written truncated
>    files if the last write happened to fail (which shouldn't normally
>    have ever happened).
>  - virtual plugin: Fixed panic when fetching mails from virtual
>    mailboxes with IMAP BINARY extension.
>  - Many other smaller fixes
>
>

No issue compilng (and very very excited about this release, esp the Lua
code, which is already super useful).

I did have this one issue so far with the RC. I was previously using a git
checkout of ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 with some of the Lua
patches attached. That was working just fine (except for one thing I'll
mention below). I rolled the RC and got this (and I was actually testing
for the issue I had with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015):

# doveadm -D acl set -u [hidden email] INBOX user=[hidden email]
read  list
Debug: Loading modules from directory: /usr/lib/dovecot/modules
Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib20_virtual_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_lucene_plugin.so
Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined
symbol: expire_set_deinit (this is usually intentional, so just ignore this
message)
Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so
Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so
Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed:
/usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined
symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so
just ignore this message)
doveadm([hidden email])<37433><>: Debug: auth USER input:
[hidden email] uid=200 home=/m/mail//7/1/fe/test.com/test1-shared
gid=200 quota=maildir:User quota mail_location=maildir:/m/mail//7/1/fe/
test.com/test1-shared/maildir
doveadm([hidden email])<37433><>: Debug: Added userdb setting:
mail_location=maildir:/m/mail//7/1/fe/test.com/test1-shared/maildir
doveadm([hidden email])<37433><>: Debug: Added userdb setting:
plugin/quota=maildir:User quota
doveadm([hidden email]): Debug: Effective uid=200, gid=200,
home=/m/mail//7/1/fe/test.com/test1-shared
doveadm([hidden email]): Panic: file buffer.c: line 97
(buffer_check_limits): assertion failed: (buf->used <= buf->alloc)
doveadm([hidden email]): Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(+0xc5d7a) [0x6ba69ea77d7a] ->
/usr/lib/dovecot/libdovecot.so.0(+0xc5de9) [0x6ba69ea77de9] ->
/usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x6ba69e9ea521] ->
/usr/lib/dovecot/libdovecot.so.0(buffer_write+0x160) [0x6ba69ea72210] ->
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so(+0x43fe)
[0x6ba6997c33fe] ->
/usr/lib/dovecot/libdovecot-storage.so.0(hook_mail_user_created+0x20a)
[0x6ba69ed72a2a] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_user_init+0x220)
[0x6ba69ed78870] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_service_next_with_session_suffix+0x494)
[0x6ba69ed76314] -> doveadm(+0x296ff) [0x5bde3fb56ff] -> doveadm(+0x2a3ab)
[0x5bde3fb63ab] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d)
[0x5bde3fb71bd] -> doveadm(doveadm_cmd_run_ver2+0x575) [0x5bde3fc6c15] ->
doveadm(doveadm_cmd_try_run_ver2+0x37) [0x5bde3fc6c67] ->
doveadm(main+0x1d2) [0x5bde3fa5f52] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x6ba69e609830] ->
doveadm(_start+0x29) [0x5bde3fa6329]
Aborted


This same command was working with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015
earlier today -- this is run from command line history. The issue I had
noticed with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 (and was going to ask
about it today anyway) is that when I would use doveadm acl to add an ACL
with *just* 'read', dovecot would create the ACL file correctly, but using
a file-based acl_shared_dict, it would create a 0 byte acl_shared_dict
file. However using anything else for the ACL, e.g. 'read list', it would
create the acl_shared_dict file with what you'd expect. I don't know if
that's expected behavior or not (but couldn't find anything on google to
say so). So just to sum up: With just 'read', it'd create an empty
acl_shared_dict file; with 'read list' or anything else, it'd create a
working, non-empty acl_shared_dict file.

I was going to see if it was different in the RC but instead, there's just
the above error.

This is just a test rig, so I can try out whatever would be useful to
debugging. Happy to post doveconf -n, if helpful. The only config changes
from ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 to RC1 that I made were just
the things that needed to be done to let dovecot start up (removing some of
the stats stuff, getting rid of ssl_protocols, etc).
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Kenneth Porter
In reply to this post by James-4
On 12/18/2017 10:53 AM, James wrote:

> On 18/12/2017 18:15, Stephan Bosch wrote:
>> Op 12/18/2017 om 6:16 PM schreef James:
>
>>> "smtp-server-cmd-helo.c", line 140: void function cannot return value
>>> "smtp-server-cmd-helo.c", line 148: void function cannot return value
>>
>> Just wondering: which system/compiler is giving these errors?
>
> Sun/Oracle Studio compiler on Solaris.
>
> $ cc -V
> cc: Studio 12.5 Sun C 5.14 SunOS_i386 2016/05/31

This is valid syntax for C++14 but invalid for C11. See this answer:

https://stackoverflow.com/questions/35987493/return-void-type-in-c-and-c
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Stephan Bosch-2
Op 12/18/2017 om 10:59 PM schreef Kenneth Porter:

> On 12/18/2017 10:53 AM, James wrote:
>> On 18/12/2017 18:15, Stephan Bosch wrote:
>>> Op 12/18/2017 om 6:16 PM schreef James:
>>
>>>> "smtp-server-cmd-helo.c", line 140: void function cannot return value
>>>> "smtp-server-cmd-helo.c", line 148: void function cannot return value
>>>
>>> Just wondering: which system/compiler is giving these errors?
>>
>> Sun/Oracle Studio compiler on Solaris.
>>
>> $ cc -V
>> cc: Studio 12.5 Sun C 5.14 SunOS_i386 2016/05/31
>
> This is valid syntax for C++14 but invalid for C11. See this answer:
>
> https://stackoverflow.com/questions/35987493/return-void-type-in-c-and-c

Yes, I know why it is wrong. I was just wondering why our usual set of
compilers didn't at least warn about this stupidity. :)

Regards,

Stephan.
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Mark Moseley
In reply to this post by Mark Moseley
On Mon, Dec 18, 2017 at 1:16 PM, Mark Moseley <[hidden email]> wrote:

> On Mon, Dec 18, 2017 at 7:23 AM, Timo Sirainen <[hidden email]> wrote:
>
>> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
>> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig
>>
>> It's finally time for v2.3 release branch! There are several new and
>> exciting features in it. I'm especially happy about the new logging and
>> statistics code, which will allow us to generate statistics for just about
>> everything. We didn't have time to implement everything we wanted for them
>> yet, and there especially aren't all that many logging events yet that can
>> be used for statistics. We'll implement those to v2.3.1, which might also
>> mean that some of the APIs might still change in v2.3.1 if that's required.
>>
>> We also have new lib-smtp server code, which was used to implement SMTP
>> submission server and do a partial rewrite for LMTP server. Please test
>> these before v2.3.0 to make sure we don't have any bad bugs left!
>>
>> BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1.
>>
>> Some of the larger changes:
>>
>>  * Various setting changes, see https://wiki2.dovecot.org/Upgrading/2.3
>>  * Logging rewrite started: Logging is now based on hierarchical events.
>>    This makes it possible to do various things, like: 1) giving
>>    consistent log prefixes, 2) enabling debug logging with finer
>>    granularity, 3) provide logs in more machine readable formats
>>    (e.g. json). Everything isn't finished yet, especially a lot of the
>>    old logging code still needs to be translated to the new way.
>>  * Statistics rewrite started: Stats are now based on (log) events.
>>    It's possible to gather statistics about any event that is logged.
>>    See http://wiki2.dovecot.org/Statistics for details
>>  * ssl_dh setting replaces the old generated ssl-parameters.dat
>>  * IMAP: When BINARY FETCH finds a broken mails, send [PARSE] error
>>    instead of [UNKNOWNCTE]
>>  * Linux: core dumping via PR_SET_DUMPABLE is no longer enabled by
>>    default due to potential security reasons (found by cPanel Security
>>    Team).
>>
>>  + Added support for SMTP submission proxy server, which includes
>>    support for BURL and CHUNKING extension.
>>  + LMTP rewrite. Supports now CHUNKING extension and mixing of
>>    local/proxy recipients.
>>  + auth: Support libsodium to add support for ARGON2I and ARGON2ID
>>    password schemes.
>>  + auth: Support BLF-CRYPT password scheme in all platforms
>>  + auth: Added LUA scripting support for passdb/userdb.
>>    See https://wiki2.dovecot.org/AuthDatabase/Lua
>>  - Input streams are more reliable now when there are errors or when
>>    the maximum buffer size is reached. Previously in some situations
>>    this could have caused Dovecot to try to read already freed memory.
>>  - Output streams weren't previously handling failures when writing a
>>    trailer at the end of the stream. This mainly affected encrypt and
>>    zlib compress ostreams, which could have silently written truncated
>>    files if the last write happened to fail (which shouldn't normally
>>    have ever happened).
>>  - virtual plugin: Fixed panic when fetching mails from virtual
>>    mailboxes with IMAP BINARY extension.
>>  - Many other smaller fixes
>>
>>
>
> No issue compilng (and very very excited about this release, esp the Lua
> code, which is already super useful).
>
> I did have this one issue so far with the RC. I was previously using a git
> checkout of ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 with some of the Lua
> patches attached. That was working just fine (except for one thing I'll
> mention below). I rolled the RC and got this (and I was actually testing
> for the issue I had with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015):
>
> # doveadm -D acl set -u [hidden email] INBOX user=[hidden email]
> read  list
> Debug: Loading modules from directory: /usr/lib/dovecot/modules
> Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/
> lib02_lazy_expunge_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/lib20_virtual_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_lucene_plugin.so
> Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
> Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_
> plugin.so
> Debug: Skipping module doveadm_expire_plugin, because dlopen() failed:
> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so:
> undefined symbol: expire_set_deinit (this is usually intentional, so just
> ignore this message)
> Debug: Module loaded: /usr/lib/dovecot/modules/
> doveadm/lib10_doveadm_quota_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/
> doveadm/lib10_doveadm_sieve_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_
> lucene_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_
> plugin.so
> Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed:
> /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so:
> undefined symbol: mail_crypt_box_get_pvt_digests (this is usually
> intentional, so just ignore this message)
> doveadm([hidden email])<37433><>: Debug: auth USER input:
> [hidden email] uid=200 home=/m/mail//7/1/fe/test.com/test1-shared
> gid=200 quota=maildir:User quota mail_location=maildir:/m/mail//7/1/fe/
> test.com/test1-shared/maildir
> doveadm([hidden email])<37433><>: Debug: Added userdb setting:
> mail_location=maildir:/m/mail//7/1/fe/test.com/test1-shared/maildir
> doveadm([hidden email])<37433><>: Debug: Added userdb setting:
> plugin/quota=maildir:User quota
> doveadm([hidden email]): Debug: Effective uid=200, gid=200,
> home=/m/mail//7/1/fe/test.com/test1-shared
> doveadm([hidden email]): Panic: file buffer.c: line 97
> (buffer_check_limits): assertion failed: (buf->used <= buf->alloc)
> doveadm([hidden email]): Error: Raw backtrace:
> /usr/lib/dovecot/libdovecot.so.0(+0xc5d7a) [0x6ba69ea77d7a] ->
> /usr/lib/dovecot/libdovecot.so.0(+0xc5de9) [0x6ba69ea77de9] ->
> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x6ba69e9ea521] ->
> /usr/lib/dovecot/libdovecot.so.0(buffer_write+0x160) [0x6ba69ea72210] ->
> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so(+0x43fe)
> [0x6ba6997c33fe] -> /usr/lib/dovecot/libdovecot-
> storage.so.0(hook_mail_user_created+0x20a) [0x6ba69ed72a2a] ->
> /usr/lib/dovecot/libdovecot-storage.so.0(mail_user_init+0x220)
> [0x6ba69ed78870] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_
> service_next_with_session_suffix+0x494) [0x6ba69ed76314] ->
> doveadm(+0x296ff) [0x5bde3fb56ff] -> doveadm(+0x2a3ab) [0x5bde3fb63ab] ->
> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) [0x5bde3fb71bd] ->
> doveadm(doveadm_cmd_run_ver2+0x575) [0x5bde3fc6c15] ->
> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x5bde3fc6c67] ->
> doveadm(main+0x1d2) [0x5bde3fa5f52] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
> [0x6ba69e609830] -> doveadm(_start+0x29) [0x5bde3fa6329]
> Aborted
>
>
> This same command was working with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015
> earlier today -- this is run from command line history. The issue I had
> noticed with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 (and was going to
> ask about it today anyway) is that when I would use doveadm acl to add an
> ACL with *just* 'read', dovecot would create the ACL file correctly, but
> using a file-based acl_shared_dict, it would create a 0
> byte acl_shared_dict file. However using anything else for the ACL, e.g.
> 'read list', it would create the acl_shared_dict file with what you'd
> expect. I don't know if that's expected behavior or not (but couldn't find
> anything on google to say so). So just to sum up: With just 'read', it'd
> create an empty acl_shared_dict file; with 'read list' or anything else,
> it'd create a working, non-empty acl_shared_dict file.
>
> I was going to see if it was different in the RC but instead, there's just
> the above error.
>
> This is just a test rig, so I can try out whatever would be useful to
> debugging. Happy to post doveconf -n, if helpful. The only config changes
> from ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 to RC1 that I made were
> just the things that needed to be done to let dovecot start up (removing
> some of the stats stuff, getting rid of ssl_protocols, etc).
>


Sorry, the backtrace went away and the command completes successfully after
installing the 0.5.0rc1 pigeonhole.

The issue with the zero byte acl_shared_dict files from an acl set of just
'read' still persists. And the acl_shared_dict file is still created and
populated correctly with anything beyond just read. E.g. it's created and
populated if the only ACL is 'lookup'. I only started playing with shared
mailboxes in 2.3 so I don't have spot where I can test on 2.2. It's not
super critical since I'd always be using 'read write-seen' at least, but I
figured I'd bring it up.
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Timo Sirainen
In reply to this post by Mark Moseley
On 18 Dec 2017, at 23.16, Mark Moseley <[hidden email]> wrote:
>
> doveadm([hidden email]): Panic: file buffer.c: line 97
> (buffer_check_limits): assertion failed: (buf->used <= buf->alloc)
..
> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so(+0x43fe)
> [0x6ba6997c33fe] ->

Since the panic is coming from pigeonhole, did you recompile it also? And what version of it?

Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Mark Moseley
On Mon, Dec 18, 2017 at 2:32 PM, Timo Sirainen <[hidden email]> wrote:

> On 18 Dec 2017, at 23.16, Mark Moseley <[hidden email]> wrote:
> >
> > doveadm([hidden email]): Panic: file buffer.c: line 97
> > (buffer_check_limits): assertion failed: (buf->used <= buf->alloc)
> ..
> > /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so(+0x43fe)
> > [0x6ba6997c33fe] ->
>
> Since the panic is coming from pigeonhole, did you recompile it also? And
> what version of it?
>
>
The previous version (that was running happily with
ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015) was 0.4.21. I had compiled 0.4.21
against ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015. RC1 + 0.5.0rc1 stopped
backtracing on me and works ok (minus the 'read' thing I mentioned).
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Stephan Bosch-2
In reply to this post by Timo Sirainen
Op 12/18/2017 om 4:23 PM schreef Timo Sirainen:
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig 
>
> It's finally time for v2.3 release branch! There are several new and exciting features in it. I'm especially happy about the new logging and statistics code, which will allow us to generate statistics for just about everything. We didn't have time to implement everything we wanted for them yet, and there especially aren't all that many logging events yet that can be used for statistics. We'll implement those to v2.3.1, which might also mean that some of the APIs might still change in v2.3.1 if that's required.
>
> We also have new lib-smtp server code, which was used to implement SMTP submission server and do a partial rewrite for LMTP server. Please test these before v2.3.0 to make sure we don't have any bad bugs left!
>
> BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1.

Preliminary Xi Debian release packages are now available as follows:

deb http://xi.dovecot.fi/debian/ stable-rc-auto/dovecot-2.3 main

These are currently only available for stretch/amd64, since I still need to update the slave builders (there are source packages though).

Regards,

Stephan.

Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Eray Aslan-2
In reply to this post by Timo Sirainen
On Mon, Dec 18, 2017 at 05:23:28PM +0200, Timo Sirainen wrote:
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig 

Some source based distros like Gentoo already enable -D_FORTIFY_SOURCE=2
by default resulting in the following warnings during build:

<command-line>:0:0: warning: "_FORTIFY_SOURCE" redefined
<built-in>: note: this is the location of the previous definition

Please consider the attached patch to get rid of the warnings.

--
Eray

dovecot-2.3-fortify.patch (426 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Eray Aslan-2
In reply to this post by Timo Sirainen
On Mon, Dec 18, 2017 at 05:23:28PM +0200, Timo Sirainen wrote:
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
> https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig 

Please do not feed libraries into LDFLAGS.  It is wrong and breaks with
--as-needed:

[...]
checking for libsodium... yes
checking for crypto_pwhash_str_verify... no
configure: error: Can't build with libsodium: not found

Please see the attached patch for a possible solution.

Thank you
--
Eray

dovecot-2.3-sodium.patch (785 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Odhiambo Washington-4
What am I missing here:

OS = FreeBSD 8.4

Here is how it fails during `gmake`:

mv -f .deps/test_auth_cache-test-auth-cache.Tpo
.deps/test_auth_cache-test-auth-cache.Po
gcc -DHAVE_CONFIG_H -I. -I../..  -I../../src/lib -I../../src/lib-auth
-I../../src/lib-test -I../../src/lib-dict -I../../src/lib-dns
-I../../src/lib-http -I../../src/lib-sql -I../../src/lib-settings
-I../../src/lib-old-stats -I../../src/lib-ntlm -I../../src/lib-otp
-I../../src/lib-master -I../../src/lib-oauth2 -I../../src/lib-ssl-iostream
-I../../src/lib-lua
-DAUTH_MODULE_DIR=\""/opt/dovecot2.3/lib/dovecot/auth"\"
-DPKG_LIBEXECDIR=\""/opt/dovecot2.3/libexec/dovecot"\"
-DPKG_RUNDIR=\""/opt/dovecot2.3/var/run/dovecot"\"   -fPIE -DPIE
 -std=gnu99 -g -O2 -fstack-protector -D_FORTIFY_SOURCE=2 -Wall -W
-Wmissing-prototypes -Wmissing-declarations -Wpointer-arith
-Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime
-Wstrict-aliasing=2 -I/usr/local/include   -MT auth-main.o -MD -MP -MF
.deps/auth-main.Tpo -c -o auth-main.o `test -f 'main.c' || echo './'`main.c
mv -f .deps/auth-main.Tpo .deps/auth-main.Po
gmake[3]: *** No rule to make target '-L/usr/local/lib', needed by 'auth'.
Stop.
gmake[3]: Leaving directory
'/usr/home/wash/Tools/Dovecot/2.3/dovecot-2.3.0.rc1/src/auth'
Makefile:562: recipe for target 'all-recursive' failed
gmake[2]: *** [all-recursive] Error 1
gmake[2]: Leaving directory
'/usr/home/wash/Tools/Dovecot/2.3/dovecot-2.3.0.rc1/src'
Makefile:680: recipe for target 'all-recursive' failed
gmake[1]: *** [all-recursive] Error 1
gmake[1]: Leaving directory
'/usr/home/wash/Tools/Dovecot/2.3/dovecot-2.3.0.rc1'
Makefile:524: recipe for target 'all' failed
gmake: *** [all] Error 2


And here are my configure options:

./configure \
        --prefix=/opt/dovecot2.3 \
        --with-ioloop=kqueue \
        --with-notify=kqueue \
        --with-sql=yes \
        --with-mysql \
        --with-zlib \
        --with-bzlib \
        --with-ssl=openssl



On 19 December 2017 at 12:25, Eray Aslan <[hidden email]> wrote:

> On Mon, Dec 18, 2017 at 05:23:28PM +0200, Timo Sirainen wrote:
> > https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz
> > https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig
>
> Please do not feed libraries into LDFLAGS.  It is wrong and breaks with
> --as-needed:
>
> [...]
> checking for libsodium... yes
> checking for crypto_pwhash_str_verify... no
> configure: error: Can't build with libsodium: not found
>
> Please see the attached patch for a possible solution.
>
> Thank you
> --
> Eray
>



--
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft."
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Thomas Leuxner
In reply to this post by Timo Sirainen
* Timo Sirainen <[hidden email]> 2017.12.18 16:23:

Hi,

what is the correct way of implementing carbon stats with 2.3?

/etc/dovecot/conf.d/90-stats.conf:
old_stats_carbon_server=127.0.0.1:2003
old_stats_carbon_name=host_domain_tld
old_stats_carbon_interval=60s

/etc/dovecot/conf.d/20-imap.conf:

mail_plugins =

I changed imap_stats to imap_old_stats, however this yields the following error:

Dec 20 10:20:30 edi dovecot: imap([hidden email])<26352><9VA9GMJgns4FkqmS>: Error: module /usr/lib/dovecot/modules/lib95_imap_old_stats_plugin.so: dlsym(imap_old_stats_plugin_init) failed: /usr/lib/dovecot/modules/lib95_imap_old_stats_plugin.so: undefined symbol: imap_old_stats_plugin_init
Dec 20 10:20:30 edi dovecot: imap([hidden email])<26352><9VA9GMJgns4FkqmS>: Error: module /usr/lib/dovecot/modules/lib95_imap_old_stats_plugin.so: dlsym(imap_old_stats_plugin_deinit) failed: /usr/lib/dovecot/modules/lib95_imap_old_stats_plugin.so: undefined symbol: imap_old_stats_plugin_deinit
Dec 20 10:20:30 edi dovecot: imap([hidden email]): Error: Couldn't load required plugin /usr/lib/dovecot/modules/lib95_imap_old_stats_plugin.so: Module doesn't have init function

Regards
Thomas

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: v2.3.0 release candidate released

Michael Grimm
In reply to this post by Odhiambo Washington-4
Hi,

Odhiambo Washington <[hidden email]> wrote:

> What am I missing here:
>
> OS = FreeBSD 8.4
>
> Here is how it fails during `gmake`:

[snip]

Hmm, FBSD 8.4 has reached End of Life a long time ago, namely on August 1, 2015. It has not seen security updates ever since :-(

Thus, I am just curious: but why can't you upgrade to either 10.x or 11.x?

Regards,
Michael

12