Maildir locking by LDA of dovecot

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Maildir locking by LDA of dovecot

Mike Grozak
Hi all!

First of all, I want ti thank the whole Dovecot community (developers
for developing and fast responses, users for populating and responses
too). Dovecot is a really good and fast IMAP server - it serves near
3000 in our installation.

Now I have a problem and request community's help its resolving;)
Dovecot version: 1.0.13, compiled with vpopmail support.
I have an account [hidden email] (:-), in which we put all our spam (we
gather statistic later). There can be 10 and more delivery attempts in a
second to maildir of this account during the spam attacks on our server.
The queue of our server can grow up to 20k messages, and all of these
messages are sent to [hidden email]. I found a lot of such strings in
dovecot.log file during such DoS attack:

dovecot.log:deliver([hidden email]): May 16 04:18:48 Info:
msgid=<01c8b6da$a8bacf80$b7af2abe@sale>: save failed to INBOX: Timeout
while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:18:59 Info:
msgid=<[hidden email]>:
save failed to INBOX: Timeout while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:20:06 Info:
msgid=<01c8b6a8$7b552e00$a0a4034c@sale>: save failed to INBOX: Timeout
while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:20:14 Info:
msgid=<[hidden email]>: save failed to INBOX:
Timeout while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:20:50 Info:
msgid=<01c8b6b1$0df9cb00$cc36dfc9@tek2>: save failed to INBOX: Timeout
while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:21:00 Info:
msgid=<087901c7ca2d$ef272640$[hidden email]>: save
failed to INBOX: Timeout while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:21:01 Info:
msgid=<000701c8b6d2$01ff999f$c491aca7@snoueiij>: save failed to INBOX:
Timeout while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:22:32 Info:
msgid=<000801c8b69b$060c0230$3cd8a4b7@cpbvlrx>: save failed to INBOX:
Timeout while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:22:45 Info:
msgid=<000801c8b6a5$04ba6412$9910848d@aldldr>: save failed to INBOX:
Timeout while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:22:45 Info:
msgid=<031401c772e2$adc5be40$[hidden email]>: save failed to INBOX:
Timeout while waiting for lock
dovecot.log:deliver([hidden email]): May 16 04:23:08 Info:
msgid=<01c8b6db$0222fe80$b7af2abe@info>: save failed to INBOX: Timeout
while waiting for lock

What can I do for resolving this locks? The queue of my server becomes
really big!
May be, while looking into uidlist file, can dovecot LDA lock it? And
other deliver processes can't open it and wait for him?

$ dovecot --version
1.0.13

# dovecot -n
# 1.0.13: /etc/dovecot.conf
log_path: /var/log/dovecot.log
protocols: imap pop3
ssl_disable: yes
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(pop3): /usr/libexec/dovecot/pop3-login
login_greeting: Server is ready.
login_process_per_connection: no
login_greeting_capability(default): yes
login_greeting_capability(imap): yes
login_greeting_capability(pop3): no
login_max_connections: 1024
first_valid_uid: 39
last_valid_uid: 39
first_valid_gid: 39
last_valid_gid: 39
fsync_disable: yes
maildir_copy_with_hardlinks: yes
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(pop3): /usr/libexec/dovecot/pop3
mail_plugins(default): quota imap_quota
mail_plugins(imap): quota imap_quota
mail_plugins(pop3): quota
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/pop3
auth default:
   mechanisms: plain login cram-md5
   default_realm: tversu.ru
   cache_size: 10240
   user: vpopmail
   username_format: %Lu
   passdb:
     driver: vpopmail
     args: cache_key=%u dovecot
   userdb:
     driver: vpopmail
   socket:
     type: listen
     client:
       path: /var/spool/postfix/private/auth
       mode: 438
       user: postfix
       group: postfix
     master:
       path: /var/run/dovecot/auth-master
       mode: 438
plugin:
   quota: maildir:ignore=Trash

---------------------------
WBR, Mike Grozak, TvSU IC

Reply | Threaded
Open this post in threaded view
|

Re: Maildir locking by LDA of dovecot

Charles Marcus
On 5/22/2008, Mike Grozak ([hidden email]) wrote:
>
> First of all, I want ti thank the whole Dovecot community (developers
> for developing and fast responses, users for populating and responses
> too). Dovecot is a really good and fast IMAP server - it serves near
> 3000 in our installation.

What filesystem is this on?

--

Best regards,

Charles
Reply | Threaded
Open this post in threaded view
|

Re: Maildir locking by LDA of dovecot

Mike Grozak
Charles Marcus wrote:
> On 5/22/2008, Mike Grozak ([hidden email]) wrote:
>>
>> First of all, I want ti thank the whole Dovecot community (developers
>> for developing and fast responses, users for populating and responses
>> too). Dovecot is a really good and fast IMAP server - it serves near
>> 3000 in our installation.
>
> What filesystem is this on?
ext3.

How can I escape dovecot-uidlist locking? it really locks my system!
---------------------------
WBR, Mike Grozak, TvSU IC
Reply | Threaded
Open this post in threaded view
|

Re: Maildir locking by LDA of dovecot

Charles Marcus
On 5/22/2008 7:06 AM, Mike Grozak wrote:
>>> First of all, I want ti thank the whole Dovecot community (developers
>>> for developing and fast responses, users for populating and responses
>>> too). Dovecot is a really good and fast IMAP server - it serves near
>>> 3000 in our installation.

>> What filesystem is this on?

> ext3.
>
> How can I escape dovecot-uidlist locking? it really locks my system!

Have you read closely this page?

http://wiki.dovecot.org/MailboxFormat/Maildir

It discusses how to properly use dovecot-uidlist.lock

For example:

"The dovecot-uidlist file doesn't need to be locked for reading. When
writing dovecot-uidlist.lock file needs to be created. The
dovecot-uidlist file must never be directly modified, it can only be
replaced with rename() call."

and

"Because Dovecot uses its own non-standard locking (dovecot-uidlist.lock
dotlock file), other MUAs accessing the maildir don't support it. This
means that if another MUA is updating messages' flags or expunging
messages, Dovecot might temporarily lose some message. After the next
sync when it finds it again, an error message may be written to log and
the message will receive a new UID."

It may be a vpopmail issue, but I don't that...

Otherwise, I guess you'll have to wait to see if Timo can help...

--

Best regards,

Charles
Reply | Threaded
Open this post in threaded view
|

Re: Maildir locking by LDA of dovecot

Timo Sirainen
In reply to this post by Mike Grozak
On Thu, 2008-05-22 at 10:14 +0400, Mike Grozak wrote:
> dovecot.log:deliver([hidden email]): May 16 04:18:48 Info:
> msgid=<01c8b6da$a8bacf80$b7af2abe@sale>: save failed to INBOX: Timeout
> while waiting for lock

You can't really do much about these on Dovecot's side, but you could
try reducing the max. number of simultaneous deliver processes your MTA
launches.

Another possibility is to upgrade to Dovecot v1.1. There deliver doesn't
wait for dovecot-uidlist lock.


signature.asc (196 bytes) Download Attachment