Random duplicated emails

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Random duplicated emails

Dovecot mailing list
Hello,

I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id.

I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic  emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients.

Where should I look to diagnostic this issue?

Thanks.

Server config:
# 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.21 (92477967)
doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (2500)
doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503)
# OS: Linux 4.15.0-60-generic x86_64 Ubuntu 18.04.3 LTS
auth_default_realm = **edited**
auth_master_user_separator = *
auth_mechanisms = plain login
default_process_limit = 500
first_valid_uid = 200
last_valid_uid = 200
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_plugins = " notify replication zlib quota"
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.debug
mdbox_rotate_size = 10 M
namespace inbox {
  inbox = yes
  location =
  mailbox "Éléments envoyés" {
    special_use = \Sent
  }
  mailbox "Éléments supprimés" {
    special_use = \Trash
  }
  mailbox Brouillon {
    special_use = \Drafts
  }
  mailbox "Courrier indésirable" {
    special_use = \Junk
  }
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix = INBOX/
  separator = /
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
passdb {
  args = /etc/dovecot/passwd.masterusers
  driver = passwd-file
  master = yes
}
plugin {
  mail_replica = tcps:**edited**:53683
  quota = count:User quota
  quota_grace = 10%%
  quota_rule = *:storage=10G
  quota_rule2 = INBOX/Trash:storage=+100M
  quota_status_nouser = DUNNO
  quota_status_overquota = 552 5.2.2 La boite courriel du destinataire est pleine - Mailbox is full
  quota_status_success = DUNNO
  quota_vsizes = yes
  quota_warning = storage=100%% quota-warning 100 %u
  quota_warning2 = storage=95%% quota-warning 95 %u
  quota_warning3 = storage=90%% quota-warning 90 %u
  quota_warning4 = storage=85%% quota-warning 85 %u
  quota_warning5 = storage=75%% quota-warning 75 %u
  quota_warning6 = -storage=80%% quota-warning '-80' %u
  quota_warning7 = -storage=100%% quota-warning '-100' %u
  sieve = /var/vmail/domains/%Ld/%Ln/.dovecot.sieve
  sieve_before = /var/vmail/sieve/before.sieve
  sieve_default = /var/vmail/sieve/default.sieve
  sieve_extensions = +vnd.dovecot.debug
  sieve_global = /var/vmail/sieve/global
  zlib_save = lz4
}
protocols = " imap lmtp sieve"
service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0666
  }
  unix_listener auth-userdb {
    group = vmail
    mode = 0666
    user = vmail
  }
}
service imap-login {
  inet_listener imap {
    port = 0
  }
  process_min_avail = 10
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service quota-status {
  client_limit = 1
  executable = quota-status -p postfix
  inet_listener {
    address = **edited**
    port = 53684
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
    user = vmail
  }
}
ssl = required
ssl_cert = </etc/ssl/**edited**/**edited**.crtbundle
ssl_client_ca_dir = /etc/ssl/certs
ssl_key =  # hidden, use -P to show it
userdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
protocol lmtp {
  mail_plugins = " notify replication zlib quota sieve"
  postmaster_address = postmaster@**edited**
}
protocol imap {
  mail_plugins = " notify replication zlib quota imap_zlib imap_quota"
}

-- 
Francis

Reply | Threaded
Open this post in threaded view
|

Re: Random duplicated emails

Dovecot mailing list
On 9 Sep 2019, at 10:08, Francis <[hidden email]> wrote:
> Where should I look to diagnostic this issue?

Look at the raw stored messages. Your MTA probably has an internal ID (queuid in postfix) that should tell you if the messages are getting duped before they get to dovecot.

the raw messages should have time stamps showing when they were written, so that will give you something else to check in the logs.

Both you mail and dovecot logs, of course.

Since these are local emails on one server, is it possible that server is sending mail to the other server that is coming back as a second copy?


--
Try to realize it's all within yourself/No one else can make you change

Reply | Threaded
Open this post in threaded view
|

Re: Random duplicated emails

Dovecot mailing list
In reply to this post by Dovecot mailing list

I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id.

I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic  emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients.

Where should I look to diagnostic this issue?

Thanks.

Server config:
# 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.21 (92477967)

may be you hit this problem:


I don't know if it's fixed yet.

You could log mail events and check if it's related to dsync:

mail_plugins = ... mail_log

plugin {
  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename
  mail_log_fields = uid box msgid size subject
}

Best regards,
Gerald

Reply | Threaded
Open this post in threaded view
|

Re: Random duplicated emails

Dovecot mailing list
In reply to this post by Dovecot mailing list
Le lun. 9 sept. 2019 à 12:08, Francis <[hidden email]> a écrit :
Hello,

I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id.

I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic  emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients.

Where should I look to diagnostic this issue?


Hello,

Thanks for your hints. The message from  Битус Павел in another thread today seem to be similar to my problem. I asked my users if the affected messages are processed by a local rules in Thunderbird and they said yes. I asked them to replaces theirs local rules by sieve rules instead. We'll see if it fixes the problem.

Thanks!

-- 
Francis
Reply | Threaded
Open this post in threaded view
|

Re: Random duplicated emails

Dovecot mailing list

Hello,

When using a sieve, there is no problem with duplicates, but it is impossible to transfer all users to a sieve.

10.09.2019 20:17, Francis via dovecot пишет:
Le lun. 9 sept. 2019 à 12:08, Francis <[hidden email]> a écrit :
Hello,

I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id.

I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic  emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients.

Where should I look to diagnostic this issue?


Hello,

Thanks for your hints. The message from  Битус Павел in another thread today seem to be similar to my problem. I asked my users if the affected messages are processed by a local rules in Thunderbird and they said yes. I asked them to replaces theirs local rules by sieve rules instead. We'll see if it fixes the problem.

Thanks!

-- 
Francis
--


ROZETKA

Bitus Pavlo
System administrator
rozetka.ua