Fast replication

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Fast replication

@4OS
Hi,

I'm trying to get replication to work with a 2 machines setup.

So far full replication is happening at the specified interval (24hrs)
without problem but fast sync is not happening.
The fast sync counter is always equal to full sync unless i exec dsync
manually.

doveadm replicator status '*'
username priority fast sync full sync success sync failed
[hidden email] none     00:06:33  00:06:33  00:21:33     -
[hidden email] none     00:06:34  00:06:34  00:06:34     -
[hidden email] none     00:06:34  00:06:34  00:06:34     -

The fast replication part is not clear and i cannot find much about it.
There is nothing related with dovecot -a and the documentation does not
say much about it.

How the fast replication is triggered? Manually or it is supposed to be
automatic through some type of config.

Some light over this would help me understand the way it work and how to
get it setup right.

yves
Reply | Threaded
Open this post in threaded view
|

Re: Fast replication

Aki Tuomi-2


On 16.10.2017 20:34, @4OS wrote:

> Hi,
>
> I'm trying to get replication to work with a 2 machines setup.
>
> So far full replication is happening at the specified interval (24hrs)
> without problem but fast sync is not happening.
> The fast sync counter is always equal to full sync unless i exec dsync
> manually.
>
> doveadm replicator status '*'
> username priority fast sync full sync success sync failed
> [hidden email] none     00:06:33  00:06:33  00:21:33     -
> [hidden email] none     00:06:34  00:06:34  00:06:34     -
> [hidden email] none     00:06:34  00:06:34  00:06:34     -
>
> The fast replication part is not clear and i cannot find much about it.
> There is nothing related with dovecot -a and the documentation does
> not say much about it.
>
> How the fast replication is triggered? Manually or it is supposed to
> be automatic through some type of config.
>
> Some light over this would help me understand the way it work and how
> to get it setup right.
>
> yves

Fast replication should occur when there are some mails being saved.

Aki

Aki
Reply | Threaded
Open this post in threaded view
|

Re: Fast replication

@4OS


On 2017-10-17 01:53, Aki Tuomi wrote:

>
> On 16.10.2017 20:34, @4OS wrote:
>> Hi,
>>
>> I'm trying to get replication to work with a 2 machines setup.
>>
>> So far full replication is happening at the specified interval (24hrs)
>> without problem but fast sync is not happening.
>> The fast sync counter is always equal to full sync unless i exec dsync
>> manually.
>>
>> doveadm replicator status '*'
>> username priority fast sync full sync success sync failed
>> [hidden email] none     00:06:33  00:06:33  00:21:33     -
>> [hidden email] none     00:06:34  00:06:34  00:06:34     -
>> [hidden email] none     00:06:34  00:06:34  00:06:34     -
>>
>> The fast replication part is not clear and i cannot find much about it.
>> There is nothing related with dovecot -a and the documentation does
>> not say much about it.
>>
>> How the fast replication is triggered? Manually or it is supposed to
>> be automatic through some type of config.
>>
>> Some light over this would help me understand the way it work and how
>> to get it setup right.
>>
>> yves
> Fast replication should occur when there are some mails being saved.
>
> Aki
>
> Aki
Hi,
That where is my problem. The fast sync is not triggering and i cannot
find any hint of what is wrong.
I moved that setup to a production server with a thousand active
mailboxes and i get the same result.
The only way to get the fast sync to work is calling dsync directly :
doveadm sync -d -N -l 30 -U -u '*'
Maybe it is in my config.

Setup is 2 dovecot vm with nfs storage (distinct server each) (debian
jessie 8.9 + backports)
about a thousand mailboxes receiving an average of 18K mail daily

here is the dovecot -n for both server:

# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.9.0-0.bpo.3-amd64 x86_64 Debian 8.9
auth_master_user_separator = +
auth_mechanisms = plain login
auth_verbose = yes
disable_plaintext_auth = no
doveadm_password =  # hidden, use -P to show it
doveadm_port = 4711
listen = *
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_plugins = " notify replication quota"
mail_privileged_group = vmail
passdb {
   args = /etc/dovecot/dovecot-sql.conf
   driver = sql
}
passdb {
   args = /etc/dovecot/dovecot-superuser.conf
   driver = sql
   master = yes
}
plugin {
   mail_replica = tcp:xxx.xxx.xxx.xxx:4711
   quota = dict:user::file:/var/vmail/%d/%n/.quotausage
   quota_exceeded_message = %u :: Courriel non remis : Boite pleine.
   quota_rule2 = Trash:storage=+10%%
   quota_warning = storage=95%% quota-warning 95 %u
   quota_warning2 = storage=80%% quota-warning 80 %u
   sieve = /var/vmail/%d/%n/.sieve
   sieve_max_redirects = 25
}
protocols = imap pop3
service aggregator {
   fifo_listener replication-notify-fifo {
     mode = 0666
     user = vmail
   }
   unix_listener replication-notify {
     mode = 0666
     user = vmail
   }
}
service auth {
   unix_listener /var/spool/postfix/private/auth {
     group = postfix
     mode = 0660
     user = postfix
   }
   unix_listener auth-userdb {
     group = vmail
     mode = 0600
     user = vmail
   }
   user = root
}
service config {
   unix_listener config {
     user = vmail
   }
}
service doveadm {
   inet_listener {
     port = 4711
   }
   user = vmail
}
service imap-login {
   client_limit = 1000
   process_limit = 512
}
service lmtp {
   unix_listener /var/spool/postfix/private/dovecot-lmtp {
     group = postfix
     mode = 0600
     user = postfix
   }
}
service quota-warning {
   executable = script /usr/local/bin/quota-warning.sh
   unix_listener quota-warning {
     group = vmail
     mode = 0666
     user = vmail
   }
   user = vmail
}
service replicator {
   process_limit = 1
   process_min_avail = 1
   unix_listener replicator-doveadm {
     mode = 0666
   }
}
ssl_ca = </etc/ssl/private/*.xxx.xxx.bundle
ssl_cert = </etc/ssl/private/*.xxx.xxx.crt
ssl_cipher_list =
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:DES-CBC3-SHA:!aNULL:!MD5:!DSS:!EXPORT:!LOW:!eNULL
ssl_dh_parameters_length = 2048
ssl_key =  # hidden, use -P to show it
ssl_prefer_server_ciphers = yes
ssl_protocols = !SSLv2 !SSLv3
userdb {
   driver = prefetch
}
userdb {
   args = /etc/dovecot/dovecot-sql.conf
   driver = sql
}
protocol imap {
   mail_plugins = quota imap_quota
}
protocol pop3 {
   mail_plugins = quota
   pop3_uidl_format = %08Xu%08Xv
}
protocol lda {
   mail_plugins = sieve quota
   postmaster_address = webmaster@localhost
}
protocol lmtp {
   mail_plugins = quota sieve
   postmaster_address = webmaster@localhost
}