Dovecot Index Corruption and dovecot-uidlist Issues

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Dovecot Index Corruption and dovecot-uidlist Issues

Dovecot mailing list

Hello,

 

We have recently implemented dovecot directors to resolve an ongoing NFS lock/corruption issue. Everything was working great! So we updated our servers from centos 7.3 to 7.7 and dovecot 2.2.10 to 2.3.8

This was a bit of a failure and we were getting spammed with corrupt index files and logs. After we clear the index files with rm rf/mail/d/e/username/Maildir/dovecot*, they just come back in.

From:

dovecot-mysql-2.3.8-4.x86_64

dovecot-2.3.8-4.x86_64

dovecot-pigeonhole-2.3.8-4.x86_64

To:

dovecot-2.2.10-8.el7.x86_64

dovecot-mysql-2.2.10-8.el7.x86_64

dovecot-pigeonhole-2.2.10-8.el7.x86_64

 

We rolled back nfs-utils and libs due to a case on redhats bug report “https://bugzilla.redhat.com/show_bug.cgi?id=1521071

libnfsidmap-0.25-19.el7.x86_64

nfs-utils-1.3.0-0.65.el7.x86_64

to

libnfsidmap-0.25-15.el7.x86_64

nfs-utils-1.3.0-0.33.el7.x86_64

 

We rolled back our dovecot version to 2.2.10 and are still getting the following logs.

 

Any advice would help

 

Network view:

 

|Dovecot Directors x 3|

                     |

|Dovecot servers x4| Storage is shared between all 4 servers

                     |

|Netapp NFSv3 Storage| Nothing changed here

 

 

Oct 25 01:33:25 <IP> dovecot: imap(<username>): Error: Log synchronization error at seq=89,offset=32340 for /mail/j/k/<username>/Maildir/dovecot.index: Extension record update for invalid uid=6798

Oct 25 01:33:25 <IP> dovecot: imap(<username>): Error: Log synchronization error at seq=89,offset=32384 for /mail/j/k/<username>/Maildir/dovecot.index: Extension record update for invalid uid=6798

Oct 25 01:33:25 <IP> dovecot: imap(<username>): Error: Synchronization corrupted index header /mail/j/k/<username>/Maildir/dovecot.index: first_recent_uid 6799 > next_uid 6798

Oct 25 01:33:25 <IP> dovecot: imap(<username>): Warning: fscking index file /mail/j/k/<username>/Maildir/dovecot.index

Oct 25 01:33:25 <IP> dovecot: imap(<username>): Error: Fixed index file /mail/j/k/<username>/Maildir/dovecot.index: first_recent_uid 6799 -> 6798

Oct 25 01:33:26 <IP> dovecot: imap(<username>): Error: Broken file /mail/j/k/<username>/Maildir/dovecot-uidlist line 6461: Invalid data:

Oct 25 01:33:26 <IP> dovecot: imap(<username>): Error: Log synchronization error at seq=89,offset=32488 for /mail/j/k/<username>/Maildir/dovecot.index: Append with UID 6798, but next_uid = 6799

Oct 25 01:33:26 <IP> dovecot: imap(<username>): Warning: fscking index file /mail/j/k/<username>/Maildir/dovecot.index

Oct 25 01:47:14 <IP> dovecot: imap(<username>): Warning: fscking index file /mail/d/e/<username>/Maildir/dovecot.index

Oct 25 01:47:14 <IP> dovecot: imap(<username>): Error: Failed to map transaction log /mail/d/e/<username>/Maildir/dovecot.index.log at sync_offset=9900 after locking: pread() failed: Stale file handle

Oct 25 01:47:14 <IP> dovecot: imap(<username>): Error: Index /mail/d/e/<username>/Maildir/dovecot.index: Lost log for seq=16 offset=192: Missing middle file seq=16 (between 16..4294967295, we have seqs 14): Requested newer log than exists: Log inode is unchanged (initial_mapped=1, reason=Index mapped)

Oct 25 01:51:14 <IP> dovecot: imap(<username>): Error: Broken file /mail/d/e/<username>/Maildir/dovecot-uidlist line 65: Invalid data:

 

Dovecot -n

 

# 2.2.10: /etc/dovecot/dovecot.conf

# OS: Linux 3.10.0-1062.1.2.el7.x86_64 x86_64 CentOS Linux release 7.7.1908 (Core)

auth_mechanisms = plain login

auth_username_format = %Ln

auth_verbose = yes

auth_verbose_passwords = plain

disable_plaintext_auth = no

first_valid_uid = 300

lda_mailbox_autocreate = yes

lda_mailbox_autosubscribe = yes

mail_fsync = always

mail_location = maildir:~/Maildir

mail_nfs_index = yes

mail_nfs_storage = yes

maildir_very_dirty_syncs = yes

managesieve_notify_capability = mailto

managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave

mmap_disable = yes

namespace inbox {

  inbox = yes

  location =

  mailbox Drafts {

    special_use = \Drafts

  }

  mailbox Junk {

    special_use = \Junk

  }

  mailbox Sent {

    special_use = \Sent

  }

  mailbox "Sent Messages" {

    special_use = \Sent

  }

  mailbox Trash {

    special_use = \Trash

  }

  prefix =

}

passdb {

  args = /etc/dovecot/sql.conf.ext

  driver = sql

}

plugin {

  sieve = ~/.dovecot.sieve

  sieve_dir = ~/sieve

  sieve_global_dir = /mail/sieve/global/

  sieve_global_path = /mail/sieve/default.sieve

}

postmaster_address = postmaster@%d

protocols = imap pop3 lmtp sieve

service auth {

  client_limit = 0

  inet_listener {

    port = 11010

  }

  process_limit = 1

  process_min_avail = 0

  service_count = 0

}

service dict {

  unix_listener dict {

    group = vmail

    mode = 0600

    user = vmail

  }

}

service imap-login {

  inet_listener imap {

    port = 143

  }

  inet_listener imaps {

    port = 993

    ssl = yes

  }

  process_limit = 2

  process_min_avail = 2

  service_count = 0

}

service imap {

  client_limit = 1

  executable = imap

  process_limit = 2048

  process_min_avail = 50

  service_count = 1

}

service lmtp {

  client_limit = 1

  process_limit = 0

  process_min_avail = 0

  service_count = 0

  unix_listener /var/spool/postfix/private/dovecot-lmtp {

    group = postfix

    mode = 0660

    user = postfix

  }

}

service managesieve-login {

  inet_listener sieve {

    port = 4190

  }

}

service pop3-login {

  inet_listener pop3 {

    port = 110

  }

  inet_listener pop3s {

    port = 995

    ssl = yes

  }

  process_limit = 2

  process_min_avail = 2

  service_count = 0

}

service pop3 {

  client_limit = 1

  executable = pop3

  process_limit = 256

  process_min_avail = 25

  service_count = 1

}

ssl_ca = </etc/ssl/mdc/mdc.ca

ssl_cert = </etc/ssl/mdc/mdc.crt

ssl_cipher_list = !RC4:HIGH:MEDIUM:+TLSv1:!MD5:!SSLv2:+SSLv3:!ADH:!aNULL:!eNULL:!NULL:!DH:!ADH:!EDH:!AESGCM

ssl_key = </etc/ssl/mdc/mdc.key

ssl_protocols = !SSLv2 !SSLv3

userdb {

  args = /etc/dovecot/sql.conf.ext

  driver = sql

  override_fields = plain_pass=%w

}

protocol lmtp {

  mail_plugins = " quota sieve"

}

protocol imap {

  mail_max_userip_connections = 40

}

protocol sieve {

  info_log_path = /var/log/dovecot-sieve.log

  log_path = /var/log/dovecot-sieve-errors.log

  managesieve_implementation_string = dovecot

  managesieve_max_line_length = 65536

}

 

 

Billy Luedtke, Microsoft MCP

System Administrator II, WIN