Broken Header uidvalidity problem after upgrade

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Broken Header uidvalidity problem after upgrade

Robert Henjes
Hi, we use a current Debian etch distribution as basis for our servers,
which are running within a vmware server environment. Our current
configuration consists of a dovecot server with imap enabled and a ldap
based authentication. Mails are stored using the Maildir format on a
central nfs store. We do not use any virtual mailboxes.

Since upgrading from dovecot version 1.0.rc15 (debian stable) to 1.0.13
(debian etch-backport) we got the following error logs.

Excerpt from mail.err:
Apr 21 12:04:38 servera deliver(usera):
/var/mail/Maildir/usera/dovecot-uidlist: Broken header (uidvalidity = 0,
next_uid=9)
Apr 21 15:37:50 servera deliver(userb):
/var/mail/Maildir/userb/dovecot-uidlist: Broken header (uidvalidity = 0,
next_uid=16)
Apr 23 10:37:17 servera dovecot: IMAP(userc):
/var/mail/Maildir/userc/.Sent/dovecot-uidlist: Broken header
(uidvalidity = 0, next_uid=2)
Apr 25 17:00:17 servera dovecot: IMAP(userd):
/var/mail/Maildir/userd/dovecot-uidlist: Broken header (uidvalidity = 0,
next_uid=8)

I did already some mailinglist reviews. They confirmed in some way the
problem, but there is still no explanation where it comes from, and how
it can be solved or if it may affect the proper operation of the system.

Here are some links with a short comment, which are related to this problem:
http://www.dovecot.org/list/dovecot/2007-September/025541.html
This post also uses nfs, but pam for authentication, error message is
exactly the same as in our case.

http://www.dovecot.org/list/dovecot/2007-July/024001.html
Version upgrade from 1.0.0 to one above, also problems with uidvalidity.
They described, that it happens only once for each user after upgrading.
This is similar to our observations. No user complains as well ;). So it
would be only of interest if it is so.

http://wiki.dovecot.org/Clients/NegativeUIDs
The number for uidvalidity is zero, so it is non-negative. But maybe it
is correlated with these hints.

 From my point of view, there could be some problems with time
synchronization as well between nfs server and dovecot server machine.
But we checked this and they differ never more than 2 seconds.

In total we are running two servers in parallel, each has it's own
userspace, all accounts are strictly separated. One server is running
the rc15 version, the other (more experimental) is running the backport
1.0.13. The described problem occurs only on the 1.0.13 server, which
was previously updated. All other configurations are exaclty the same.

Any hints?

Best regards,
Robert

--
Dipl.-Inform. Robert Henjes
University of Wuerzburg,
Institute of Computer Science,
Department of Distributed Systems (Informatik III),
Am Hubland, 97074 Wuerzburg, Germany

[hidden email]
Tel: +49 931 888-6652
Fax: +49 931 888-6632

Reply | Threaded
Open this post in threaded view
|

Re: Broken Header uidvalidity problem after upgrade

Timo Sirainen
On Apr 29, 2008, at 12:19 PM, Robert Henjes wrote:
> Excerpt from mail.err:
> Apr 21 12:04:38 servera deliver(usera): /var/mail/Maildir/usera/
> dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=9)
> Apr 21 15:37:50 servera deliver(userb): /var/mail/Maildir/userb/
> dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=16)

To me these look like the older version just hadn't noticed broken  
uidvalidity values. Newer version has sanity checks and fixes the  
problem. Or does the error message happen more than once for the same  
mailbox?

I think some versions of my Courier migration script could have  
written zero uidvalidity values.


PGP.sig (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Broken Header uidvalidity problem after upgrade

Robert Henjes
Timo Sirainen schrieb:

> On Apr 29, 2008, at 12:19 PM, Robert Henjes wrote:
>> Excerpt from mail.err:
>> Apr 21 12:04:38 servera deliver(usera):
>> /var/mail/Maildir/usera/dovecot-uidlist: Broken header (uidvalidity =
>> 0, next_uid=9)
>> Apr 21 15:37:50 servera deliver(userb):
>> /var/mail/Maildir/userb/dovecot-uidlist: Broken header (uidvalidity =
>> 0, next_uid=16)
>
> To me these look like the older version just hadn't noticed broken
> uidvalidity values. Newer version has sanity checks and fixes the
> problem. Or does the error message happen more than once for the same
> mailbox?
>
> I think some versions of my Courier migration script could have
> written zero uidvalidity values.
>
Ok I think you're right. Your explanation does completley match our
history of installing dovecot.

We installed the old dovecot version and converted the mails using mb2md
(debian etch version 3.20) from mbox format to maildir format. Formerly
we used uw-imap as mailserver, which seems after a short review not to
care about this uidvalidity property at all. So after upgrading from
1.0.rc15 to 1.0.13 these sanity checks detected the problem and
corrected them. So as observed until now, this happens only once for
certain mailbox users either during deliver or while accessing using
imap. But it doesn't happen necessarily to each mailbox user.

Is there a possibility to check the uidvalidity from administrators
point of view on the server?




Reply | Threaded
Open this post in threaded view
|

Re: Broken Header uidvalidity problem after upgrade

Timo Sirainen
On Tue, 2008-04-29 at 17:10 +0200, Robert Henjes wrote:
> Is there a possibility to check the uidvalidity from administrators
> point of view on the server?

Something like:

find . -name dovecot-uidlist|xargs head -1|grep '^1 0 '

The point is just that UIDVALIDITY must not be zero. Dovecot relies on
this internally in some places, and also it's illegal to send such
UIDVALIDITY using IMAP protocol. Strange that it worked with you
before. :)


signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Broken Header uidvalidity problem after upgrade

Robert Henjes
Timo Sirainen schrieb:

> On Tue, 2008-04-29 at 17:10 +0200, Robert Henjes wrote:
>  
>> Is there a possibility to check the uidvalidity from administrators
>> point of view on the server?
>>    
>
> Something like:
>
> find . -name dovecot-uidlist|xargs head -1|grep '^1 0 '
>
> The point is just that UIDVALIDITY must not be zero. Dovecot relies on
> this internally in some places, and also it's illegal to send such
> UIDVALIDITY using IMAP protocol. Strange that it worked with you
> before. :)
>
>  
So after a while of testing and playing around I figured out the following:

If you create a new and empty directory with version 1.0.rc15 then there
is no dovecot-uidlist file.
After moving a new mail to this folder one is created

1 0 2
1 1209486091.P3655Q5.serverb:2,S

Moving another mail to this dir leads to:

1 0 3
1 1209486091.P3655Q5.serverb:2,S
2 1209486279.P3655Q6.serverb:2,S

tataaaa .. and there is the zero value, which should not be there
according to your report.
So it seems to be a bug of this debian version.
After reviewing all the files of the users on our system, it seems that
all new folders created between the migration to 1.0.rc15  and the
upgrade to 1.0.13 had this problem. I didn't verify the 1.0.13 version
yet due to the lack of time.

Any hints, to solve this problem, besides simply upgrading?

Best regards,
Robert

Reply | Threaded
Open this post in threaded view
|

Re: Broken Header uidvalidity problem after upgrade

Robert Henjes

> 1 0 2
> 1 1209486091.P3655Q5.serverb:2,S
>
> Moving another mail to this dir leads to:
>
> 1 0 3
> 1 1209486091.P3655Q5.serverb:2,S
> 2 1209486279.P3655Q6.serverb:2,S
>
In the mean time I repeated the test with creating a new mailbox
(folder) using version 1.0.13 (debian etch-backport). Using this version
the uidvalidity seems to be ok. See the ecxcerpt of the log file:

1 1209544994 2
1 1209544994.P18795Q0M587290.servera:2,S

So I think simply upgrading to version 1.0.13 should be the savest way
to repair this bug and correct the wrong uids.

    Robert


--
Dipl.-Inform. Robert Henjes
University of Wuerzburg,
Institute of Computer Science,
Department of Distributed Systems (Informatik III),
Am Hubland, 97074 Wuerzburg, Germany

[hidden email]
Tel: +49 931 888-6652
Fax: +49 931 888-6632