dsync having problems with @-sign in mailbox names

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

dsync having problems with @-sign in mailbox names

Dovecot mailing list
Hi there

We are running Cyrus-to-Dovecot migrations using dsync on destination Dovecot server (Debian Stretch / latest Dovecot 2.3.7.2 from community repo) like this:

  $ doveadm -o mail_fsync=never backup -R -u <username> imapc:

For imapc configuration to connect to remote Cyrus server, see below [1].
While this works great for hundreds of mailaccounts, dsync fails with the following error on accounts with @-sign in mailbox names:

  dsync(<username>): Error: Failed to access mailbox INBOX/entrance/admin@example: Invalid mailbox name

I can confirm, Dovecot supports @-sign in mailbox names and no client (tested with Thunderbird, Apple Mail, Roundcube webmail) has issue with this character.
Also Cyrus was always supporting this. So I guess this is an issue with dsync. Can we somehow tune charset (UTF-8 ?) in imapc configuration?

Thanks,
Philip


[1] /etc/dovecot/conf.d/90-migration.conf:

## IMAP
imapc_host = imap.example.com
imapc_port = 993
imapc_ssl = imaps
imapc_ssl_verify = yes
imapc_user = %u
imapc_master_user = cyrus
imapc_password = **********

imapc_features = rfc822.size fetch-headers
mail_prefetch_count = 20
imapc_list_prefix = INBOX


## POP3
pop3c_host = pop3.example.com
pop3c_port = 995
pop3c_ssl = pop3s
pop3c_ssl_verify = yes
pop3c_user = %u
pop3c_master_user = cyrus
pop3c_password = **********

namespace {
  prefix = POP3-MIGRATION-NS/
  location = pop3c:
  list = no
  hidden = yes
}
protocol doveadm {
  mail_plugins = $mail_plugins pop3_migration
}
plugin {
  pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX
}



Reply | Threaded
Open this post in threaded view
|

Re: dsync having problems with @-sign in mailbox names

Dovecot mailing list


> On 28 Sep 2019, at 12.21, Philip Iezzi via dovecot <[hidden email]> wrote:
>
> Hi there
>
> We are running Cyrus-to-Dovecot migrations using dsync on destination Dovecot server (Debian Stretch / latest Dovecot 2.3.7.2 from community repo) like this:
>
>  $ doveadm -o mail_fsync=never backup -R -u <username> imapc:
>
> For imapc configuration to connect to remote Cyrus server, see below [1].
> While this works great for hundreds of mailaccounts, dsync fails with the following error on accounts with @-sign in mailbox names:
>
>  dsync(<username>): Error: Failed to access mailbox INBOX/entrance/admin@example: Invalid mailbox name
>
> I can confirm, Dovecot supports @-sign in mailbox names and no client (tested with Thunderbird, Apple Mail, Roundcube webmail) has issue with this character.
> Also Cyrus was always supporting this. So I guess this is an issue with dsync. Can we somehow tune charset (UTF-8 ?) in imapc configuration?
>

Hi,

can you please try to record imapc rawlogs for such an session? I would like to see what kind of mUTF7 name string Cyrus sends for these folders that have special characters.
We have been able to work around some of the broken mUTF7 implementations with some BROKENCHAR magic.

Sami


Reply | Threaded
Open this post in threaded view
|

Re: dsync having problems with @-sign in mailbox names

Dovecot mailing list

> On 28 Sep 2019, at 11:28, Sami Ketola via dovecot <[hidden email]> wrote:
>
>
>
>> On 28 Sep 2019, at 12.21, Philip Iezzi via dovecot <[hidden email]> wrote:
>>
>> Hi there
>>
>> We are running Cyrus-to-Dovecot migrations using dsync on destination Dovecot server (Debian Stretch / latest Dovecot 2.3.7.2 from community repo) like this:
>>
>> $ doveadm -o mail_fsync=never backup -R -u <username> imapc:
>>
>> For imapc configuration to connect to remote Cyrus server, see below [1].
>> While this works great for hundreds of mailaccounts, dsync fails with the following error on accounts with @-sign in mailbox names:
>>
>> dsync(<username>): Error: Failed to access mailbox INBOX/entrance/admin@example: Invalid mailbox name
>>
>> I can confirm, Dovecot supports @-sign in mailbox names and no client (tested with Thunderbird, Apple Mail, Roundcube webmail) has issue with this character.
>> Also Cyrus was always supporting this. So I guess this is an issue with dsync. Can we somehow tune charset (UTF-8 ?) in imapc configuration?
>>
>
> Hi,
>
> can you please try to record imapc rawlogs for such an session? I would like to see what kind of mUTF7 name string Cyrus sends for these folders that have special characters.
> We have been able to work around some of the broken mUTF7 implementations with some BROKENCHAR magic.
>
> Sami

Hi Sami,

Thanks for your support. I have tried the following...
Added this to conf.d/20-imap.conf:

protocol imap {
  rawlog_dir = /tmp/rawlog/%u
}

Then restarted Dovecot, and re-run the following dsync migration:

$ doveadm -o mail_fsync=never backup -R -u <username> imapc:
dsync(<username>): Info: imapc(imap.example.com:993): Connected to x.x.x.x:993 (local y.y.y.y:45726)
dsync(<username>): Error: Failed to access mailbox INBOX/entrance/admin@example: Invalid mailbox name

Then checked /tmp/rawlog, which is still empty. According to https://wiki.dovecot.org/Debugging/Rawlog, setting `rawlog_dir` should be the only thing needed, right? What did I do wrong?
I am glad to provide you the raw logs as soon as I can generate them.

Thanks,
Philip
Reply | Threaded
Open this post in threaded view
|

Re: dsync having problems with @-sign in mailbox names

Dovecot mailing list

> On 28 Sep 2019, at 11:48, Philip Iezzi via dovecot <[hidden email]> wrote:
>
>
>> On 28 Sep 2019, at 11:28, Sami Ketola via dovecot <[hidden email]> wrote:
>>
>>
>>
>>> On 28 Sep 2019, at 12.21, Philip Iezzi via dovecot <[hidden email]> wrote:
>>>
>>> Hi there
>>>
>>> We are running Cyrus-to-Dovecot migrations using dsync on destination Dovecot server (Debian Stretch / latest Dovecot 2.3.7.2 from community repo) like this:
>>>
>>> $ doveadm -o mail_fsync=never backup -R -u <username> imapc:
>>>
>>> For imapc configuration to connect to remote Cyrus server, see below [1].
>>> While this works great for hundreds of mailaccounts, dsync fails with the following error on accounts with @-sign in mailbox names:
>>>
>>> dsync(<username>): Error: Failed to access mailbox INBOX/entrance/admin@example: Invalid mailbox name
>>>
>>> I can confirm, Dovecot supports @-sign in mailbox names and no client (tested with Thunderbird, Apple Mail, Roundcube webmail) has issue with this character.
>>> Also Cyrus was always supporting this. So I guess this is an issue with dsync. Can we somehow tune charset (UTF-8 ?) in imapc configuration?
>>>
>>
>> Hi,
>>
>> can you please try to record imapc rawlogs for such an session? I would like to see what kind of mUTF7 name string Cyrus sends for these folders that have special characters.
>> We have been able to work around some of the broken mUTF7 implementations with some BROKENCHAR magic.
>>
>> Sami
>
> Hi Sami,
>
> Thanks for your support. I have tried the following...
> Added this to conf.d/20-imap.conf:
>
> protocol imap {
>  rawlog_dir = /tmp/rawlog/%u
> }
>
> Then restarted Dovecot, and re-run the following dsync migration:
>
> $ doveadm -o mail_fsync=never backup -R -u <username> imapc:
> dsync(<username>): Info: imapc(imap.example.com:993): Connected to x.x.x.x:993 (local y.y.y.y:45726)
> dsync(<username>): Error: Failed to access mailbox INBOX/entrance/admin@example: Invalid mailbox name
>
> Then checked /tmp/rawlog, which is still empty. According to https://wiki.dovecot.org/Debugging/Rawlog, setting `rawlog_dir` should be the only thing needed, right? What did I do wrong?
> I am glad to provide you the raw logs as soon as I can generate them.
>
> Thanks,
> Philip

okay, I probably just need to configure rawlog as post-logins script...

service imap {
  executable = imap postlogin
}
service pop3 {
  executable = pop3 postlogin
}
service postlogin {
  executable = script-login -d rawlog
  unix_listener postlogin {
  }
}

Need to change trains now, will try later.

Best,
Philip
Reply | Threaded
Open this post in threaded view
|

Re: dsync having problems with @-sign in mailbox names

Dovecot mailing list


> On 28 Sep 2019, at 12.52, Philip Iezzi via dovecot <[hidden email]> wrote:
>>
>> Hi Sami,
>>
>> Thanks for your support. I have tried the following...
>> Added this to conf.d/20-imap.conf:
>>
>> protocol imap {
>> rawlog_dir = /tmp/rawlog/%u
>> }


just add setting imapc_rawlog_dir=/path/to/%u

and make sure the directory exists and is writable.

Sami

Reply | Threaded
Open this post in threaded view
|

Re: dsync having problems with @-sign in mailbox names

Dovecot mailing list


> On 28 Sep 2019, at 12.55, Sami Ketola <[hidden email]> wrote:
>
>
>
>> On 28 Sep 2019, at 12.52, Philip Iezzi via dovecot <[hidden email]> wrote:
>>>
>>> Hi Sami,
>>>
>>> Thanks for your support. I have tried the following...
>>> Added this to conf.d/20-imap.conf:
>>>
>>> protocol imap {
>>> rawlog_dir = /tmp/rawlog/%u
>>> }
>
>
> just add setting imapc_rawlog_dir=/path/to/%u
>
> and make sure the directory exists and is writable.
>

and make it global setting. not just for specific protocol. And if you want to do that instead then it would be protocol doveadm not imap.

Sami