dovecot full text search

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

dovecot full text search

Dovecot mailing list
what FTP module should i use instead of squat that is probably no longer
supported or no longer at all?

i want to upgrade my dovecot installation. it currently uses squat but i
found it often crashes on FTS on large mailboxes.

i found "xapian" addon for dovecot but while it works excellent AFTER
database is created, i found it need like 20 or so minutes to index less
than 10GB of mails and while doing this - generate many tens of
megabytes/s constant write traffic on it's database files.

Excellent way of killing SSD.

something must be broken.

my config is

plugin {
         plugin = fts fts_xapian

         fts = xapian
         fts_xapian = partial=2 full=20 verbose=0

         fts_autoindex = yes
         fts_enforced = yes

#       fts_autoindex_exclude = \Junk
#       fts_autoindex_exclude2 = \Trash
}


any ideas?
Reply | Threaded
Open this post in threaded view
|

Re: dovecot full text search

Dovecot mailing list
Where do write ops take place?
Maybe mount that path to a RAM disk rather than looking for anorher solution.

>
> Am 10.12.2019 um 15:50 schrieb Wojciech Puchar via dovecot <[hidden email]>:
>
> what FTP module should i use instead of squat that is probably no longer supported or no longer at all?
>
> i want to upgrade my dovecot installation. it currently uses squat but i found it often crashes on FTS on large mailboxes.
>
> i found "xapian" addon for dovecot but while it works excellent AFTER database is created, i found it need like 20 or so minutes to index less than 10GB of mails and while doing this - generate many tens of megabytes/s constant write traffic on it's database files.
>
> Excellent way of killing SSD.
>
> something must be broken.
>
> my config is
>
> plugin {
>        plugin = fts fts_xapian
>
>        fts = xapian
>        fts_xapian = partial=2 full=20 verbose=0
>
>        fts_autoindex = yes
>        fts_enforced = yes
>
> #       fts_autoindex_exclude = \Junk
> #       fts_autoindex_exclude2 = \Trash
> }
>
>
> any ideas?

Reply | Threaded
Open this post in threaded view
|

Re: dovecot full text search

Dovecot mailing list
> Where do write ops take place?

to the xapian index  subdirectory

> Maybe mount that path to a RAM disk rather than looking for anorher solution.
>
not a solution for a problem but workaround


>>
>> Am 10.12.2019 um 15:50 schrieb Wojciech Puchar via dovecot <[hidden email]>:
>>
>> what FTP module should i use instead of squat that is probably no longer supported or no longer at all?
>>
>> i want to upgrade my dovecot installation. it currently uses squat but i found it often crashes on FTS on large mailboxes.
>>
>> i found "xapian" addon for dovecot but while it works excellent AFTER database is created, i found it need like 20 or so minutes to index less than 10GB of mails and while doing this - generate many tens of megabytes/s constant write traffic on it's database files.
>>
>> Excellent way of killing SSD.
>>
>> something must be broken.
>>
>> my config is
>>
>> plugin {
>>        plugin = fts fts_xapian
>>
>>        fts = xapian
>>        fts_xapian = partial=2 full=20 verbose=0
>>
>>        fts_autoindex = yes
>>        fts_enforced = yes
>>
>> #       fts_autoindex_exclude = \Junk
>> #       fts_autoindex_exclude2 = \Trash
>> }
>>
>>
>> any ideas?
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: dovecot full text search

Joan Moreau-2

Hi

The first run of indexing on a large existing mailbox is indeed slow, and I would run "doveadm index -A -q \*" before putting the system in production.

Besides the Ram disk, what kind of solution would you suggest ?


 


On 2019-12-10 19:28, Wojciech Puchar via dovecot wrote:

Where do write ops take place?

to the xapian index  subdirectory

Maybe mount that path to a RAM disk rather than looking for anorher solution.

not a solution for a problem but workaround



Am 10.12.2019 um 15:50 schrieb Wojciech Puchar via dovecot <[hidden email]>:

what FTP module should i use instead of squat that is probably no longer supported or no longer at all?

i want to upgrade my dovecot installation. it currently uses squat but i found it often crashes on FTS on large mailboxes.

i found "xapian" addon for dovecot but while it works excellent AFTER database is created, i found it need like 20 or so minutes to index less than 10GB of mails and while doing this - generate many tens of megabytes/s constant write traffic on it's database files.

Excellent way of killing SSD.

something must be broken.

my config is

plugin {
       plugin = fts fts_xapian

       fts = xapian
       fts_xapian = partial=2 full=20 verbose=0

       fts_autoindex = yes
       fts_enforced = yes

#       fts_autoindex_exclude = \Junk
#       fts_autoindex_exclude2 = \Trash
}


any ideas?