fts-solr: fresh start or reindex large mailboxes

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

fts-solr: fresh start or reindex large mailboxes

Anton Borovoy
Hi,

Install solr (in the end 7.1.0) there were errors:
Caused by: org.eclipse.jetty.io.EofException: Early EOF
and from dovecot:
Error: Mailbox INBOX: Transaction commit failed: FTS transaction
commit failed: backend

I could not find for a long time what the problem was and set the
nginx as a proxy, and see that dovecot breaks the connect after 60
seconds.
After examining the code, I found the fixed connection time:
http_set.request_timeout_msecs = 60 * 1000;
which is clearly not enough for large data.
After searching, he also found this post
https://dovecot.org/pipermail/dovecot-cvs/2015-March/025551.html

1. I think it would be a good idea to remove these settings in the
configuration file dovecot.conf

A bit of statistics from nginx, solr and nginx on same server (req_len
== post body):
60 seconds, requests do not pass:
[25 / Oct / 2017: 23: 07: 11 +0700] POST / solr / dovecot / update
HTTP / 1.1 resp_time - msec 1508947631.364 req_time 86.863 req_len
171401961 499
[25 / Oct / 2017: 23: 07: 25 +0700] POST / solr / dovecot / update
HTTP / 1.1 resp_time - msec 1508947645.149 req_time 74.657 req_len
70140598 499
[25 / Oct / 2017: 23: 11: 28 +0700] POST / solr / dovecot / update
HTTP / 1.1 resp_time - msec 1508947888.834 req_time 90.776 req_len
205474676 499
600 seconds, requests do not pass:
[26 / Oct / 2017: 04: 50: 50 +0700] POST / solr / dovecot / update
HTTP / 1.1 resp_time - msec 1508968250.522 req_time 811.480 req_len
3083215798 499
[26 / Oct / 2017: 04: 51: 06 +0700] POST / solr / dovecot / update
HTTP / 1.1 resp_time - msec 1508968266.432 req_time 666.089 req_len
556157906 499
[26 / Oct / 2017: 05: 06: 18 +0700] POST / solr / dovecot / update
HTTP / 1.1 resp_time - msec 1508969178.612 req_time 797.983 req_len
3083215798 499

2. the index is more than 3g is transmitted painfully, can U make a
queue limit on the size of the request, segment them, adjust max size
of post body in the configuration file dovecot.conf

thx

solr: 7.1.0
dovecot: 2.2.32
server: Intel(R) Xeon(TM) CPU 2.93GHz x24, 72G RAM