Manual thread breaking from Thunderbird

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Manual thread breaking from Thunderbird

Christian Jaeger-2
Hello

I'm looking for a way to manually break threads apart when independent
threads are mixed up, mostly by users which aren't aware that they
should not be replying to existing threads when starting a new one.

Since Thunderbird afaik does the threading itself, it would require a
solution that actually changes the In-Reply-To header (and maybe also
that other (non-standard?) header which is being sent by the M$
clients). For me modifying the actual emails on disk would be fine, i.e.
renaming the In-Reply-To header to Old-In-Reply-To and such. (I'd
actually *prefer* a solution that modifies the server side store, as
opposed to one which keeps that info on the client side only like mutt
seems to be doing.)

Is there a way to do this already? Seems like a task for a dovecot
plugin--or for the xexec plugin and an external perl script or some such?

For triggering the renaming, a hacky solution could be to misuse labels
(e.g. marking a message starred and "Later" and "Important" at the same
time would trigger the modification and remove the labels).

When implementing such a thing: maybe also offer a way to modify
subjects (for those cases where people are using short meaningless and
recurring subjects--to both help breaking up the ensuing threading as
well as giving them a useful replacement), maybe as an IMAP extension or
such (to be used by Thunderbird plugins (?), or by web apps).

(I can code in C and Perl, but don't generally have much time.)

Christian.

Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bugzilla from rosenfield.albert@gmail.com
Christian Jaeger wrote:
>  I'm looking for a way to manually break threads apart when independent
> threads are mixed up, mostly by users which aren't aware that they should
> not be replying to existing threads when starting a new one.

Sounds like a great idea!

I'm curious though, how do convince your Thunderbird and Dovecot to
work like Gmail and keep everything threaded in the same folder in the
first place?

(Assuming that you do this and apply labels to tell things apart..)
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Christian Jaeger-2
[hidden email] wrote:

> Christian Jaeger wrote:
>  
>>  I'm looking for a way to manually break threads apart when independent
>> threads are mixed up, mostly by users which aren't aware that they should
>> not be replying to existing threads when starting a new one.
>>    
>
> Sounds like a great idea!
>
> I'm curious though, how do convince your Thunderbird and Dovecot to
> work like Gmail and keep everything threaded in the same folder in the
> first place?
>  

I'm not sure whether I understand you correctly: are you asking how to
turn on threading in Thunderbird? (That would be in the menu View ->
Sort by -> Threaded.) But that's Thunderbird's doing, Dovecot isn't
really involved afaik, I assume Thunderbird will look at the message-id
and in-reply-to headers of the mails.

Or are you asking about the Gmail feature of not using subfolders but
keep everything in the same place and only use search to find your
stuff? (Well, I haven't used Gmail much, one of the reason being that
when I tried it I thought how would you keep mailing lists separated
sanely? (Without having to create separate custom queries for each of
them or so.))

Christian.

Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

mouss-2
Christian Jaeger wrote:

> [hidden email] wrote:
>> Christian Jaeger wrote:
>>  
>>>  I'm looking for a way to manually break threads apart when independent
>>> threads are mixed up, mostly by users which aren't aware that they
>>> should
>>> not be replying to existing threads when starting a new one.
>>>    
>>
>> Sounds like a great idea!
>>
>> I'm curious though, how do convince your Thunderbird and Dovecot to
>> work like Gmail and keep everything threaded in the same folder in the
>> first place?
>>  
>
> I'm not sure whether I understand you correctly: are you asking how to
> turn on threading in Thunderbird? (That would be in the menu View ->
> Sort by -> Threaded.) But that's Thunderbird's doing, Dovecot isn't
> really involved afaik, I assume Thunderbird will look at the
> message-id and in-reply-to headers of the mails.

and References and few other heuristics.

>
> Or are you asking about the Gmail feature of not using subfolders but
> keep everything in the same place and only use search to find your stuff?

I guess that's what he means. In short, how to have the same message
appear in different folder (be them "virtual" or not). and I guess, if
you delete a message, you no more see it in the said folders.

I believe the right place for this feature is in the MUA (thunderbird)
to avoid downloading the same message N times.

> (Well, I haven't used Gmail much, one of the reason being that when I
> tried it I thought how would you keep mailing lists separated sanely?
> (Without having to create separate custom queries for each of them or
> so.))
>
> Christian.
>

Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bill Cole-4
In reply to this post by Bugzilla from rosenfield.albert@gmail.com
At 1:41 PM +0200 5/7/08, [hidden email] wrote:
>Christian Jaeger wrote:
>>   I'm looking for a way to manually break threads apart when independent
>>  threads are mixed up, mostly by users which aren't aware that they should
>>  not be replying to existing threads when starting a new one.
>
>Sounds like a great idea!

The IMAP extension that supports threading
(http://tools.ietf.org/html/draft-ietf-imapext-sort-20) has no
support for clients 'breaking' threads. In fact, it does not
explicitly or implicitly require that the IMAP server preserve any
state about threading that cannot be reconstructed from the messages,
but rather implies that an IMAP server rebuilds  the thread map from
the messages for each THREAD command.

This means that if you want to break threads, it has to be entirely
done in the client. Personally, I think that's where all threading
belongs anyway.

I suppose one could also try to define a new IMAP threading extension
or to convince Mark Crispin to make radical changes to the one he's
been working on (and implementing in UW-IMAP) for 8 years, but I
suspect that those approaches would be high-effort and low-yield.

>I'm curious though, how do convince your Thunderbird and Dovecot to
>work like Gmail and keep everything threaded in the same folder in the
>first place?

Dovecot isn't really a player in that question, which is really about
how to make TBird do what you want it to do with messages and present
them in a particular organizational style.


--
Bill Cole
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Timo Sirainen
On May 7, 2008, at 5:59 PM, Bill Cole wrote:

> At 1:41 PM +0200 5/7/08, [hidden email] wrote:
>> Christian Jaeger wrote:
>>>  I'm looking for a way to manually break threads apart when  
>>> independent
>>> threads are mixed up, mostly by users which aren't aware that they  
>>> should
>>> not be replying to existing threads when starting a new one.
>>
>> Sounds like a great idea!
>
> The IMAP extension that supports threading (http://tools.ietf.org/html/draft-ietf-imapext-sort-20 
> ) has no support for clients 'breaking' threads.
Thunderbird and most other clients don't use server-side threading, so  
this doesn't matter there.

> I suppose one could also try to define a new IMAP threading  
> extension or to convince Mark Crispin to make radical changes to the  
> one he's been working on (and implementing in UW-IMAP) for 8 years,  
> but I suspect that those approaches would be high-effort and low-
> yield.

The existing THREAD=REFERENCES can't be changed, but new ones can be  
added. Dovecot v1.2 will have THREAD=X-REFERENCES2 which I'll probably  
try to get standardized.

It would be possible to make a new threading algorithm that started a  
new thread if a message had e.g. "newthread" keyword. But I don't  
really see it being worth the trouble.

>> I'm curious though, how do convince your Thunderbird and Dovecot to
>> work like Gmail and keep everything threaded in the same folder in  
>> the
>> first place?

Virtual mailboxes would do that trick. Dovecot v1.2 will have support  
for that, although many clients also support it already.

PGP.sig (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bugzilla from rosenfield.albert@gmail.com
In reply to this post by Bill Cole-4
Bill Cole:
> Dovecot isn't really a player in that question, which is really about how
> to make TBird do what you want it to do with messages and present
> them in a particular organizational style.

Well, Dovecot "deliver" puts the messages in place, and you want
incoming messages to land in the same folder as the conversation it is
already is in, and also have other virtual labels applied depending on
filter rules (eg dovecot-ml emails should have the "dovecot" label
applied).

Christian Jaeger:
> Or are you asking about the Gmail feature of not using subfolders but keep
> everything in the same place and only use search to find your stuff?

That's rather simplified - it's "using search", plus it's labels, an
"archive" button, a new threading interface, combining multiple
messages into one, etc.  But yeah.  I had assumed that you made your
TBird behave like that..

Timo Sirainen:
> Virtual mailboxes would do that trick. Dovecot v1.2 will have support
> for that, although many clients also support it already.

Awesome :-)..
Any more information somewhere?
Which clients?
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Timo Sirainen
On May 7, 2008, at 6:54 PM, [hidden email] wrote:
> Timo Sirainen:
>> Virtual mailboxes would do that trick. Dovecot v1.2 will have support
>> for that, although many clients also support it already.
>
> Awesome :-)..
> Any more information somewhere?

http://dovecot.org/list/dovecot/2008-March/029546.html

> Which clients?

Mail.app and Evolution at least. And doesn't Outlook also support them  
nowadays?


PGP.sig (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Charles Marcus
On 5/7/2008, Timo Sirainen ([hidden email]) wrote:
> Mail.app and Evolution at least. And doesn't Outlook also support
> them nowadays?

Thunderbird does - but the last time I tried to use them, the
performance was terrible when large folders were included in the search...

Its been a while though, so maybe it has gotten better...

--

Best regards,

Charles
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bugzilla from rosenfield.albert@gmail.com
In reply to this post by Timo Sirainen
> Mail.app and Evolution at least.
> And doesn't Outlook also support them nowadays?

Ah, you mean "saved search" containers in the tree view?
Thunderbird has these too.  Doesn't quite make it Gmail though.

I mean, it's a cute feature, but it does not modify the work flow
drastically enough to be even remotely useful as anything else than a
curiosity.  Gmail modifies, or rather, fixes the entire work flow ;-).
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Timo Sirainen
On May 7, 2008, at 7:41 PM, [hidden email] wrote:

>> Mail.app and Evolution at least.
>> And doesn't Outlook also support them nowadays?
>
> Ah, you mean "saved search" containers in the tree view?
> Thunderbird has these too.  Doesn't quite make it Gmail though.
>
> I mean, it's a cute feature, but it does not modify the work flow
> drastically enough to be even remotely useful as anything else than a
> curiosity.  Gmail modifies, or rather, fixes the entire work flow ;-).

I haven't used it myself, but .. Hmm. Wasn't it about showing entire  
threads of messages in INBOX? With Dovecot it would be done by:

mailboxes = INBOX "Sent Messages"
search = INTHREAD REFERENCES MAILBOX INBOX

So it matches entire threads of messages in INBOX. INTHREAD would need  
adding support for INTHREAD extension.

PGP.sig (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bill Cole-4
In reply to this post by Bugzilla from rosenfield.albert@gmail.com
At 5:54 PM +0200 5/7/08, [hidden email] wrote:
>Bill Cole:
[in regards to making TBird and Dovecot work like GMail]
>>  Dovecot isn't really a player in that question, which is really about how
>>  to make TBird do what you want it to do with messages and present
>>  them in a particular organizational style.
>
>Well, Dovecot "deliver" puts the messages in place, and you want
>incoming messages to land in the same folder as the conversation it is
>already is in, and also have other virtual labels applied depending on
>filter rules (eg dovecot-ml emails should have the "dovecot" label
>applied).

Yes, but 'deliver' does not itself make those decisions. If you want
mail delivered anywhere other than INBOX, you need to either call
deliver with a -m argument or use a plugin (e.g. sieve) and as far as
I can tell, only the sieve plugin can set IMAP message flags on
delivery. As for 'virtual labels,' I assume you are referring to
TBird's 'Tags' which map loosely to IMAP custom message flags,
(a.k.a. keywords.) The looseness is that TBird seems to ignore
keywords that it has not been explicitly configured to show, so if
sieve does add a keyword to a message, it has to be one known to
TBird in order to be used by TBird.

There has to be user-specific and MUA-specific configuration for the
usage model you seem to want. There really isn't much more that a
LDA/IMAP combination can do to support a GMail-like presentation,
that has to be done by the MUA, i.e TBird.

--
Bill Cole
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bugzilla from rosenfield.albert@gmail.com
In reply to this post by Timo Sirainen
> I haven't used it myself, but ..

If you're interested in the client part of mailing, perhaps create a
dummy account, join the dovecot and postfix mailing lists, create two
new filter rules that apply either "postfix" or "dovecot" labels and
archives (= unapplies "Inbox" label) messages depending on the to/cc
line, and try it out :-).

> Hmm. Wasn't it about showing entire
> threads of messages in INBOX?

Yes, or under any other label than "Inbox".

Also, the messages does not need to have the "Inbox" label to show up
under Inbox - they can have only the "Sent" label, and they will still
show up in the thread as long as they're part of it as per
In-Reply-To: or whatever.

Obviously useful since if you send an item ("Sent" label applied) and
receive an answer ("Inbox" label applied), the exact same conversation
will show up under both labels, and with the full list of messages
relevant to it.

> With Dovecot it would be done by:
>
>  mailboxes = INBOX "Sent Messages"
>  search = INTHREAD REFERENCES MAILBOX INBOX
>
>  So it matches entire threads of messages in INBOX.

So if I understand correctly, this will basically tell the IMAP client
*all* the messages relating to a given conversation, regardless of
whether they were sent or incoming?

(With the flip-side that software which expects to use different
folders, eg say a Horde webmail alternative for the account, will stop
working against this mailbox... Or is this all or none regarding which
mailbox are affected?)

> INTHREAD would need
> adding support for INTHREAD extension.

Ok, what does that do?
(Apologies for being in learning mode.)
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bugzilla from rosenfield.albert@gmail.com
In reply to this post by Bill Cole-4
> delivered anywhere other than INBOX, you need to either call deliver with a
> -m argument or use a plugin (e.g. sieve) and as far as I can tell, only the

Fair enough.

(Could never get sieve (cmusieve) to work with Dovecot - what does
other people use that's simple to set up and works efficiently?)

> labels,' I assume you are referring to TBird's 'Tags' which map loosely to
> IMAP custom message flags, (a.k.a. keywords.)

Sure, whatever.  The principal difference between the two, I guess, is
that a gmail label, when created, are automatically given an item in
the tree view, which when clicked shows a list of all threads where at
least one non-spam, non-trash mail item has the relevant label
applied.

(How to configure TBird to do something like this with Tags is either
impossible or beyond me, I'm afraid.)

> There has to be user-specific and MUA-specific configuration for the usage
> model you seem to want.

That's a fair enough response, I'm willing to accept that ;-).
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Timo Sirainen
In reply to this post by Bugzilla from rosenfield.albert@gmail.com
On Wed, 2008-05-07 at 19:49 +0200, [hidden email] wrote:
> > I haven't used it myself, but ..
>
> If you're interested in the client part of mailing, perhaps create a
> dummy account,

I've a test account which I've used a couple of times, but haven't
bothered too much with it.

> > Hmm. Wasn't it about showing entire
> > threads of messages in INBOX?
>
> Yes, or under any other label than "Inbox".
>
> Also, the messages does not need to have the "Inbox" label to show up
> under Inbox - they can have only the "Sent" label, and they will still
> show up in the thread as long as they're part of it as per
> In-Reply-To: or whatever.

Right.

> > With Dovecot it would be done by:
> >
> >  mailboxes = INBOX "Sent Messages"
> >  search = INTHREAD REFERENCES MAILBOX INBOX
> >
> >  So it matches entire threads of messages in INBOX.
>
> So if I understand correctly, this will basically tell the IMAP client
> *all* the messages relating to a given conversation, regardless of
> whether they were sent or incoming?
Yes, as long as at least one message in the thread exists in INBOX (the
"MAILBOX INBOX" search term).

> (With the flip-side that software which expects to use different
> folders, eg say a Horde webmail alternative for the account, will stop
> working against this mailbox... Or is this all or none regarding which
> mailbox are affected?)

I'm not sure what you mean. There are still different mailboxes. This
just creates virtual mailboxes on top of the existing ones.

> > INTHREAD would need
> > adding support for INTHREAD extension.
>
> Ok, what does that do?
> (Apologies for being in learning mode.)

Just makes the search match all messages in a thread if at least one
message in the thread matches the search terms.

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bugzilla from rosenfield.albert@gmail.com
> I've a test account which I've used a couple of times, but haven't
> bothered too much with it.

Guess it's not for everyone.

(Personally, I dislike that my email is stored at Googleplex, and that
I cannot fiddle with the server setup.  Otherwise, it's pretty darn
great.)

>  > > With Dovecot it would be done by:
>  > >
>  > >  mailboxes = INBOX "Sent Messages"
>  > >  search = INTHREAD REFERENCES MAILBOX INBOX
...
> Yes, as long as at least one message in the thread exists in INBOX
> (the "MAILBOX INBOX" search term).

So the above "mailboxes=, search=" is not a Dovecot configuration
option but a search performed by an IMAP client, right..  (learning
mode!)

> I'm not sure what you mean.

I just misunderstood something basic (see above).

> There are still different mailboxes. This
>  just creates virtual mailboxes on top of
> the existing ones.

"Creates" as in "the client has to keep track of them client-side, the
server knows nothing about them", right?

> [INTHREAD] just makes the search match all messages in a thread if at least
> one message in the thread matches the search terms.

Niice..  Can an unmodified MUA be expected to correctly fetch the
individual messages found with INTHREAD?  Or does it need to "switch
folders" or perform other IMAP magic.
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Timo Sirainen
On Wed, 2008-05-07 at 20:13 +0200, [hidden email] wrote:

> >  > > With Dovecot it would be done by:
> >  > >
> >  > >  mailboxes = INBOX "Sent Messages"
> >  > >  search = INTHREAD REFERENCES MAILBOX INBOX
> ...
> > Yes, as long as at least one message in the thread exists in INBOX
> > (the "MAILBOX INBOX" search term).
>
> So the above "mailboxes=, search=" is not a Dovecot configuration
> option but a search performed by an IMAP client, right..  (learning
> mode!)
No, I did meant that they were Dovecot configuration options that
describe a single virtual mailbox (although the actual configuration
format is currently a bit different).

> > There are still different mailboxes. This
> >  just creates virtual mailboxes on top of
> > the existing ones.
>
> "Creates" as in "the client has to keep track of them client-side, the
> server knows nothing about them", right?

Clients could of course do this also, but I can't really do anything
about them.

> > [INTHREAD] just makes the search match all messages in a thread if at least
> > one message in the thread matches the search terms.
>
> Niice..  Can an unmodified MUA be expected to correctly fetch the
> individual messages found with INTHREAD?  Or does it need to "switch
> folders" or perform other IMAP magic.

The configured virtual mailbox would look exactly like a normal mailbox
to the client.

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bugzilla from rosenfield.albert@gmail.com
> The configured virtual mailbox would look exactly
> like a normal mailbox  to the client.

The advantage being that Dovecot would index messages in all physical
folders for super-fast access when asking via IMAP for messages
relating to a specific thread-id.  (Hope I got this right.)

In case I wanted to modify the MUA to support better threading and
tags/labels, is that enough?  Can I simply tell Dovecot via IMAP to
create a new virtual folder that aggregates virtual messages from all
the other folders and it will start indexing them for thread-locality?

(Would be great, because it's always easier when you only have to
change either the MUA or the "post office" side of things.)

Is it possible to ask Dovecot via IMAP to look for other things in the
virtual folder, for example all items with the flag "huey lewis"
applied?  Is this sort of query fast / indexed?
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Timo Sirainen
On Wed, 2008-05-07 at 20:37 +0200, [hidden email] wrote:
> > The configured virtual mailbox would look exactly
> > like a normal mailbox  to the client.
>
> The advantage being that Dovecot would index messages in all physical
> folders for super-fast access when asking via IMAP for messages
> relating to a specific thread-id.  (Hope I got this right.)

That could be one use case, but since no clients do that I doubt it's a
very useful one for a while. :) I'm mostly thinking about using virtual
mailboxes for custom webmails, at least initially.

> In case I wanted to modify the MUA to support better threading and
> tags/labels, is that enough?  Can I simply tell Dovecot via IMAP to
> create a new virtual folder that aggregates virtual messages from all
> the other folders and it will start indexing them for thread-locality?

Currently only admin can configure them, but it would be possible to
create a new extension for configuring them via IMAP. There are already
a couple drafts related to it.

> Is it possible to ask Dovecot via IMAP to look for other things in the
> virtual folder, for example all items with the flag "huey lewis"
> applied?  

Yes (other than spaces not being allowed in flags). It's built using a
fully functional IMAP SEARCH query.

> Is this sort of query fast / indexed?

Current implementation is a bit slow, but in future it will be fast.

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Manual thread breaking from Thunderbird

Bugzilla from rosenfield.albert@gmail.com
> That could be one use case, but since no clients do that I doubt it's a
>  very useful one for a while. :) I'm mostly thinking about using virtual
>  mailboxes for custom webmails, at least initially.

;-)..
I find it to be a problem that you're the one who decides what it's
useful for and what it's not :-P.

>  > Is it possible to ask Dovecot via IMAP to look for other things in the
>  > virtual folder, for example all items with the flag "huey lewis"
>  > applied?
>
> Yes (other than spaces not being allowed in flags). It's built using a
>  fully functional IMAP SEARCH query.
>
>  > Is this sort of query fast / indexed?
>
> Current implementation is a bit slow, but in future it will be fast.

So at what point will it be possible to have Dovecot index whatever
the client asks it to index? :-)
12