How to limit Apple Mail (desktop)?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

How to limit Apple Mail (desktop)?

Rupert Gallagher
By default, Apple Mail downloads all e-mails from  server's account. Previous versions of this client allowed to opt-out. The latest two versions? however, only allow to opt-out from downloading the attachments.

The stress on the server is unbearable. We cannot ask users to be considerate: this is the default behaviour of Apple Mail.

We need a server-side solution to the problem.

Please share your ideas.
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Robert Schetterer-2
Am 30.10.2017 um 10:38 schrieb Rupert Gallagher:
> By default, Apple Mail downloads all e-mails from  server's account. Previous versions of this client allowed to opt-out. The latest two versions? however, only allow to opt-out from downloading the attachments.
>
> The stress on the server is unbearable. We cannot ask users to be considerate: this is the default behaviour of Apple Mail.
>
> We need a server-side solution to the problem.
>
> Please share your ideas.
>

first check if you can ident Apple Mail versions related, i recent have
none in my  log , but i.e Android does

---log
ID sent: name=com.samsung.android.email.provider, os=android,
os-version=7.0; NRD90M, vendor=samsung, x-android-device-model=SM-G930F
-----

then you need a procedure for limit....i have no idea which one

Best Regards
MfG Robert Schetterer

--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Robert Schetterer-2
Am 30.10.2017 um 17:50 schrieb Robert Schetterer:

> Am 30.10.2017 um 10:38 schrieb Rupert Gallagher:
>> By default, Apple Mail downloads all e-mails from  server's account. Previous versions of this client allowed to opt-out. The latest two versions? however, only allow to opt-out from downloading the attachments.
>>
>> The stress on the server is unbearable. We cannot ask users to be considerate: this is the default behaviour of Apple Mail.
>>
>> We need a server-side solution to the problem.
>>
>> Please share your ideas.
>>
>
> first check if you can ident Apple Mail versions related, i recent have
> none in my  log , but i.e Android does
>
> ---log
> ID sent: name=com.samsung.android.email.provider, os=android,
> os-version=7.0; NRD90M, vendor=samsung, x-android-device-model=SM-G930F
> -----
>
> then you need a procedure for limit....i have no idea which one
>
> Best Regards
> MfG Robert Schetterer
>

I dont know Apple Mail very good, but as workaround you might use sieve
to presort mails on the server at incomming in (sub)folders which arent
synced at default i.e in a date named folder, for sure users ( you
should know your apple mail users ) then need configure an extra
subscribe on these folders. Perhaps a combi with virtual folders may
usefull , for now not better idea , iam nearly sure this is not what you
you expected and/or wanted

seems google has a feature to "hide" mails

see

https://www.guidingtech.com/44581/prevent-mail-app-space-mac/

however if feel very strange with this

Best Regards
MfG Robert Schetterer

--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Rupert Gallagher
In reply to this post by Rupert Gallagher
When Apple Mail connects to an IMAP account for the very first time, it downloads all e-mails to build a local mirror.

When the user changes the name of a folder, Apple Mail downloads the whole subtree and erases the old one.

We have Apple Mail users with >20GB worth of e-mails, downloaded multiple times (horrified emoticon here).

Sent from ProtonMail Mobile

On Mon, Oct 30, 2017 at 10:38 AM, Rupert Gallagher <[hidden email]> wrote:

> By default, Apple Mail downloads all e-mails from  server's account. Previous versions of this client allowed to opt-out. The latest two versions? however, only allow to opt-out from downloading the attachments.
>
> The stress on the server is unbearable. We cannot ask users to be considerate: this is the default behaviour of Apple Mail.
>
> We need a server-side solution to the problem.
>
> Please share your ideas.
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Sami Ketola

> On 31 Oct 2017, at 8.42, Rupert Gallagher <[hidden email]> wrote:
>
> When Apple Mail connects to an IMAP account for the very first time, it downloads all e-mails to build a local mirror.
>
> When the user changes the name of a folder, Apple Mail downloads the whole subtree and erases the old one.
>
> We have Apple Mail users with >20GB worth of e-mails, downloaded multiple times (horrified emoticon here).
>
> Sent from ProtonMail Mobile


I can’t think of any easy way to rate limit Apple mail. Only thing that crossed my mind is to enable ID response
logging and then with some logwatch process use some traffic shaping solution to limit the bandwith of that particular
connection after some period of time.

Sami
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Moritz Augsburger
In reply to this post by Rupert Gallagher
Hi,

On 30.10.2017 10:38, Rupert Gallagher wrote:
> We need a server-side solution to the problem.

more powerfull hardware?

What's in your mind as solution?

Don't tell the client about the mails -> user will miss old mails.

Stop him from fetching body with a temporary "UNAVAILABLE" failure? ->
Don't know how apple mail will react, but probably present some error to
the user for every mail.

The only possibility that would make sense to me is some form of rate
limiting to reduce the server load. Give them some tens of Megabytes w/o
any restriction, then reduce.

Regards
Moritz
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Rupert Gallagher
> What's in your mind as solution?

When dovecot receives many full body downloads from a client, it could respond by sending the header only.

Sent from ProtonMail Mobile

On Tue, Oct 31, 2017 at 11:29 AM, <[hidden email]> wrote:

> Hi, On 30.10.2017 10:38, Rupert Gallagher wrote: > We need a server-side solution to the problem. more powerfull hardware? What's in your mind as solution? Don't tell the client about the mails -> user will miss old mails. Stop him from fetching body with a temporary "UNAVAILABLE" failure? -> Don't know how apple mail will react, but probably present some error to the user for every mail. The only possibility that would make sense to me is some form of rate limiting to reduce the server load. Give them some tens of Megabytes w/o any restriction, then reduce. Regards Moritz
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Aki Tuomi-2


On 31.10.2017 14:44, Rupert Gallagher wrote:
>> What's in your mind as solution?
> When dovecot receives many full body downloads from a client, it could respond by sending the header only.
This sounds rather dangerous. Client is expecting full body download,
not headers.

Aki

> Sent from ProtonMail Mobile
>
> On Tue, Oct 31, 2017 at 11:29 AM, <[hidden email]> wrote:
>
>> Hi, On 30.10.2017 10:38, Rupert Gallagher wrote: > We need a server-side solution to the problem. more powerfull hardware? What's in your mind as solution? Don't tell the client about the mails -> user will miss old mails. Stop him from fetching body with a temporary "UNAVAILABLE" failure? -> Don't know how apple mail will react, but probably present some error to the user for every mail. The only possibility that would make sense to me is some form of rate limiting to reduce the server load. Give them some tens of Megabytes w/o any restriction, then reduce. Regards Moritz
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Sami Ketola
In reply to this post by Rupert Gallagher


> On 31 Oct 2017, at 14.44, Rupert Gallagher <[hidden email]> wrote:
>
>> What's in your mind as solution?
>
> When dovecot receives many full body downloads from a client, it could respond by sending the header only.


That would not be nice. Breaking IMAP protocol that way. Some rate limiting would be better solution.

Sami
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

María Arrea
In reply to this post by Rupert Gallagher

    If you use Linux you can use netfilter (iptables) mark feature to
track individual connections from a given IP, and above a threshold you
can shape the bandwitdh based on those iptables mark. For example, if an
IMAP connection from ip X exceed 100 megabytes of data, bandwitdh for
that ip would be limited to 100 KB/sec (or whatever number). First 100
megabytes, top speed, above that they get slower.

     Regards

     María


El 30/10/17 a las 10:38, Rupert Gallagher escribió:
> By default, Apple Mail downloads all e-mails from  server's account. Previous versions of this client allowed to opt-out. The latest two versions? however, only allow to opt-out from downloading the attachments.
>
> The stress on the server is unbearable. We cannot ask users to be considerate: this is the default behaviour of Apple Mail.
>
> We need a server-side solution to the problem.
>
> Please share your ideas.
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Rupert Gallagher
In reply to this post by Aki Tuomi-2
Aki, the IMAP client can receive the e-mails with an empty body without any damage. This is how IMAP works normally. The full body is queried again by the client when reading the e-mail for real.

On Tue, Oct 31, 2017 at 1:52 PM, Aki Tuomi <[hidden email]> wrote:

>>> What's in your mind as solution?

>> When dovecot receives many full body downloads from a client, it could respond by sending the header only.

> This sounds rather dangerous. Client is expecting full body download, not headers. Aki
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Robert Schetterer-2
In reply to this post by María Arrea
Am 31.10.2017 um 14:06 schrieb María Arrea:
>
>    If you use Linux you can use netfilter (iptables) mark feature to
> track individual connections from a given IP, and above a threshold you
> can shape the bandwitdh based on those iptables mark. For example, if an
> IMAP connection from ip X exceed 100 megabytes of data, bandwitdh for
> that ip would be limited to 100 KB/sec (or whatever number). First 100
> megabytes, top speed, above that they get slower.

dont think this will work
tons of cons can come from a nat ip
so you would punish everyone behind that ip ,not only apple mail

i see no real solution for the problem, cause if the server works out
some slow down this would only lead to longer download times, but will
no solve the orig wanted task. It will all be workarounds for a "buggy"
client




>
>     Regards
>
>     María
>
>
> El 30/10/17 a las 10:38, Rupert Gallagher escribió:
>> By default, Apple Mail downloads all e-mails from  server's account.
>> Previous versions of this client allowed to opt-out. The latest two
>> versions? however, only allow to opt-out from downloading the
>> attachments.
>>
>> The stress on the server is unbearable. We cannot ask users to be
>> considerate: this is the default behaviour of Apple Mail.
>>
>> We need a server-side solution to the problem.
>>
>> Please share your ideas.



Best Regards
MfG Robert Schetterer

--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64
Schleißheimer Straße 26/MG, 80333 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Rupert Gallagher
In reply to this post by Rupert Gallagher
I think I am not mistaken in saying that the original purpose of IMAP was to access a remote database, vs POP's approach to download it. When the IMAP client opens a folder, the server uploads the *index* of its content. When the client requests a specific item, the server uploads the item while keeping the original. If the client renames a folder, the server just renames the folder.

Apple Mail behaves like a POP client that wants to download everything while keeping the db on the server. Further down Apple's madness, if the client renames a folder, Apple mail asks the server to upload the full content of the "new" folder. Batshit crazy!

The very best therapy here is, in my opinion, to serve Apple Mail's request for mass download as if it were a regular IMAP index request, if technically feasible. If it is not feasible, then the alternative is not to rate limit the connection, but to ban Apple Mail entirely.

Sent from ProtonMail Mobile

On Tue, Oct 31, 2017 at 2:46 PM, Rupert Gallagher <[hidden email]> wrote:

> Aki, the IMAP client can receive the e-mails with an empty body without any damage. This is how IMAP works normally. The full body is queried again by the client when reading the e-mail for real.
>
> On Tue, Oct 31, 2017 at 1:52 PM, Aki Tuomi <[hidden email]> wrote:
>
>>>> What's in your mind as solution?
>
>>> When dovecot receives many full body downloads from a client, it could respond by sending the header only.
>
>> This sounds rather dangerous. Client is expecting full body download, not headers. Aki
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Joseph Tam-2
In reply to this post by Rupert Gallagher
Rupert Gallagher writes:

> By default, Apple Mail downloads all e-mails from? server's account.
> Previous versions of this client allowed to opt-out.  The latest two
> versions? however, only allow to opt-out from downloading the
> attachments.
>
> The stress on the server is unbearable.  We cannot ask users to be
> considerate: this is the default behaviour of Apple Mail.

(I'm coming into this thread late.)

I'm having a hard time visualizing the cause of your problem.  Most big
messages have the bulk of their data as attachments (which you state the
mail reader can defer downloading).  Also, readers usually cache local
copies, so they're downloaded once.

So what the composition of all this traffic?  Are you saying the mail
client is ultra dumb and repeatedly downloading entire messages, read
and unread, attachment and all (i.e. you're truly bandwidth limited?)
Do your users typically have many concurrent readers for the same mailbox?
What kind of session numbers are you supporting? 100s? 1000s? 10K+?

Your data outflow has an incoming counterpart: if you're having a
hard time serving it out, is your MTA also having a hard time with the
incoming volume?  Can it simply be the case you've underscaled your
remote mail service?

Joseph Tam <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

@lbutlr
On Oct 31, 2017, at 5:01 PM, Joseph Tam <[hidden email]> wrote:

> Rupert Gallagher writes:
>
>> By default, Apple Mail downloads all e-mails from? server's account.
>> Previous versions of this client allowed to opt-out.  The latest two
>> versions? however, only allow to opt-out from downloading the
>> attachments. The stress on the server is unbearable.  We cannot ask users to be
>> considerate: this is the default behaviour of Apple Mail.
>
> (I'm coming into this thread late.)
>
> I'm having a hard time visualizing the cause of your problem.  Most big messages have the bulk of their data as attachments (which you state the mail reader can defer downloading).  Also, readers usually cache local copies, so they're downloaded once.

Yep.

> So what the composition of all this traffic?  Are you saying the mail client is ultra dumb and repeatedly downloading entire messages, read and unread, attachment and all (i.e. you're truly bandwidth limited?)

It most certainly does not do that.

> Do your users typically have many concurrent readers for the same mailbox?

Most of my users have 3 devices, a desktop, an iPhone, and an iPad. It doesn't seem to cause any issues.

--
Apple broke AppleScripting signatures in Mail.app, so no random signatures.
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

Joseph Tam-2
In reply to this post by Rupert Gallagher
"@lbutlr" <[hidden email]> writes:

(Are you the OP, or have I mistakenly atributed this to Rupert Gallagher?)

>> So what the composition of all this traffic? Are you saying the mail
>> client is ultra dumb and repeatedly downloading entire messages, read
>> and unread, attachment and all (i.e.  you're truly bandwidth limited?)
>
> It most certainly does not do that.

If this is the case for the OP, it's back to square one to find log
entries, and/or nfsstat/iostat/vmstat/etc output to see what is "killing"
the server.  File locking issues, mail_max_userip_connections and process
limit values could also cause service flakiness.  Apple mail readers
seems to gobble up IMAP connections like candy.

Joseph Tam <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: How to limit Apple Mail (desktop)?

María Arrea
In reply to this post by Robert Schetterer-2
El 31/10/17 a las 15:30, Robert Schetterer escribió:

> Am 31.10.2017 um 14:06 schrieb María Arrea:
>>     If you use Linux you can use netfilter (iptables) mark feature to
>> track individual connections from a given IP, and above a threshold you
>> can shape the bandwitdh based on those iptables mark. For example, if an
>> IMAP connection from ip X exceed 100 megabytes of data, bandwitdh for
>> that ip would be limited to 100 KB/sec (or whatever number). First 100
>> megabytes, top speed, above that they get slower.
> dont think this will work
> tons of cons can come from a nat ip
> so you would punish everyone behind that ip ,not only apple mail

Not true. You set a connection tracking using ip+port, not only IP.

>
> i see no real solution for the problem, cause if the server works out
> some slow down this would only lead to longer download times, but will
> no solve the orig wanted task. It will all be workarounds for a "buggy"
> client
>

     Original post talked about "unbearable server load". Shapping
network traffic will definitely help with that.


     María.