Lua Auth

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Lua Auth

Mark Moseley
Just happened to be surfing the docs and saw this. This is beyond awesome:

https://wiki2.dovecot.org/AuthDatabase/Lua

Any words of wisdom on using it? I'd be putting a bunch of mysql logic in
it. Any horrible gotchas there? When it says 'blocking', should I assume
that means that a auth worker process will *not* accept any new auth
lookups until both auth_passdb_lookup() and auth_userdb_lookup() have
completed (in which I'd be doing several mysql calls)? If that's the case,
I assume that the number of auth workers should be bumped up.

And is a 2.3 release fairly imminent?
Reply | Threaded
Open this post in threaded view
|

Re: Lua Auth

Aki Tuomi-2

> On November 29, 2017 at 4:37 AM Mark Moseley <[hidden email]> wrote:
>
>
> Just happened to be surfing the docs and saw this. This is beyond awesome:
>
> https://wiki2.dovecot.org/AuthDatabase/Lua
>
> Any words of wisdom on using it? I'd be putting a bunch of mysql logic in
> it. Any horrible gotchas there? When it says 'blocking', should I assume
> that means that a auth worker process will *not* accept any new auth
> lookups until both auth_passdb_lookup() and auth_userdb_lookup() have
> completed (in which I'd be doing several mysql calls)? If that's the case,
> I assume that the number of auth workers should be bumped up.
>
> And is a 2.3 release fairly imminent?

Hi!

This feature was added very recently, and there is very little operational experience on it. As the docs should say, blocking=yes means that an auth worker is used, and yes, it will block each auth worker during authentication, but what we tried, it should perform rather nicely.

The most important gotcha is to always test your lua code rigorously, because there is not much we can do to save you.

It should be present in master branch, so if someone feels like trying it out, please let us know if you find any bugs or strangeness. It's not present in nightlies yet.

We are planning on releasing 2.3.0 this year.

Aki
Reply | Threaded
Open this post in threaded view
|

Re: Lua Auth

Stephan Bosch


Op 29-11-2017 om 6:17 schreef Aki Tuomi:

>> On November 29, 2017 at 4:37 AM Mark Moseley <[hidden email]> wrote:
>>
>>
>> Just happened to be surfing the docs and saw this. This is beyond awesome:
>>
>> https://wiki2.dovecot.org/AuthDatabase/Lua
>>
>> Any words of wisdom on using it? I'd be putting a bunch of mysql logic in
>> it. Any horrible gotchas there? When it says 'blocking', should I assume
>> that means that a auth worker process will *not* accept any new auth
>> lookups until both auth_passdb_lookup() and auth_userdb_lookup() have
>> completed (in which I'd be doing several mysql calls)? If that's the case,
>> I assume that the number of auth workers should be bumped up.
>>
>> And is a 2.3 release fairly imminent?
> Hi!
>
> This feature was added very recently, and there is very little operational experience on it. As the docs should say, blocking=yes means that an auth worker is used, and yes, it will block each auth worker during authentication, but what we tried, it should perform rather nicely.
>
> The most important gotcha is to always test your lua code rigorously, because there is not much we can do to save you.
>
> It should be present in master branch, so if someone feels like trying it out, please let us know if you find any bugs or strangeness. It's not present in nightlies yet.
>
> We are planning on releasing 2.3.0 this year.

The Xi package builder has this feature enabled since yesterday. It is
available in the dovecot-lua package; the first Xi package that doesn't
have an official Debian equivalent (yet anyway).

Regards,

Stephan.
Reply | Threaded
Open this post in threaded view
|

Re: Lua Auth

Mark Moseley
On Thu, Nov 30, 2017 at 5:26 AM, Stephan Bosch <[hidden email]> wrote:

>
>
> Op 29-11-2017 om 6:17 schreef Aki Tuomi:
>
>> On November 29, 2017 at 4:37 AM Mark Moseley <[hidden email]>
>>> wrote:
>>>
>>>
>>> Just happened to be surfing the docs and saw this. This is beyond
>>> awesome:
>>>
>>> https://wiki2.dovecot.org/AuthDatabase/Lua
>>>
>>> Any words of wisdom on using it? I'd be putting a bunch of mysql logic in
>>> it. Any horrible gotchas there? When it says 'blocking', should I assume
>>> that means that a auth worker process will *not* accept any new auth
>>> lookups until both auth_passdb_lookup() and auth_userdb_lookup() have
>>> completed (in which I'd be doing several mysql calls)? If that's the
>>> case,
>>> I assume that the number of auth workers should be bumped up.
>>>
>>> And is a 2.3 release fairly imminent?
>>>
>> Hi!
>>
>> This feature was added very recently, and there is very little
>> operational experience on it. As the docs should say, blocking=yes means
>> that an auth worker is used, and yes, it will block each auth worker during
>> authentication, but what we tried, it should perform rather nicely.
>>
>> The most important gotcha is to always test your lua code rigorously,
>> because there is not much we can do to save you.
>>
>> It should be present in master branch, so if someone feels like trying it
>> out, please let us know if you find any bugs or strangeness. It's not
>> present in nightlies yet.
>>
>> We are planning on releasing 2.3.0 this year.
>>
>
> The Xi package builder has this feature enabled since yesterday. It is
> available in the dovecot-lua package; the first Xi package that doesn't
> have an official Debian equivalent (yet anyway).
>
>
>
I've been playing with Lua auth and so far no issues. I was previously
putting together a very ugly MySQL stored procedure. Using Lua would be a
lot easier (esp when it comes to returning an arbitrary number of columns).

I'd love to see any test Lua code that the dovecot team has been playing
around with (and realize it's not remotely production-ready, so don't worry
about caveats

I did have a couple of questions though:

1) Is the data returned by Lua auth not cacheable? I've got the following
settings (and I'm just using Lua in the userdb lookup, not passdb -- passdb
is doing a lightweight SQL lookup for username/password):

auth_cache_negative_ttl = 1 mins
auth_cache_size = 10 M
auth_cache_ttl = 10 mins

but I notice that every time I auth, it'll redo all the queries in my Lua
code. I'd have expected that data to be served out of cache till the 10min
TTL is up


2) Is there an appropriate way to return data with spaces in it (or
presumably other non-alphanum chars. My quota name had a space in it, which
somehow got interpreted as 'yes' , i.e.:

imap: Error: Failed to initialize quota: Invalid quota root quota: Unknown
quota backend: yes

I simply changed the space to an underscore as a workaround, but I'm
curious if there's a better way. I tried various quoting without success.
Didn't try escaping yet.


3) Can you elaborate on the "auth_request#response_from_template(template)"
and "auth_request#var_expand(template)" functions? Specifically how to use
them. I'm guessing that I could've used one of them to work around #2 (that
it would have done the escaping for me)


Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: Lua Auth

Aki Tuomi-2
In reply to this post by Mark Moseley

-------- Original message --------From: Mark Moseley <[hidden email]> Date: 02/12/2017  02:34  (GMT+02:00) To:  Cc: Dovecot Mailing List <[hidden email]> Subject: Re: Lua Auth
On Thu, Nov 30, 2017 at 5:26 AM, Stephan Bosch <[hidden email]> wrote:

>
>
> Op 29-11-2017 om 6:17 schreef Aki Tuomi:
>
>> On November 29, 2017 at 4:37 AM Mark Moseley <[hidden email]>
>>> wrote:
>>>
>>>
>>> Just happened to be surfing the docs and saw this. This is beyond
>>> awesome:
>>>
>>> https://wiki2.dovecot.org/AuthDatabase/Lua
>>>
>>> Any words of wisdom on using it? I'd be putting a bunch of mysql logic in
>>> it. Any horrible gotchas there? When it says 'blocking', should I assume
>>> that means that a auth worker process will *not* accept any new auth
>>> lookups until both auth_passdb_lookup() and auth_userdb_lookup() have
>>> completed (in which I'd be doing several mysql calls)? If that's the
>>> case,
>>> I assume that the number of auth workers should be bumped up.
>>>
>>> And is a 2.3 release fairly imminent?
>>>
>> Hi!
>>
>> This feature was added very recently, and there is very little
>> operational experience on it. As the docs should say, blocking=yes means
>> that an auth worker is used, and yes, it will block each auth worker during
>> authentication, but what we tried, it should perform rather nicely.
>>
>> The most important gotcha is to always test your lua code rigorously,
>> because there is not much we can do to save you.
>>
>> It should be present in master branch, so if someone feels like trying it
>> out, please let us know if you find any bugs or strangeness. It's not
>> present in nightlies yet.
>>
>> We are planning on releasing 2.3.0 this year.
>>
>
> The Xi package builder has this feature enabled since yesterday. It is
> available in the dovecot-lua package; the first Xi package that doesn't
> have an official Debian equivalent (yet anyway).
>
>
>
I've been playing with Lua auth and so far no issues. I was previously
putting together a very ugly MySQL stored procedure. Using Lua would be a
lot easier (esp when it comes to returning an arbitrary number of columns).

I'd love to see any test Lua code that the dovecot team has been playing
around with (and realize it's not remotely production-ready, so don't worry
about caveats

I did have a couple of questions though:

1) Is the data returned by Lua auth not cacheable? I've got the following
settings (and I'm just using Lua in the userdb lookup, not passdb -- passdb
is doing a lightweight SQL lookup for username/password):

auth_cache_negative_ttl = 1 mins
auth_cache_size = 10 M
auth_cache_ttl = 10 mins

but I notice that every time I auth, it'll redo all the queries in my Lua
code. I'd have expected that data to be served out of cache till the 10min
TTL is up


2) Is there an appropriate way to return data with spaces in it (or
presumably other non-alphanum chars. My quota name had a space in it, which
somehow got interpreted as 'yes' , i.e.:

imap: Error: Failed to initialize quota: Invalid quota root quota: Unknown
quota backend: yes

I simply changed the space to an underscore as a workaround, but I'm
curious if there's a better way. I tried various quoting without success.
Didn't try escaping yet.


3) Can you elaborate on the "auth_request#response_from_template(template)"
and "auth_request#var_expand(template)" functions? Specifically how to use
them. I'm guessing that I could've used one of them to work around #2 (that
it would have done the escaping for me)


Thanks!

-----

1) I'll have to check this.
 

  2) Instead of string, return a key value table. you can have spaces in values.
 

  3) response_from_template expands a key=value string into table by var expanding values. 
 

  var_expand can be used to interpolation for any purposes. it returns a string. see https://wiki.dovecot.org/Variables for details on how to use it.
 

  Individual variable access is more efficient to do directly.
 

  ---
Aki Tuomi