imap proxy and untagged commands

[Maxim Dounin]

Hello!

On Thu, Mar 14, 2013 at 06:25:55PM +0200, Timo Sirainen wrote:

> Hi,
> 
> On Wed, 2013-03-13 at 18:48 +0400, Maxim Dounin wrote:
> 
> > > In real world Dovecot can already send some extra untagged replies if
> > > there is some trouble (e.g. heavy load) with its authentication process.
> > > What happens is:
> > > 
> > > S: * OK Waiting for authentication process to respond..
> > > C: a LOGIN {3}
> > > S: * OK Waiting for authentication process to respond..
> > > 
> > > and nginx fails:
> > > 
> > > 2013/03/13 15:38:55 [error] 7257#0: *15 upstream sent invalid response:
> > > "* OK Waiting for authentication process to respond.." while reading
> > > response from upstream, client: 127.0.0.1, server: 0.0.0.0:10143, login:
> > > "tss", upstream: 127.0.0.1:143
> > > 
> > > I attempted to fix this, but it looks like the current code doesn't make
> > > this very easy. Especially since it seems to be handling data one "read
> > > block" at a time, which can contain multiple lines, each of which should
> > > be handled separately (the untagged and non-untagged replies may arrive
> > > in the same IP packet).
> > 
> > Yes, thanks, it's a known issue - untagged responses are not 
> > handled properly (and there is a similar issue with SMTP multiline 
> > replies, quick and dirty patch at [1]).  This isn't considered as 
> > a major problem since nginx is expected to work with controlled 
> > IMAP backends, and it's usually trivial to avoid such untagged 
> > responses.  It would be good to fix it though.
> 
> The other reason why I was hoping for this to get fixed was to be able
> to add some new functionality to nginx more easily and efficiently.
> Things that I'd like to have in nginx proxy, in the order of importance:
> 
> 1. If IMAP banner has [CAPABILITY .. ID ..], then send to the server:
> 
> a ID ("x-originating-ip" "1.2.3.4" "x-originating-port" "12345"
> "x-connected-ip" "4.3.2.1" "x-connected-port" "993")
> 
> That could be pipelined directly to server with the LOGIN command
> without first waiting for server to reply. Except then the server could
> send both replies in one IP packet.. But yeah, it could be implemented
> without pipelining also.
> 
> The command can also send "x-session-id" to give the session a unique ID
> (in Dovecot unique for 9 years). If each log line related to the session
> has this ID logged, it makes it easier to track the session through the
> proxy/backend logs.
> 
> 2. If POP3 banner has [XCLIENT], send "XCLIENT ADDR=1.2.3.4 PORT=12345"
> to the server. (And again maybe some other stuff.)
> 
> 3. After logging in to backend server issue CAPABILITY command once
> more, and send its result to the client. This is only needed/useful if
> the client asked for CAPABILITY command before login. Anyway this allows
> nginx to advertise only minimal capabilities and have client update them
> automatically after login. (Dovecot v2.x does this too and it works with
> all the major clients.)
> 
> 4. Authentication could support master users and have nginx log in using
> AUTHENTICATE PLAIN.
> 
> 5. If backend server supports LITERAL+, the LOGIN command could use
> LITERAL+ to avoid waiting for server. nginx could also support LITERAL+
> itself.
> 
> 6. nginx could support SASL-IR extension.
> 
> 7. nginx could support ID extension itself also by remembering what the
> client sent, and sending it to the backend server during login.
> 
> So .. would anyone want to help with any of this? :) I think these would
> make my life easier in future, so I'm planning on implementing at least
> some of them. (In some projects nginx makes SSL configuration easier
> since it can be used for proxying all the imap/pop3/smtp/http traffic
> with one config.)

Overall this looks interesting.

I likely will commit some of my mail-related patches in the near 
future (including the one referenced), I think it will improve 
things a bit and may make further progress a bit easier.

Unfortunately, I'm a bit busy and unlikely will be able to 
actively help with writing code, but feel free to ask question if 
you have any and/or submit patches for review.

-- 
Maxim Dounin
http://nginx.org/en/donation.html