nginx mailing-list and sender filtering (vs BATV)

[Maxim Dounin]

Hello!

On Fri, Mar 01, 2013 at 03:59:59PM -0500, Phil Pennock wrote:

> On 2013-03-01 at 17:12 +0400, Maxim Dounin wrote:
> > On Fri, Mar 01, 2013 at 03:22:51AM -0500, Phil Pennock wrote:
> > > PS: nginx mail-server configuration is broken; it's checking SMTP Envelope
> > >     Sender against the subscription list, not the RFC5322.From: header, so
> > >     breaks on things such as PRVS.  Posting via manual injection to your
> > >     mail-server.  :(
> > 
> > Unfortunately, there is no way to properly reject messages at SMTP 
> > level (i.e. to avoid sending bounces) and doing checks based on 
> > message headers at the same time.
> > 
> > If you use different envelope from and message from addresses and 
> > have problems with posting - just subscribe your envelope from 
> > address to the mailing list with mail delivery disabled.
> 
> I understand the problem you're fighting here, and why you're doing this
> at SMTP RCPT time, since Mailman doesn't have content scanning hooks to
> check if the message should be allowed based on the message headers.
> You can do these checks safely enough, but it requires more caution.

You probably didn't understand the problem deep enough: content 
scanning hooks, even if implemented, won't help.  To properly 
reject message at SMTP level you have to check envelope sender, 
and if you've accepted RCPT TO - it's too late to reject message 
at DATA stage, as the message might have other valid recipients.

So the only way to properly check list membership is to check 
envelope addresses.  Anything else means sending bounces, which is 
not acceptable nowadays.

> When violating normal SMTP expectations by making a RCPT appear to only
> exist for certain MAIL FROM senders, it's important to understand
> variations in senders at SMTP time: the checks you're doing are not the
> same as the membership tests done by Mailman itself, which looks at the
> headers.
> 
> I did the same thing as you, for expediency and to avoid forking yet
> more extra processes for scanning, but I made sure that the form of the
> address being checked for membership has had VERP and BATV variations
> stripped out first, to check a _normalized_ address against the Mailman
> membership roster.
> 
> BATV changes the SMTP Envelope Sender, with a crypto-hash embedded in
> the address, and a secret and a daily timestamp going into the hash
> inputs, so that if all messages _from_ a domain are sent with BATV, then
> bounces inherently *must* be to BATV targets if they're legitimate.
> 
> This is the only tool that prevents joe-job backscatter from flooding
> mailboxes.
> 
> So that's a non-standard address-existence test breaking when exposed to
> an address variation that does have an Internic draft, albeit expired.
> 
> I've sucked it up and configured up an exception mechanism, adding this
> mailing-list to that, accepting that any time I enable the backscatter
> filter, I'll lose bounce messages from this list to me, with rejections
> dropping into a blackhole.  That's got a lower risk of being triggered
> than a joe-job (unfortunately) (and this varies depending on your
> involvement with email infrastructure and how much spammers dislike
> you).
> 
> Next time you're touching your mailserver setup, could you please take a
> look at adding a canonicalisation step to the addresses being checked
> against list membership?

I personally think that BATV is awful, but normalization shouldn't 
make things worse and probably worth implementing.

-- 
Maxim Dounin
http://nginx.org/en/donation.html