Denying illegal host headers blocks bots sometimes

[Grozdan]

Hi,

I've set my conf to deny illegal host headers, as per below option.
However, I've noticed that bots from Google, Bing, Baidu, etc, when
trying to fetch /robots.txt, often get blocked by nginx as it just
terminates the connection with 444 response. Sometimes they succeed
but more often they get blocked. Yes, I've checked to see if these
bots are not impostors but they look legit to me. My nginx version is
1.7.6 and the code I use is below

if ($host !~ ^(mydomain.net|www.mydomain.net)$ ) {
return 444;
}

I've also tried with !~* but I get the same outcome. Am I doing something wrong?

-- 
Yours truly