nginx reload fails with [emerg] host not found in upstream

[Ruslan Ermilov]

On Fri, Dec 07, 2012 at 01:22:50AM -0500, groknaut wrote:
> maybe this one's better:
> 
> 12/07 06:19[root at proxy2-prod-ue1 ~]# cat gethostbyname.c 
> #include <stdio.h>
> #include <netdb.h>
> #include <sys/socket.h>
> #include <arpa/inet.h>
> #include <netinet/in.h>
> 
> int main(int argc, char *argv[])
> {
>     if (argc < 2) {
>         fprintf(stderr, "usage: %s hostname\n", argv[0]);
>         return 1;
>     }
> 
>     // skip 0 because that is the program name
>     for (int i = 1; i < argc; ++i) {
> 
>         struct hostent *lh = gethostbyname( argv[i] );
> 
>         if (lh) {
>                 struct in_addr **addr_list;
>                 addr_list = (struct in_addr **) lh->h_addr_list;
> 
>             printf("%-14s %s\n",
>                 inet_ntoa( *addr_list[0] ),
>                 lh->h_name
>             );
>         }
>         else {
>             herror("gethostbyname");
>         }
>     }
> 
>     return 0;
> }
> 12/07 06:20[root at proxy2-prod-ue1 ~]# gcc -std=c99 gethostbyname.c -o
> gethostbyname.bin
> 12/07 06:20[root at proxy2-prod-ue1 ~]# ./gethostbyname.bin
> webapp02c.prod.romeovoid.com
> 10.51.23.17  webapp02c.prod.romeovoid.com
> 12/07 06:21[root at proxy2-prod-ue1 ~]# ./gethostbyname.bin
> webapp06c.prod.romeovoid.com
> 10.195.76.80 webapp06c.prod.romeovoid.com
> 
> 
> our DNS does work..

Like was already told, nginx internally does gethostbyname()
to resolve hostnames during configuration, so if a problem
disappears by moving the hostnames into /etc/hosts, I'd not
suspect nginx.  (It does gethostbyname() twice due to how
it's currently coded, so it's expected.)

To emulate what nginx does internally when processing this
upstream, run it like this:

./gethostbyname.bin webapp02c webapp02c webapp06c webapp06c roapp02c roapp02c

WITHOUT hostnames in /etc/hosts.  Do it several times in a row.

If that doesn't reveal the problem, do you have an ability
to recompile nginx from sources?