how can i avoid google index my site properly jail directory only allow html extension.

lifeisjustabout nginx-forum at nginx.us
Tue Mar 19 08:10:57 UTC 2013


Hello 
I have small pdf search site searches pdf over internet when somebody search
URL look like www.mypdfsearchwebsite.com/pdf/nginx.html  with (html
extension)

google index this address with html and without html version so there are
duplicate record on google index each search lies without html
www.mypdfsearchwebsite.com/pdf/nginx  and with html
www.mypdfsearchwebsite.com/pdf/nginx.html version

what i would like to do is if i can make it around without html version
(www.mypdfsearchwebsite.com/pdf/nginx) will be returned 404 page rather than
200 it could be also jail /pdf/ directory only allow html if html couldn't
find go to nginx 404 page.

I am not nginx guru but the newbie if you send me exactly location where i
put suggested code i would be highly appreciated 

thank you very much for your help


my nginx conf is below 



server {
listen  1.2.3.4:80;

         server_name mypdfwebsite.com www.mypdfwebsite.com;

         # log_format  awstatcomp  '$host $remote_addr - $remote_user
[$time_local] "$request" ' '$status $body_bytes_sent "$$

       #  access_log /var/log/nginx/mypdfwebsite.com.access.log main;
        access_log /var/log/nginx/mypdfwebsite.com.access.log awstatcomp;
         error_log /var/log/nginx/mypdfwebsite.com.error.log;
         root /home/mypdfwebsite/www;
         index index.php index.html;

        location / {
                try_files $uri $uri/ /index.php?q=$request_uri;
        }

     location ~ \.php$ {
             #root html;
             fastcgi_pass 127.0.0.1:9000;
             fastcgi_index index.php;
             fastcgi_param SCRIPT_FILENAME
/home/mypdfwebsite/www$fastcgi_script_name;
             include fastcgi_params;
        }



         location ~ /\. { deny all; }

        location ~ \.pl$ {
          gzip off;
          include /etc/nginx/fastcgi_params;
          #fastcgi_pass unix:/tmp/php.sock;
          #statistic perl  
          fastcgi_pass  127.0.0.1:8999;
          fastcgi_index index.pl;
          fastcgi_param  SCRIPT_FILENAME       
/home/mypdfwebsite/www/$fastcgi_script_name;
          allow 127.0.0.1;
          allow 1.2.3.34;
          deny all;
  }

}

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,237527,237527#msg-237527



More information about the nginx mailing list