How to add robots.txt to a Django project
So we had this nice, unfinished project, publicly shared on a public server being indexed like a pig’s behind on google servers. Some searches on google had hits to full length urls to this unfinished project on the test server. Easily stopped using the robots.txt, except that its a Django site and its not as… Read More »