This site combines information from two separate servers. The address
pwall.netresolves to a server run by a hosting service — currently Gandi in Paris, France. That server holds most of the content for the website, but some of the bulkier files, for example full-size images, are stored on my home server in Sydney.
Both of these servers use HTTP software that I wrote using node.js, and both keep server logs. The logs record the date and time of the request, the URL, the IP address of the requesting machine, the “User-Agent” string of the software (e.g. the browser) that made the request and the URL of the page that linked to this request.
The main purpose of the logs is to monitor intrusion attempts. I want to see what forms of attack are being made on my website, and whether any of those attacks have any chance of success.
The secondary purpose is to see which parts of the site are the most popular, and that can lead to some surprising findings.
Shortly after I started using my home server to serve the images for this website I noticed a request to download this image:
I was a little mystified — the image had at one time been on the main page of the website, but by the time of this download it had been relegated to the Picture Archive pages. And the requesting IP address resolved to somewhere like Kansas. Why was someone in Kansas downloading this image?
After two or three more downloads of the same image, all to equally unexpected locations, I decided to investigate. On a hunch, I tried a Google image search for “geese flying”, and there was my image in the first page of search results (when I tried again recently it had disappeared from the results — fame is fleeting).
At the time I had no server logs to refer to, but now I can check the “Referer” (sic) header of the request to find where it came from, and if it came from a search, what the query string was.
That is the purpose of the logs — to find out where the vistors to the site (benign and otherwise) are coming from and why, and to help keep out the baddies and improve the experience for the friends or the merely curious.