- How to See What’s Happening On Your Site in Real-time Using Your Log File - February 17, 2012
- Turntable.fm vs. Blip.fm: Two Fantastic Online Music Communities - October 3, 2011
- Comparing Tumblr and Posterous - June 16, 2011
Have you had a look at your website’s log file lately? No? Let me tell you, once you set your eyes on that thing you’re going to have trouble tearing yourself away. It will show you every file your Apache server is fetching, as it happens. This means if someone is visiting your website, you’ll see “GET nameofwebpage.com/specificpost.html,” and it will show you that as it happens in real time. You can see if the next-random-user of your website bounces, or if their IP address goes on to click on something else, and what that something is, and how long they stay there. Creepy-awesome, isn’t it?
Here’s how you do it:
Baby Simple Instructions on How To Access Your Log File
First, you’re going to need a command-line program. The Windows one won’t do, as that’s just going to give you DOS. I recommend Putty. Go to http://putty.nl and find the download page. Putty is such a simple little file it doesn’t unpackage anything; when you save it that is the install. So be sure to save it wherever you plan on accessing it.
Now that you have Putty, open it. In the box that pops up, type the website you use to get on your site using FTP. At the bottom of the screen, click “Open.”
The black box that pops up is where your log is going to read out. It should say Login. Give it the username you use to login to FTP and, after hitting Enter, the password that goes with it.
With me so far? You’re now logged into your website on the command-line.
Now type: tail -f /var/log/ but don’t hit Enter just yet!
I can’t tell you exactly what to type because I don’t know how it is written on your site. We’re going to get Putty to tell us by using the Tab key. Hit it once and it will automatically complete the line if there is only one option with what you’ve typed so far. Hit it a second time, and it will list all ways that the file you’ve started typing can be completed.
So after tail -f /var/log/ try typing ht and hit tab twice.
Did tail -f /var/log/httpd/yoursite come up?
If yes, sit tight while I catch the others up. If that doesn’t work, backspace back to tail -f /var/log/ and try adding ap and hitting tab twice. Hopefully you’ll see it tab-complete to apache/thenameofyoursite. If there are too many to list, start typing the name of your site, and hit tab once to get it to complete the name of your site. So far, your screen should either say:
tail -f /var/log/httpd/yoursite
OR
tail -f /var/log/apache/yoursite
If you were sitting in the back throwing spitballs, you may be able to type one of the commands right there to get caught up. But still, don’t hit Enter just yet! We’re almost there.
Now when you hit tab the second time, one of the options should have the word “log” in it. If there are a bunch of them with numbers, just type out the simple one without any numbers. You could always copy-paste this exactly, once you know what to type. I’m using the tab-complete technique because it’s faster, it will tell you what to type if you don’t know, and most importantly, you can be sure everything is spelled correctly. The biggest pitfall of the command-line is that one spelling error and not a thing works. By having Putty finish the command with the Tab key, you have Putty spell it out for you.
Once you have the entire command, it will look something like this:
tail -f /var/log/httpd/yoursite_log
Or possibly the same thing, with “apache” instead of “httpd”
NOW, hit Enter.
Is there a bunch of gobbledegook flowing up and off the screen? Congratulations, that’s your Apache server’s log! I like to hit Enter a bunch of times so there’s a big blank space where I am looking. Mostly, what you will see on the screen will be an IP address (who’s checking out your page) followed by the command and file (the webpage they are accessing).
You can use this same process to figure out why something on your site isn’t working. Let’s get to that next.
How to Access Your Website’s Error Log
Say, for example, that you’ve created an .htaccess file and when you try to use it you keep getting a message like “Server Access Error.” There’s a log of that error, and you want to see it to try to unravel why your page isn’t working. To get to the Error Log, do exactly what we did above, all they way up to the part where you are tab-completing to get the name of your site. But instead of choosing the Log file, type an e, hit Tab and, if that doesn’t complete the line, hit it again to see your options. You want the one that says Error. So you’re here:
tail -f /var/log/httpd/yoursite …[Tab] [Tab]
or conversely,
tail -f /var/log/apache/yoursite …[Tab][Tab]
Once you have the full command, with error at the end, Hit Enter. It should show you the most recent error messages. Again, if you want to trigger the error again (by doing whatever is causing the error, in another window), hit Enter a bunch of times to create some space so you’ll see the right thing as soon as it pops up.
Troubleshooting
A few tips:
- Unlike MS Windows, this land is case-sensitive. If you capped something that shouldn’t be, or vice versa, it’s not going to work. Using tab-complete will insure you have the right case.
- In the command-line, spaces matter. If you added one or took one out, your command will fail.
- If you want to run a command you already ran, hit the Up key to scroll to it.
- To copy something to paste it elsewhere, you need only select it. Selected text in Putty is copied automatically.
- To paste something into Putty, hold down both mouse keys at once.
- If your log isn’t at tail -f /var/log/httpd/yoursite or tail -f /var/log/apache/yoursite try reading your Apache configuration file.
- If your server isn’t run on Apache, this probably won’t work for you.
- Want to see both your log and any errors at the same time? Type this: tail -f /var/log/httpd/fyoursite_{error,log} …or whatever tab-complete has offered you, plus {error,log}.
Help! How Do I Get Out of This Thing?
If you want to get out without closing the window, press CTRL + C and it will take you out of the log and back to your command prompt.
I’ve tried to make this baby-simple for folks who aren’t generally comfortable on the command-line. I hope it helps!
David@theOnlyCog says:
Google Analytics now has Real TIme reporting built in. Much easier.
Nice article though — good detail
.
February 17, 2012 — 3:57 pm
Karma says:
Thanks for the tip David! I confess haven’t mastered Google Anayltics yet, but I can say that they aren’t giving you everything. I say this because when I go to the page for my traffic, it will show significantly less traffic than my log shows. Perhaps this is because Analytics doesn’t show spiders? I’m not sure why, but I will see more action on the log file an hour than Analytics will claim hit my blog in a week.
I’m also curious if Analytics will show the error log so you can fix problems you encounter on your site.
February 17, 2012 — 4:28 pm
David@theOnlyCog says:
Karma, sorry I should have been a little more specific. After I read it back I realised I had not said enough but I could not edit it.
What I was eluding to, in almost no words at all, is that Google Analytics is a great way to monitor site traffic, and now in realtime.
It does not provide nearly as much detail as your article would allow users to explore. However, not all web site users have access to their logs files or the command line so my comment was aimed at them!
February 17, 2012 — 4:51 pm
Karma says:
Tee-hee===>”What I was eluding to, in almost no words at all”
No worries. It at least got me to go log into my Google Analytics account, which I so infrequently do.
February 17, 2012 — 8:04 pm
Angela Rodrigues says:
I like the valuable information you provide in your articles. I will bookmark your blog and check again here frequently. I am quite sure I will learn many new stuff right here! Best of luck for the next!
February 18, 2012 — 3:16 pm
cercacasa.it says:
All Oѵer The World On Yοur Own Τime
December 1, 2012 — 10:07 am
Karma says:
Hey, I was just thinking how great it is that your links are still reliable, when I noticed your note about keeping this old post up. I really appreciate it is, as I’ve linked to all the posts I’ve written for you on my Writing Samples page. (Which I’ll not link to here, as the crosslinks could the tiny bit of SEO you get from my link—though I’ll bet your comments are no-follow!)
March 7, 2017 — 3:11 pm
Karma says:
…Could *hurt* the tiny bit of SEO…
Looks like I left out a word there in my haste to show my approval.
March 7, 2017 — 3:14 pm