Thoughts on Debugging when you have no debugger
Jan
12
Written by:
Tuesday, January 12, 2010 1:24 PM
This is a philosophical post targeted at anyone who messes about with code, and from time to time comes across intractable problems that they just can’t get around.
When I first started writing code for a living (as opposed to messing about) I had a mentor who got placed into the role by simple fact that she sat at the desk behind me in my very first job. As the newbie on the team, I was pretty much less than useless, because not only could I not produce anything worthwhile, I actively took up peoples time trying to figure out why I couldn’t produce anything worthwhile. I knew this for a fact when I was assigned to ‘tidy up the team bookshelf’ – then a mess of programming and system manuals in those natty clip-open binders, so you could plug in the latest updates. Remember those?
Anyway, my self-appointed mentor (shamefully I can’t remember her name, though I’d run with Christina) basically got to the point where she told me “you can solve the problem, just keep tracing it until it presents itself”. I’m paraphrasing here, I’m pretty sure it was along the lines of “go away, you annoying person, and figure out your own problems. Why don’t you try using the debugger?”. However, in those days, point-and-click debuggers with nice little yellow highlighters and instant variable values were features not available to me. It was code, complile, and check to see if you got it right. That, and piling in lots of little logging statements to spit out what the program was doing. Eventually this paid off, because my manager figured she’d put me onto an esoteric bug which had so far eluded all the actual developers on the team. I had nothing else to do, so I mercilessly tracked this bug until my desk was covered in dot-matrix printouts of programs with little annotations on the side, flow charts – probably even an ER diagram or two. The one day, I actually found the bug and worked out a fix. It was a revelation. I added value to the team. A manager came by to thank me. I walked out of the building and thought ‘yeah, I’m a programmer now’.
Quite aside from my reminiscing of times long gone by, this early lesson taught me something that has stuck with me the whole time. No matter how frustrating the bug, no matter how elusive, all bugs can be tracked, found and fixed. It’s just a matter of persistence, tools and lots of thinking. My first bug was solved not by my genius, but because I just kept plugging away at it. Sadly, not all of us are freshly-minted graduates with days of spare time to fill, so sometimes bugs go into the ‘I’ll never sort that out’ pile. But you can’t pretend they are solved even if the official status is ‘unable to reproduce’ – eventually that bug is going to re-appear somewhere along the line.
Tracking Down a PayPal IPN Bug
If you read my prior post on this blog, you’ll know I’ve been doing some work with PayPal IPN listeners. An IPN listener waits for PayPal to send a Http Post to a Url on your site, called an ‘Instant Payment Notification’. Your listener must respond to this post, and tell PayPal that it received it OK. You must send the entire contents of the post back to PayPal, so that PayPal knows it sent the post (and not some nefarious scammer). If you respond correctly, you get a ‘VERIFIED’ response.
If you’re writing your IPN listener, in C#, there’s a handy code sample to use at https://cms.paypal.com/cms_content/US/en_US/files/developer/IPN_ASP_NET_C.txt
How this relates to this debugging themed post? I took the PayPal example, implemented it ; all working OK. I then added bits and pieces to the sample code, refactored it for my own purposes, generally started to build my listener. That’s when I hit the date problem in the previous post. The problem with debugging programs like this, is that when you’re developing an IPN listener, you’ve got to have a valid Url on the internet for PayPal to send the IPN to. You can’t host it on ‘localhost’, because the PayPal server can’t send the IPN to that. You have to host it on the internet somewhere. And that means no handy debugger. No stepping through code, no breakpoints, none of that. Back to where I started ; code listings, in-line logging and some head scratching and figuring things out.*
But the date problem isn’t what prompted this post : after I figured out the difference between the Sandbox and Live dates, I uploaded my new listener to the server, and consistently got responses of ‘INVALID’. This means that PayPal didn’t like the response I was sending back to them to verify what they sent. But the code, as much as I could tell, was much like their sample. Here’s where I would like to find a handy intern or graduate, assign them the task of tracking down why it isn’t working, and wait for a ‘huzzah’ to come from their cubicle. But there certainly is no graduate or intern around here. Equally, there was no possibility of endless days of debugging, with so many other more important things to do.
This is where a principle I now subscribe to frequently comes in : scrap it and start again. Whenever debugging something that doesn’t seem to work, it often takes less time to start again, and incrementally add the changes back, this time carefully testing each iteration.
So while I was paddling around in the Pacific Ocean this morning (marvelling the fact that mid-summer ocean temperatures in my part of the world are a balmy 25 degrees Celsius) - why keep trying to figure out where the problem is, when it would just take 20 minutes to build a whole new version. I would build a new one based on the PayPal example, and then incrementally add the other changes back in and check that it was working between each iteration. That’s just what I did, and it worked perfectly. I’ve now junked the old code. I don’t know what was wrong with it, and I never will. Total time to solve the problem was less than 30 minutes.
The moral of this story are the two point I’ve made here:
1) Never assume a bug is a one-off. There are very few one-off bugs, and chances are your bug isn’t one of those. It will re-appear.
2) All bugs can be tracked down with the right tools, approach and persistence
3) It’s usually quicker to take a different approach and start again, rather than try and work out where your code is flawed
4) Blog posts are a great way to procrastinate against further bug fixing.
That last statement is there because I realise this whole post is probably preaching to the converted, but I’m hoping someday one person will read it, find the energy to tackle an unsolved bug, and solve it with a new approach.
*I know you can do remote debugging of servers. You also have to open security holes everywhere on your server in order to make it happen. You just about have to sign a disclaimer in binary before Microsoft will let you do that. I figured the server would be full of bots and viruses faster than I could possibly fix it, and the performance would be horrible.