I try to do this as much as I can, but since I use the dreaded Blogger it's harder to respond than it is with Typepad and Wordpress. Then again, I don't get many comments anyway unless they're entering a giveaway.
Like you, I try to respond to all comments even if it is just to say thanks. There have been a few times I haven't gotten to a comment, but I try to make sure to always go to the commenter's blog and comment.
I used to send emails to new commenters, but I have gotten lazy and I don't do it as much anymore. I always reply in the comment section. However, if it is personal or comment that needs attention I will send an email. I do try to leave a comment on their blog. But I feel bad that I only leave one comment on their blog to possibly never return to it. I don't want to get in the habit of 'I will only comment on your blog, if you comment on mine.' I can't possibly keep up with everyone's personal blog that follows mine. I would like to, but I can't.
I just started an automated thank you email to new commenters. I also try to respond to all comments, even if it's just with a "Hi, thanks for the kind words". I will also visit their blog and comment if I have anything to say, stumble it or share it in other ways if I really enjoy it.