Do you reply to all of your comments, some of your comment or none of your comments? I am of the mindset that it would be rude NOT to reply, but feel like if someone's just saying "cute kid," they probably don't expect a reply.
Which brings us to my second thought, do you reply by email or in the comment section of your blog?
When I have time, I try to go back to their blogs and leave them a comment. The few occasions where I actually emailed them to respond to a comment, I think I scared them off! They didn't reply and I don't think they ever left another comment!
I was about to post this same question, when I came across your post :). I usually go visit their blog and leave a comment for them. Every now and then I respond in my own blog but I don't know if people come back to read or not, or if they check the option to track responses. A couple of times I have emailed people back but it took me a while to realize I could do that and then I found that a lot of people used generic "no-reply" blogger email addresses so I couldn't email them back. I'm a little embarrassed to admit this but for a while there I also didn't realize that some profiles have a link at the left for "my web page" so if I didn't see any blogs listed at the bottom of their profile I didn't where to go to visit them. Now I know to check at the left too :).
It's so great to see what everybody's commenting preferences are! I have to say, I checked the box on a blog recently to have the follow-up comments emailed to me. I wound up with a dozen emails or more! I decided I wasn't going to do that ever again. I'd get excited that I had an email, and then see that it was just a comment from that blog. I was bummed!
I have an autoresponder that sends an email when someone comments, but I try to respond to just about every comment in the comment section of my blog. But if a person has asked a specific question and clearly expects and answer, I try to follow up my comment response with an email as well.
Well, I try to visit their page and leave a comment on one of their posts, but I have to admit that I haven't been very good about it. I need to make more of an effort, but it's hard because of time constraints!