I change my blog design whenever I feel like I want something new and fresh. Your readers are coming to read your content not to just look and stare at your design, so go ahead and switch it up and let us know when you do it so we can oohhh and awwweeee at it.
I do my own web design and I'm in the middle of a change right now. I've had the same layout for about 6 months and I'm really getting bored with it! I think it's normal to change things up every once in a while and not a turn off for readers at all.