I watched the first 4 episodes back when the show started, and decided that it was just a quirky x-files with an annoying main character (Walter Bishop). I stopped watching. Then, surfing channels a year or so later, I caught the season 2 final episode, and decided that it was interesting enough to rent the DVD's and catch up from the beginning over the summer. It is now my favorite SciFi show.
I have to admit, I initially did not like the shuffling of all the characters at the beginning of this season. The show seemed aimless. Now, however, it has gotten interesting again, at least for me. It is certainly better than the last season of Babylon 5, which wandered aimlessly after they finished their story arc and never recovered. Both shows lost focus when they were about to be canceled, and then at the last minute got one more season. Fringe seems to be recovering nicely. I hope it survives for a season 6, but since it is Fox, I will not hold my breath.