It's been a few years, but Key West isn't high on my radar if places to return... Went there while in college but off peak, so we avoided most of the crowds. That made it better. Really there were only two memorable things for me: 1) mile marker 0 & 2) sunset... Oh, and a guy in an uncle Sam get up who tried to solicit my attention by shouting, "hey at-tur-eye guy" when I was wearing an Atari t-shirt.
I forget all the keys were stopped at, but there ones which were more fun and family friendly were the ones further north.