I read the first few books quite a long time ago, then kinda stopped when it took like 4 years for the next book to come out. Because it's been so long, I've forgotten many of the details, and I don't really notice very much when the television series strays.
It's worked out well for me, I think. I came in knowing who the main characters were, so I didn't feel lost with them introducing so many characters. But I also don't feel pissed that they changed the story to fit the condensed time frame.
It's the best thing on TV, and the ONLY reason I am paying for ****ing HBO.