Y'see, I didn't see it that way with the Winter Soldier's limited appearance. He wasn't physically in every shot, but he was an intimidating background presence through the whole movie. You don't want to overuse a character like that, or he loses the mystery that was integral to the character.
And you could not have removed Winter Soldier from the movie; he was supremely essential to, if nothing else, Cap's arc.
As for what's in your spoiler . . .