You're pretty close. The FBI
shows 12,664 homicides in 2011, 8,583 of which were by firearms. Of those, 6,220 were by handguns, 323 by rifles, 356 by shotguns, and 1,684 by firearms of an indeterminate type.
A little quick math tells us that 3.76% of firearm deaths, and 2.55% of all homicides, were known to have been committed by rifles.
(The number, of course, must be more, because some of those 'indeterminate type's are bound to have been rifles also, but these are just the known rifle homicides.)
(Also note that, for whatever reason, the FBI doesn't include information from Alabama or Florida.)
The more you know!