How do I kill these damned saplings?

[alnorth]

So, maybe I'm naive but where I came from we didn't have this problem because it was hard as hell to get anything to grow. Up here in Iowa, I've learned that if you so much as drop a seed onto the ground the plant may impale your feet as it rockets up from the earth.

I wasn't paying much attention to what was going on near some big annual flowering privacy bushes in the backyard and I discovered earlier this summer that I've got 4 of these healthy-looking saplings growing through them, which I promptly cut and disposed of. They must have started a couple years ago and I guess the previous owner did nothing about them, because we're talking 2-3 inch little stumps now. They are growing right in the bushes so it is too awkward to attempt to dig them out without harming the plants, and they keep trying to sprout back. Now that I'm aware of these monstrosities, I've been murdering these baby trees when they come up (had a couple try to sneak up right against the house).

I dont think I can use any chemicals because I may harm the huge bushes they are re-growing in. How do I kill these things? If I just keep cutting them down over and over again several times a year, do they give up and die? Any other tips?