You and I discussed the hold item issue via AIM awhile back. I'll repeat the recommendation I made then: White Herb would be very useful in combination with In Fight, as it would restore your lost stats without requiring a time-consuming switch. (Switching so frequently to avoid the drawback can be costly, as it leaves whatever Pokemon enters the battle wide open for a free move.)
Shell Bell makes little sense for a sweeper, as the point is to deal as much damage as possible before your fighter falls. You'll want high enough Speed to keep it from being hit first, anyway, so the only reason to recover (ideally) would be Flare Drive and any opponents that survive your first move. That's the basic idea of sweeping.
For that same reason, I'm a little cautious about Slack Off. It seems a little weird to use a healing move on a sweeper - especially when it's a Pokemon that you're already worried may have a glass jaw.
The one concern you may have beyond that is Swords Dance. Be sure that when Goukazaru comes into the battle, it'll last more than one attack against whatever's in the fight. It'll need to take one strike while it powers up with Swords Dance. Then feel free to crush whatever's in your way.
For EV training, a physical sweeper obviously needs to have high Attack and Speed; I recommend pushing those to the max. Then you can throw whatever's left into HP (or one of the defensive stats if you need to cover a weak point in IVs).