Damian, Zelda is rooted in video game tradition. That is, the Arch-Villain is incredibly lazy, using all his resources to avoid fighting you.

Or we can come up with some excuse that Ganon is busy looking for the Triforce or whatever. Does it really matter? Kill the monsters already.