If you also change the internal ID of the voice file to match that of the new character, it should work. But be warned that voice clips may play in different places, and some animations may be completely mute depending on how many voice clips the original character has.
If you ask me, you’re better off remaking the voice mod for the target character instead.
As Tougon already pointed out, you're better off remaking the voice mod so that it'll be compatible with a specific character. For an example, a voice mod for Bowser can't properly be used for Giga Bowser; you'll have to modify Giga Bowser's .nus3bank files as well.