There are certain cases where the voice client can get into a bad state and think that you’re not connected when you are. Try using the force argument for VoiceClient.disconnect and setting it to True when you disconnect.
1
solved Can’t make Discord bot leave the voice channel in Python