So yakamo and I have been managed to do it again, the most recent episode of the unreasonable podcast, episode 0x03 appeared yesterday.
This episode should now be missing large chunks of silence and features a much deeper sexier voice for me. Turns out there were some config issues with my microphone. There were also some observation issues, I didn't actually listen to the whole show or check the waveform before doing the renders.
Should all be fixed now and it will sound silky smooth.