I love Sister Rosetta Tharpe, and I love hearing Johnny Cash praise that old-time religion, but I also happen to think God's Song is the greatest number
ever written from a non-believer's point-of-view. (You can read it as the lament of a troubled believer,if you like.) Here's a young, shy and retiring Randy Newman performing it on The Old Grey Whistle Test, back in 1972. He delivers an even better, growlier version on that magnificent solo disc, The Randy Newman Songbook, Vol 1. Dig around on YouTube, and you'll also find him singing Political Science on The Colbert Show. Another classic, written in the spirit of Dr Strangelove. (Although it has to be delivered without smugness. I heard a British singer - who had better remain nameless - cover it not long after 9/11. At the end, she tore down the Stars & Stripes at the back of the stage. I could have slapped her.
UPDATE: For some reason, the YouTube screen can't be embedded, so here's the link instead