There is an excellent new (2006) bio of Einstein by Walter Isaacson, with a chapter called 'Einstein's God'. He had a deeply mystical view of God, and never doubted his existence, but also a scathing view of religion (although he was generally very tactful about it.) People were always trying to enlist Einstein into various causes, social, political and religious, and he would have none of it; ever the complete individualist. But he had some great quotes on the topic:
'Try and penetrate with our limited means the secrets of nature and you will find that, behind all the discernable laws and connections, there remains something subtle, intangible and inexplicable. Veneration for this force beyond anything we can comprehend is my religion'
(Mind you, doesn't stop people from trying to envisage it, comprehend it, and explain it, as they constantly do on the Forum.)
On whether he accepted the historical existence of Jesus:
'Unquestionably! no one can read the Gospels without feeling the actual presence of Jesus. His personality pulsates in every word. no myth is filled with such life'
On whether he believes in God:
'I'm not an atheist. The problem involved is too vast for our limited minds. We are in the position of a little child entering a huge library filled with books in many languages. The child knows someone must have written those books. It does not know how. It does not understand the languages in which they are written. The child dimly suspects a mysterious order in the arrangement of the books but doesn't know what it is. That, it seems to me, is the attitude of even the most intelligent human being toward God. We see the Universe marvelously arranged and obeying certain laws but only dimly understand those laws.'
The chapter goes on to stress that Einstein did not believe in a personal Deity and certainly was never a member of a religion except for as a non-observant Jew. However he was adamant that he was not an atheist:
"There are people who say there is no God. But what makes me really angry is that they quote me for support of such views."
(Incidentally, the first chapter of Dawkin's The God Delusion - 'A Very Religious Non-Believer' - does exactly that by invoking Einstein as a model of 'intelligent unbelief'. But I do know from experience that any posting on the Dawkins forum which expresses the views given by Einstein above will automatically and scathingly be dismissed as 'superstitious nonsense' and I doubt very much that the portrayal given in the Dawkin's book, and the one in the Isaacson book, can actually be reconciled.)
All quotes from Walter Isaacson - Einstein - His Life and Universe
Simon and Schuster Britian 2007, Chapter 17.