Today it just popped into my head without warning. This is how it happened: "Dunt, dunt, dunt, chhh. Dunt, dunt, dunt, chhh... what is that tune? I've heard that before... I seem to remember a hitchhiker talking to the camera... walking down the road... and... oh my god how did I forget about The fucking Hitchhiker?!"
I looked it up and the oldest I could have been during its original run was four. I might have seen it later when it was syndicated or something, but I don't know for sure. Perhaps a television show of all things is my earliest memory.
Memory is a funny thing. I barely remember what I did yesterday, but I remembered that.
(Ray Bradbury claimed to remember his own birth, in particular the pressure he felt around his head. Jay Leno once claimed the same thing on national television. Modern science, on the other hand, claims they're delusional. Given what little I know about the subject, I'd say yes, they probably are confusing imagination with memory. I personally don't believe any man who actually remembers childbirth could bring themselves to have sex with a woman like those two men presumably did.)