No, I know exactly what you're talking about. Once in Japan my best friend and I walked into my house, which was full of the smell of fresh bread. She said it smelled just like me, which was a surprise, to say the least!

I am pretty sensitive to the olefactory memory of places, because sometimes I'll have one just hit me, introduced by something else, but I'm half-smelling the place I'm remembering. It's one of those things you can't really reproduce in fiction, which can be frustrating.

I loved writing my wolf shape-shifter POV because I concentrated on smells as her primary sense, even before she knows why her nose is so important. ^-^