In one universe, Suzette has a tear run down her face, leaving a tear streak. Moments later, the tear and streak are gone.
If there are innumerable parallel universes and every time he uses the key to go through one, he not only goes to a parallel universe, but to an entirely different location, and yet, in all but one, he has some sort of relationship with one particular other person. The odds of one particular circumstance against innumerable variations is astronomical and involve multiple years, if not lifetimes.