After being compared to a Jehovah's Witness, Thirteen says, "The weak lost faith when Christ didn't return in '75, but I still believe." This refers to a misconception that Witnesses predicted when Armageddon was to transpire.
House and Thirteen are in the store looking for supplies to make the spud gun better, and she suggests using fertilizer. House replies, "By all means, Kaczynski," referring to Ted Kaczynski, who was known as the Unabomber.
House says, "Hey, Harriet the Spy," referring to the 1964 children's novel by Louise Fitzhugh (later made into a movie - Harriet the Spy (1996)) about 11-year-old Harriet M. Welsch, who spies on her friends.
The lake house where Thirteen tests the spud gun is the same location that Patrick Jane purchased to renovate on "The Mentalist", the similarly themed detective series.