In the flashback that shows Aunt Josephine in the market, one of the vendors announces she has "very fresh dill". This is a nod to the secret society V.F.D., of which the Bauldelaire parents are members.
Josephine Anwhistle is named after a mouse named Josephine in "Josephine the Singer, or the Mouse Folk" by Franz Kafka. This is clear in The Reptile Room book, as it mentions a Café Kafka at Damocles Dock.
The author's notes for The Bad Beginning state that, in the Victorian art of flower arranging, peppermint symbolizes "cordiality, warmth of feeling." The Baudelaires are allergic to peppermint, and thus lack cordiality and warmth.
The Fish Head Salesman is played by Daniel Handler, who (under the pen name Lemony Snicket) wrote the book series 'A Series of Unfortunate Events'. He cries "fish heads, fish heads, roly-poly fish heads," quoting the 1980 novelty song "Fish Heads" by Barnes & Barnes.
The map of local diners has streets named after the show's crew members - Handler St. (Daniel Handler, author of the books), Sonnenfeld St. (Barry Sonnenfeld, the director), Hermansen Ave. (Dan Hermansen, art director), Welch St. (Bo Welch, director and production designer), Goojha Ave. (Sean Goojha, assistant art director), and Shier St. (Amy Shier, office production assistant).
The yellow taxi driving through the gray Damocles Dock might be a reference to Gatsby's yellow car driving through the Valley of the Ashes in The Great Gatsby by F. Scott Fitzgerald. A Series of Unfortunate Events has a lot of Great Gatsby references.
The trivia items below may give away important plot points.
When Lemony Snicket (Patrick Warburton) is standing on the cliff where Josephine's house was, there's a stone next to him where the name "Beatrice" is engraved. Beatrice is the name of Snicket's long lost love whom he keeps referring to and begins episodes with notes addressed to her.