The first episode is actually based on the second novel in the Patrick Melrose-series.
The NY cab driver has also acted in an episode in Sherlock.
Hugo Weaving (David Melrose) and Benedict Cumberbatch (Patrick Melrose) also worked together on The Hobbit: An Unexpected Journey (2012) & The Hobbit: The Battle of the Five Armies (2014) as Elrond & Smaug/Necromancer (voice) respectively.