The castle scenes were filmed at Hatley Castle on Vancouver Island, Canada. The castle seems to be used quite a bit for movie and TV filming, previously used in Deadpool (2016), Descendants (2015), and X-Men: First Class (2011) among other film projects. A list of what movie locations used it can be found on the castle's website.
Tom's father has escaped from a prison called Millhaven. There is no Millhaven prison in the US where this movie is set, but there is one in Canada where this movie is actually filmed.