The poppy is affiliated with Remembrance Day and Veterans Day (USA) because they grew extensively on the battlefields during WWI.
The first Armistice Day was held in England on 11 Nov 1919. In 1921, the Canadian Parliament adopted the first Monday in the week of November 11 as Armistice Day. In 1931, the government changed the name to Remembrance Day, and the date was set for November 11.