After chasing Diana into the water, when returning to shore, the dog is chewing a ball in his mouth.
When Diana jumped into the water she was holding a bottle with the note. When swimming to the boat both hands are visible without the bottle. When she gets to the boat, she hands the bottle to Perry.
When Perry is doing the crime reenactment in court with the dog Fritz, he has one of his witnesses Simmons point a pistol at the different attendees as Perry calls them up one by one. When Fritz jumps and grabs Simmons' arm the pistol goes off. They would not use a real pistol, even one possibly loaded with blanks, for the demonstration.