David Harbour has made it clear he doesn’t want to fuel the media “hysteria” surrounding his relationship with Lily Allen.