Prince Harry has had “unproductive” conversations with William and Charles