If it is removed while the buff is active, this will cancel the buff. If a creature is downed in their turn, all of their actions are restored when Death Ward triggers.