But the building was short-lived. In 1781, Charles Cornwallis, a general in the British army, and his troops are said to have set fire to the barracks as they traveled east to the Battle of Yorktown.