Following his advice, a new location comprising an area of 8 acres (32,000 m2) was selected on the hillside of Battery Point just to the south of Hobart Town, and construction began on what was to become the first of a series of new defensive installations.
Upon its completion, the Mulgrave Battery soon attracted heavy criticism from those who had to serve there. Members of the Royal Artillery felt it was inadequate, and one critic is even said to have described the Battery as "a poor pitiful mud fort."
Engineers reported that the gun carriages were a danger to men firing the guns, and so new timber was sent from Macquarie Harbour in 1829 to make them safer; however, records showed that only one gun had been upgraded by 1831. The same year, the galleries were improved with large 15 metre long sections of timber, heavy bolts, braces and bars.
He devised an ambitious plan to fortify the whole inner harbour of the Derwent River with a network of heavily armed and fortified batteries located at Macquarie Point, Battery Point and Bellerive Bluff on the eastern shore. He envisaged the forts all having an interlocking firing arc, which would cover the entire approach to Sullivans Cove, making it impossible for ships to enter the docks or attack the town unchallenged.
Despite funding problems, work using convict labour did begin in 1840. Mulgrave Battery was enhanced and expanded, and a new site was located slightly further up the hillside on Battery Point, behind the location of the Mulgrave Battery, where construction also commenced in 1840.
The modern Hobart suburb of Battery Point takes its name from the Mulgrave Battery. The original guardhouse, built in 1818 which had been located nearby is the oldest building in Battery Point, and one of the oldest buildings still standing in Tasmania.
The new battery, named "Prince of Wales Battery", was completed in 1841. That year ten new 8-inch (200 mm) muzzle loading cannons were lifted into position, enhancing the firepower of the colony's defences. Despite its significant firepower, the poor location and firing angles of the new fortress soon became obvious.
As a result of these calls, the Tasmanian colonial government began to establish Volunteer Local Militia Forces. One such force, established in 1859 was the Hobart Town Artillery Company under the command of Captain A. F. Smith, formerly of the 99th. (Wiltshire) Regiment, who began to assume responsibility for the Hobart fortifications from the Royal Artillery who were increasingly being withdrawn, and had all departed well before the withdrawal of the last British forces from Tasmania in 1870. Prior to this, in 1868 a Defence Proposals paper had been published which outlined the need for greater defensive fortifications. It also suggested the need for proposed batteries further to the south of Hobart Town on either side of the river.
The arrival of three Imperial Russian Navy warships, the Africa, Plastun, and Vestnik in 1872 caused a great deal of alarm in the colony. Britain and its empire had only been fighting the Crimean war with the Russians 16 years previously. The colony was virtually defenceless, and had the Russians had hostile intent, would probably have easily fallen. Luckily the Russians were on a good will mission, however, it cause a great deal of debate about the state of the colonies defences.
To this day, the park in which the Mulgrave, Prince of Wales and Albert Batteries had been located remains a popular public park, and is named Princes Park in honour of the men who served in the batteries there, and as a reminder of the heritage of the site. The iron gate sealing the entrance to the tunnels and underground magazine rooms can still be seen at the base of the park.