It's been quite a while since I've visited New Zealand, but I still remember the excellent online maps and I got curious, so I had a closer look at the Mount Taylor question.
The next higher ground with respect to Mount Taylor is in the Arrowsmith Range to the west.
The lower valleys surrounding Mount Taylor and the nearby connected mountains are defined by various rivers and streams. To the North and East is the Rakaia River, to the South the Ashburton River, both of which head to the southeast to the Pacific Ocean. West of Mount Taylor lies Lake Heron (691 m). This lake is drained to the north by Lake Stream, which ends in the Rakaia River. Though not connected, very close to the south side of the lake lies Gentleman Smith Stream, flowing south to the Ashburton River. From this topography it follows that the key col between Mount Taylor and the Arrowsmith Range must be somewhere between the south side of Lake Heron and Gentleman Smith Stream. Since the contours of the valley floor in this area never exceed the 700 m mark, the key saddle between Mount Taylor and the Arrowsmith Range must lie between 691 m and 700 m. With Mount Taylor at 2333 m, its prominence must lie between 1633 m and 1642 m.
Another way to see this is to imagine lines starting at Lake Heron, along Lake Stream and the Rakaia River to the coast, and another one starting at Lake Heron, along Gentleman Smith Stream and the Ashburton River to the coast. The first line is never higher than Lake Heron, the second one is only a few meters higher, but never reaches 700 m. Since Mount Taylor is the highest point in the area defined by these two lines and the ocean, its prominence is at least 2333 - 700 = 1633 m.
Click here for map with the approximate location of the key saddle (marked by the green paddle); zoom out for the bigger picture.
Mount Arrowsmith from Mount Taylor (by gimpilator), with Lake Heron clearly visible in the valley below.