You're not completely wrong, but you're not completely right either. In terms of military necessity, the purchase is utterly unnecessary - there are no realistic scenarios in which New Zealand a) has a need for an air combat capability, and b) is not operating with allies or partners who already have a better air combat capability. In terms of military utility, you're far better to spend the money on transports, helicopters, OPVs, MPAs, trucks, or practically anything else. So far, so correct.
However, having an air combat capability - even if it's just a handful of armed trainers, like the Macchis - is very useful in other areas. For one thing, it signifies a certain readiness to put one's money where one's mouth is when it comes to supporting allies. It reassures partners that, if things get nasty, you won't just send engineers, medics, and transports - you're actually willing to do some of the fighting and take some of the risks, too. This isn't me making stuff up, either - after NZ got rid of the A-4s, it was very noticeable how the tone at ASEAN changed. NZ ended the 1990s as a valued and desired partner in many areas, and began the 2000s being asked questions like "so, exactly why should we care about you, again?". A great deal of credibility was lost because of what was seen as a demonstration that NZ wasn't willing to take their security concerns seriously, and this had ripple effects in other areas.
Having even a small capability also reassures the people who you expect to come to your aid if you need it. It shows them that you're willing to contribute to your own defence, not just rely on them for everything, which tends to make them feel a bit better even if it's always been acknowledged they'll be doing the heavy lifting. Note that this is not necessarily a rational response - anyone who can actually invade NZ will not be stopped by a squadron or two of A-4s, so the military value of the of the aircraft for that purpose is negligible. It does show, however, that NZ is willing to at least try, even while they're waiting for the USN to steam to their rescue.
As mentioned earlier, even a small air combat capability keeps alive the possibility of expanding it if that ever looks attractive. It's very difficult to reconstruct an air combat capability once the institutional knowledge is lost. If you need it NOW, you probably should have started 20 years ago. Singapore isn't a bad example of this - they became independent in 1965, identified a need for all sorts of military capabilities, and were willing to spend heavily to develop them. By the mid-1990s they were being taken fairly seriously, but it had taken decades of determined and sustained effort to get to that point.
To answer the question "what the hell do we need them for?", it basically comes down "having options in the future". If you're willing to accept the damage it does in terms of international relations, then maybe you can do without them. But you're taking a big risk.