Well, if you really have followed the subject closely, you'd realize just how problematic the term "Berber" is, on many levels - not merely in this instance, where we're anachronistically projecting a decidedly post-Islamic term back into classical antiquity, but also for its pejorative connotations and lack of any meaningful semantic content.
Contra your first paragraph, "Libyan" and "Berber" are definitely not the same thing. "Libyan" is purely a geographic designation whereas "Berber" is a linguistic one. The people who speak "Berber" languages really have little else in common with one another - they are sedentary and nomadic, live in the countryside and the city, occupy mountain valleys and desert oases (and perhaps even Atlantic islands), have black skin, white skin, and every shade in between - in short, the term only makes sense as a linguistic term.
The use of the term "Berber" to describe the peoples of North Africa in antiquity is a relatively recent phenomenon--no later than the 80s, really. There have always been (IMHO quite reasonable) conjectures about the relationship between the contemporary population and the population in antiquity, but these never went beyond the linguistic level, and frankly we just don't know enough about the languages of North Africa at that time to say one way or the other. Some scholars discern Berber in the so-called Libyan inscriptions from Tripolitania and Algeria, but despite the fact that we have considerable research on modern Berber languages and even bilingual Latin-Libyan and Punic-Libyan texts, no scholar has yet succeeded in "breaking the code" and satisfactorily classifying the language of these texts. Furthermore, Berber isn't the only game in town; at least one Nilo-Saharan language, Kwarandzyəy, is spoken in Algeria. That's not to say that there's a Nilo-Saharan substrate underlying the Berber in the Maghreb... that seems unlikely to me, but we can't completely discard the possibility.