I think this is a pretty good point.
I have read about Byzantine heavy cavalry, but don't know much about their infantry. How were they organized?
In which period?
In the Strategikon, the battle cavalry [defensores] are paired with half as many scouting/archery cavalry [cursores]. In theory, both forces were supposed to train with the others' weapons. It's unclear how often the early Byzantine defensores used horse armor, but, given their role, it's unlikely the cursores used horse armor. Ideally, both the cursores and defensores are professional troops, though Belisarious sometimes hired mercenaries to add to the cursores. If the cavalry operates apart from their infantry, they form 2 1/2 lines, with up to 2/3 of their strength in the first line and flanking forces, most of their reserve in the second line, and small forces in the third.
By the time of the Praecepta Militaria, fully-armored cavalry [kataphraktoi] form a wedge in the center of the first line and, if enough are available, in the third line, but they rarely account for more than 1/10 of the cavalry. I think the tagmata are regular troops, while the thematic troops are drawn from the military lands. If the cavalry operates apart from the infantry, they form three lines, with roughly 3/10 of their strength in the first line, 4/10 in the second, and 3/10 in the third.
In the Strategikon, the infantry appear as something of an afterthought, and their organization has been cribbed from post-Hellenistic Greek military theorists such as Aelian. So the Stretegikon isn't trustworthy here, and their organization probably resembled Late Roman infantry organization and/or Early Byzantine cavalry organization. And they included spearmen and archers.
I think they switched from spears/shield walls to pikes by the time of Leo VI and back to spears/shield walls by the time of Nikephoros Phokas. The infantry were to form a square to protect the baggage during battle. They were organized into forces of about 1,000 infantry each, including spearmen, specialist anti-cavalry spearmen, archers, and javelinists.
Eric McGreer's book is a good starting point, but it only covers one period, and Byzantine tactics changed considerably between different periods.