Pedro II only met Barral in 1856. She lived in France until 1848, and then stayed in her family home in Bahia until she was nominated teacher of the imperial princesses by indication of Pedro's sister Francisca. So, she wasn't the reason for Pedro and Teresa not having more children.
Barral and Pedro II were definitely lovers. Their letters let it quite clear (with lines as "the follies we made in that bad with two pillows", "I love you and I am yours with all my soul, I hug you as ardently as you desire", "I burn with the desire of covering you with caresses", and thing like this).
Actually, Pedro had relations with several women, but he was very discreet about it (the opposite of his father). A Spanish diplomat wrote that Pedro II used the library of the Palace as refuge for his adventures. The only known case in which he wasn't so discreet was in late 1870, early 1871, when he was discovered drunk by a policeman in front of the house of one of his lovers, Carolina Bregaro, the wife of the diplomat Rodrigo Delfim Pereira (he was half brother of Pedro II, as he was the son of Pedro I with the baroness of Sorocaba, who was the sister of the first emperor's most known lover, the Marchioness of Santos).
The rumours about the sexual life of Pedro II were known at the time. Writers as José do Patrocínio, Raul Pompeia and Artur Azevedo published short stories and theatre plays making fun of a scandal regarding the theft of palace jewels (the man who stole them had the keys to Pedro's bedrooms, and apparently "helped" the Emperor in "getting ladies" - despite the fact that the jewels were found in his home, the man was never prosecuted).
After the scandal of the jewels, even Barral wrote a letter to Pedro, telling him that he should change his way of life, as "when we are young many things can be excused, but in old age nothing is, and Your Majesty must give te example".