I don't think Garfield would have vetoed the Chinese Exclusion Act. Remember that in his acceptance leter to the Republican national convention in 1880, Garfield had written that Chinese immigration was "too much like an importation to be welcomed without restriction; too much like an invasion to be looked upon without solicitude."
https://books.google.com/books?id=x8OggvKjZSgC&pg=PA200 He explicitly said that if negotiations with China failed to resolve the problem, Congress would have to act to restrict immigration.
Moreover, the forged "Morey letter" was an issue in the 1880 campaign. h
ttp://historynewsnetwork.org/article/8061 Garfield as president could not afford to do anything that might seem to vindicate the Democrats who claimed the letter was genuine.
True, the West Coast states where the Chinese issue was hottest did not have very many electoral votes, but nobody could be sure in advance that they might not be decisive in a close election. And even in other areas of the country, fear of the Chinese was real, if less intense.
As for the freedmen, all the Republican presidents of the late nineteenth century expressed sympathy with them--but it is questionable what was politically feasible to help them. There was certainly no widespread northern support for bringing back Reconstruction-style military intervention.