IIRC, Galilee and Nazereth were actually a bit of cross-roads then, and not nearly as poor as they might be today.Well, the fact that he was a carpenter does not mean he had the wealth necessary to own slaves. Nor does the fact that he was literate. The Jews, as a community, were somewhat unique in that regard in that time period. Most Hebrew men of the period would have been at least somewhat literate, as it was required to read Torah in synagogue.
It is, of course, possible that Jesus's father owned slaves, although unlikely, given that he was a carpenter in a poor village in Galilee, which does not indicate a very wealthy person. Jesus himself never established his own household, and it is never indicated in the Biblical text that he owned property of any kind either before or during his ministry. So it is unlikely in the extreme that he did own slaves.
Some think that some of the missing years (between the time He became an adult and the time He starts his ministry) may have been running (helping to run) that carpentry shop until e.g. sibs got old enough to take over. The fact that we have a TOTAL blank on his life from the time he was 12 until he was 30 means it's very hard to rule out most anything for that time period. (E.g. Jesus is (fully God and) fully human. Part of the human condition is marriage. Jesus was probably married during part of that time. See, you can build LOTS of reasonable scenarios.)
Certainly, by the time he starts his ministry, he has no household or possessions of his own.