When you go to a shop, you work out all prices in cents and never use dollars? You measure all times in seconds, not minutes or hours? All weights are in pounds, never ounces, tonnes, or gallons/pints? All distances are given in feet, not miles, yards or inches? If that's the case, then all I can say is that your life is much more conveniently arranged than the examples I'm familiar with. As is the case for "grassroots" maths, I suspect that quite a lot of unit conversion goes on in people's day-to-day lives, they just don't notice it because it's so familiar to them.
But perhaps I'm unusual in that I still have a need to do basic arithmetic every so often. I suppose other people don't have to worry about such things, and so the issue just never comes up for them.
Length I generally never have to convert. I don't know how often anyone actually uses yards outside of sports. Inches, feet and miles get a lot of use but not yards. I feel like that's half the problem with metric- the meter its main unit of length measurement isn't terribly useful.