At its peak, around 1860, about 1.4% of the total U.S. population owned slaves. This comes from census data showing roughly 393,975 slaveholders out of a population of 31 million. In the South, where slavery was concentrated, about 25% of white households owned slaves, with the majority owning fewer than five. The percentage varied by state, with higher concentrations in Deep South states like Mississippi and South Carolina, where over 40% of white families were slaveholders.
Not to mention, all races have been slaves in the world, including White people.