This is always a tough discussion and everyone has a point to a certain degree, all are correct, depending on your viewpoint. I choose to understand history through my eyes and that means I have several positions or displays that may offend, ok, so be it. I also am a card carrying member of the Jewish Tribe by marriage, I am of high German descent, my relatives fought on both the Union and the Confederate sides. A Confederate Battle Flag as well as the First National Flag hung for years in my room with pictures of my relatives (both sides) as well as other historic prints. Not a member of the KKK, the flag never meant that to my family but they pilfered its use during the Civil rights movement and turned it from historic symbol or a "symbolic rebel" symbol into a symbol of hate. Well, I don't care b/c that is not what it means to me. I don't care what color, gender, religion, preference you are, it is none of my business and I get annoyed that we see it plastered as an issue every day of our daily lives.
End of day - as has been said, collect what you like, respect other's opinions, don't force your views on others (it goes both ways) and if this would occur, people would agree to disagree and get along! And on top of it all, enjoy the hobby.
And if you are a racist person who collects any toy soldier for that reason whether it be Roman, Nazi, Mongol, Civil War, etc, well, then, I feel sorry for ya, but its your business!
TD