beelzeboss,
Its derived from negro (black in spanish), nigger was and is a condesendig term. Whites enslaved blacks and brought them to the new world, not the other way around.
But the history of slavery goes much beyond that and has been practiced by most cultures and races in some way.