I will share some excerpts from the highly relevant article, "The Negro in Imperial Russia: A Preliminary Sketch" (1976), in which Allison Blakely argues that imperial Russia was
a land of opportunity where [blacks] could not only survive, but could
attain high social position. This is striking not only because of the
absence of the stigma attached to their color in the United States and
certain other places. It is also significant that this is a group
which was not part of the regular upper class and was conspicuously
Negroes present in tsarist Russia fall into three main categories. The
first is the small native Negro population which for at least two
centuries was scattered in small settlements in the Caucasus mountains
near the Black Sea. Until the beginning of the 20th century even the
general Russian public was not aware that they were there. [...] The second category of Negroes in Russia is comprised of the servants and workers who were present in large numbers in the 18th and 19th centuries, owing to the fashion among the wealthy nobility and tsars of maintaining a certain number of black servants. They were denoted by a number of terms; the one I will use here is "arap." [...] The third category of Negroes in Russia was comprised of visitors of various types, including artists, athletes, and foreign service officials. Some of this category of visitors actually stayed for periods of several years.
In addition to the specific example of Gannibal, Blakely mentions the prominent visitor Ira Aldridge, a Shakespearean actor who was made an honorary member of the Imperial Academy of Fine Art. Although negative racial stereotypes toward black people did exist in Russia, there were not the same kinds of discrimination and social barriers as in more profoundly racist societies such as the United States.