I will refer to the science based claims of the OP questions leaving aside social and political reasons. I think the other answer though factually fair is a little loaded with personal opinions which are not factual. To be precise the risk of severe adverse effects is 1 in a million.
Risk for UK fatal car accidents is 1 in 20000 or according to UK parliament almost 1800 deaths per year although it has come down from much higher.
If all the population of Britain were to be vaccinated there between 60 and 100 people would die. So the risk would seem out of all proportion with the hysteria of the risk. Of course not all people are vaccinated every year so the death toll is much lower. https://researchbriefings.parliament.uk/ResearchBriefing/Summary/CBP-7615
On the other hand there is 10 percent to 30 percent risk of death for people who contract smallpox. And a third of all smallpox cases led to blindness. In the 18th century between 10 and 30 percent of all deaths were caused by smallpox.
Even in the 19th century with poor hygiene and such benefits overwhelmingly outweighed the risks. See sciencedirect article above.
So to say the benefits outweigh the risks is a little bit like saying Everest is higher than a sand dune by the beach.
So, undeniably the vaccine was a sign of the march of progress. As everything in life it had risks. Some people might be better off not vaccinating if there are specific risk issues.