An important feature of the Soviet enrollment system was that all proposed problems should be taught in the course of the school program. If somebody was asked a question outside of the school course, such exam could be easily appealed. As such, an art of creating problems that were very difficult, yet could be solved with school techniques emerged.
The discrimination most likely arose as a policy, similar to "affirmative action" or "diversity programs" in the West.
For example, according to the Shen's paper, among the graduates of selected Moscow mathematical schools who applied to Mekhmat MSU in 1979, 47 were non-Jews while 40 were Jews (46%). This makes Jews the largest ethnic group to participate (of whom only 6 Jews and 40 non-Jews were enrolled). Further the article says that enrollment statistics from non-mathematical schools does not show ethnic discrimination.
Even after the filtering, the percentage of Jews in MSU and other universities far exceeded their percentage among the population which was about 1%. It was possibly seen as unsatisfactory by the proponents of the principle of the "equality of the result" as opposed to "equality of opportunities".
An interesting article by Mekhmat professor Ilyashenko furthers the allegation by a claim that the policy was not directed only against Jews, but against all talented and stronger pretenders. Especially it affected the graduates of the Moscow mathematical schools. In a given example, out of 100 graduates of Moscow mathematical school №57, only 3-4 could enroll MSU at best during the policy in force.
The policy as he claims, was justified by an idea to give a chance to students from "poor", "peasant" and "workers" families who showed much worse performance at expense of stronger ones. Since some teachers protested the policy, they were told "there are no bad students, there are bad teachers", a claim that even if somebody shows poor performance, it is not due to their natural skills, but due to insufficient teaching.
This was possibly supported by the story of Mikhail Lomonosov himself (after whom MSU was named). He being a peasant son made a way from the Russian North to St.Petersburg so to make a successful academic carrier.
Another justification was an idea that each student should be judged based not on universal standard, but depending on their individual skill level, that is, stronger students should be judged more strictly and should make more efforts to get the same marks as poor students, another variant of "affirmative action".
This led to the serious drop in the students' skills and abilities. As Ilyashenko tells, since the policy was implemented, the students who had all "satisfactory" (the lowest permitted to pass) marks became the majority while before the policy in force, such students were very rare. The university teachers were pressurized not to give "unsatisfactory" marks at all, so that the examiners had to justify each "unsatisfactory" before the administration, and created a special writing-book where they protocolled each "unsatisfactory" answer so to protect themselves against pressure.
By the way, there are similar accusations against the US universities as well, especially were accused the so-called "leadership" requirements which put social activity above academic skills, and allegedly were introduced to reduce the number of Jews in American universities.
P.S. The original paper by Shen: http://www.3038.org/press/shen.pdf