One of the curious features of the Nazi genocide of European Jews is the degree to which they fetishized their victims, and even encouraged (if not forced) the outward display of Jewishness. German Jews who did not have stereotypically Jewish names needed to adopt a new middle name in 1938 (Sara for girls and women; Israel for boys and men), and even after Jewish children were prevented from attending state schools, they were still allowed to attend Jewish schools - at least for a time.
As the question notes, synagogues continued to operate in many locations, but a great many were closed down. The Wilker Shul in Lodz, for example, was boarded up, and only reopened for the filming of a mock synagogue service that was incorporated into the documentary film, Der Ewige Jude ("The Eternal Jew"), which screened in 1941.
In some camps, like Terezin, religious services were permitted, and in some others, were tolerated. An ability to pray in places like Treblinka, Chelmno and Auschwitz-Birkenau was thought to keep members of the sonderkommando pacified, and they were even allowed access in some of these places to religious literature that had been brought in by other Jewish arrivals.
Were there any aspects of Jewish practice that were ever made actually illegal? There were some - most notably the ban on kosher slaughter in Germany, 1933. Other examples were more ad hoc, such as preventing people from fasting on days like Yom Kippur, or disallowing public prayer in certain places of occupation (such as in various of the ghettos, etc) - but those weren't items of actual legislation.