It is true that just "hanging around" was difficult. But not impossible.
As much as most of those going underground were depending on others to help them survive, the government (shorthand that includes police, etc) was also depending on snitches to get those hiding. The Nazi state needed Nazi citizens and collaborators to not waste extreme resources to comb through every house in every city or to turn every stone beneath a tree in the woods.
These measures were taken, of course, but not all the time, not everywhere and for every single 'wanted'. Such actions were taken when the Nazis saw a group to catch or the locals were in the mood for manhunt.
That is most surviving illegally in illegality did that in cities, like Michael Degen with false identities or completely hidden away, like Anne Frank. Charlotte Knobloch and Hans Rosenthal and their stories are further prominent examples of the more than 10000 Jews surviving in some form of hiding within Germany.
Those using 'the woods' to hide would include people like Peter Bielenberg or Otto Wolf.
Marie Jalowicz Simon: "Underground in Berlin. A Young Woman's Extraordinary Tale Of Survival In The Heart Of Nazi Germany" (Untergetaucht. Eine junge Frau überlebt in Berlin 1940–1945, transl Anthea Bell), Knopf Doubleday, 2014.
Those that had some kind of 'revolt' on their agenda, did that from cities and using the woods as well. The boundaries between these quite fluid, and as dangerous as less successful.
Barbara Weinhold: "Eine trotzkistische Bergsteigergruppe aus Dresden im Widerstand gegen den Faschismus", ("A Trotskyist Mountaineering Group from Dresden Resisting Fascism."), ISP: Köln, 2004.
Better known groups would be Edelweisspiraten like Fritz Theilen. When threatened some of these resulted to violence, including weapons use, like documented for the Ehrenfeld group.
One rare example of a fugitive really living alone and off the land would be Franz Krönauer. Well, almost alone and off the land. After deserting the Wehrmacht he fled to the woods around his home village. He hid in the forests and avoided any contacts as the village was like Germany as whole: full of Nazis and those who would have ratted him out. But some knew that he was 'around' and tried to deposit food stuff that he was to supposed to find.
He had to resort to eating grass and could only rarely talk to his brother when he was out attending goats. He managed to survive for four years among trees. (Srcs 1, 2, 3, 4)
For a more general consideration of 'surviving in the wilderness in Germany': between 0.5–4% of the whole area is today classified as wilderness, but more regions are and were less densely populated compared to Rhineland for example. In Hesse, Bavaria, Thuringia and Saxony especially there are comparatively remote areas like Hohe Tauern or Hainich. Even in economically exploited areas that are located in Mittelgebirge or alpine regions hiding would have been possible with somewhat elevated chances for some time.
For the really "criminal minds" (according to modern or conventional standards, there is one example calld Muscle Adolf head of the Ringverein Geselligkeits-Club Immertreu 1919 e.V..
Contary to Wikipedia information, where it is said the he was aprehended and probably died, he survived Nazi rule:
Muscle-Adolf, protagonist of the central story in this article, resurfaces in a post-war police file. Having survived the Third Reich’s terror, in June 1946 he was picked up for illegal gambling, in a pub in north Berlin. The police confiscated the staggering sum of more than 11,000 Reichsmark and five decks of cards, alongside precious coffee beans, half a pound of salmon and a tin of caviar. Muscle-Adolf was involved in black-marketeering, an illegal activity though almost acceptable in the context of bombed-out Berlin and shortages. (There is no evidence, though, that the Ringvereine ran the black market.)80 The omerta` of the Ringvereine still functioned: nobody would testify against Muscle-Adolf and the police had to close the case.81 Despite their claims, the Nazis had failed to wipe out the Ringvereine, which continued to operate through the political ruptures of 1933 and 1945 as their networks and rituals of belonging held their members together. The history of the Ringvereine reminds us of the continuity of German everyday life beyond the political caesura of 1933.
–– Christian Goeschel: "The Criminal Underworld in Weimar and Nazi Berlin", History Workshop Journal, Volume 75, Issue 1, Spring 2013, Pages 58–80, 2013. DOI
Unfortunately this omerta still seems to hold… We do not know what he did in that time, how he did it, etc.
But this Mafia-like form of organisation went partly underground.
Some of them continued in their profession. That "the Nazis" clamped down on criminals is true. Well, at least they tried. But that they succeeded in doing so is a nazi proganda myth:
The Nazis propounded the myth that the Weimar Republic had been subverted by criminal gangs and that they had repressed them. The increas- ingly influential legend of omnipresent criminal gangs helped undermine popular legitimacy of the Weimar Republic. It is not a coincidence that much of the surviving evidence on the Ringvereine in the Weimar Republic was preserved in the late 1930s by Nazi legal officials to reinforce the Nazified history of Weimar as overridden with crime. Many Germans perceived the Republic as a state that was unable to maintain the monopoly of violence, whether the challenge was political or criminal. Such views re- sulted from newspaper stories, books, plays like the Threepenny Opera and later films such as M. Yet crime was not simply an aspect of Weimar’s popular culture.
In the polarized political culture of the Weimar Republic accusations of criminality featured prominently. From around 1923 Hitler and the Nazis constantly denounced the founding fathers of the Republic as ‘November criminals’, Socialists, Communists and Jews, who had stabbed the victorious German army in the back in 1918. Supporters of the Republic found the term offensive, yet, conflated with the popular stab-in-the-back legend, it began to pervade political discourse.85 For the Nazis and many on the Right, the Weimar Republic was completely illegitimate: they claimed that it had been founded and was being run by criminals. The Nazis exploited the widespread concern with crime and used it to justify their violent methods to reach power and, after 1933, to consolidate it. The Nazi myth of organized crime thus linked the political and the criminal and became a very potent political instrument. The Nazi myth of crime also had a strong racist di- mension and, at least to some extent, helped legitimize Nazi racial terror. Leading Nazis, including Hitler and Himmler, identified the criminal under- world as part of a Judeo-Bolshevik conspiracy, typically equating criminal- ity, Communism and the Jews with one another, though obviously with no evidence.86 The Nazi myth of crime, dismantled in this article through a case study of the Ringvereine, was a powerful, yet complicated conflation of politics with crime and race. It was a central Nazi strategy to gain popular support, and it was so successful that the myth still resonated with many Germans for decades after 1945.
This is evidenced by the still rising numbers of criminals apprehended under the accusation of "habitual and perpetual, hardened professional criminal" throughout the nazi regime being in power.
Patrick Wagner: "Volksgemeinschaft ohne Verbrecher. Konzeption und Praxis der Kriminalpolizei in der Zeit der Weimarer Republik und des Nationalsozialismus", Hamburger Beiträge zur Sozial- und Zeitgeschichte, 34, Christians: Hamburg, 1996.
Arthur Hartmann & Klaus von Lampe: "The German underworld and the Ringvereine from the 1890s through the 1950s", Global Crime, 9:1-2, 108-135, 2008. DOI: 10.1080/17440570701862835
Nikolaus Wachsmann: "Hitler's Prisons. Legal Terror in Nazi Germany", Yale University Press: New Haven, London, 2004.