The subdivisions of heaven and the theme of a vision of an ascent to heaven originate from Jewish mysticism. Different parts of the Talmud come from different times, but this idea is very old.
During the 5th century BCE, when the works of the Tanakh were edited and canonised and the secret knowledge encrypted within the various writings and scrolls ("Megilot"), the knowledge was referred to as Ma'aseh Merkavah (Hebrew: מעשה מרכבה) and Ma'aseh B'reshit (Hebrew: מעשה בראשית), respectively "the act of the Chariot" and "the act of Creation". Merkavah mysticism alluded to the encrypted knowledge within the book of the prophet Ezekiel describing his vision of the "Divine Chariot".
They also occur in early Christian writings not recorded in the Bible and in other religions of the ancient near East. The Jews often viewed heaven in terms of a temple. God dwelled in the inner most place in the temple and the closer one came to God the more elevated and sacred that place in heaven was.
Here is an excellent source about Temple Motifs in Jewish Myticism
The idea of the celestial ascent is one of the most widespread and long-lasting religious concepts in history. 9 Archaic, nonbiblical ascension myths from Mesopotamia and Egypt date back to the early third millennium B.C. 10 Within the Jewish tradition, this idea can be seen in the writings of Isaiah (eighth century B.C.) and Ezekiel (sixth century B.C.). Related and expanded versions of the ascent to the celestial temple are found in pseudepigraphic Enoch materials dating in their current form to at least the second century B.C., Qumran documents (second century B.C. to first century A.D.), Philo (c. 20 B.C. to A.D. 50), 11 and in numerous other Jewish and Christian apocalyptic and pseudepigraphic writings
A hell with subdivisions and fallen angels as devils is probably first described in the first Book of Enoch. There might be an earlier example, but I'm uncertain. It's dated to the 2nd century BCE.