The only book to survive from ancient Rome on architecture is Marcus Vitruvius Pollio's De architectura, found by the Renaissance scholar Poggio Bracciolini in 1414 after being largely forgotten. Bracciolini was one of a number of humanists "devoted to the revival of classical studies". However, it had been referred to by several churchmen, chroniclers and scholars during Medieval times, but comprehension was perhaps hindered by lack of understanding of some technical words and also missing illustrations. An English translation is available here.
"1521 Cesare Cesariano Italian translation of De Architectura Libri Decem (The Ten Books on Architecture) by Marcus Vitruvius Pollio.". Source & Attrib.
Numerous works have been lost to fires, wars, accidents, floods etc., and some because they were not considered worth copying by later scribes (or they came low on the list of priorities), but some other technical information did survive (for example, Frontinus' De aquaeductu was found, also by Poggio Bracciolini, in 1425 and made use of during the Renaissance). However, while there were people throughout the medieval period who could understand the language in which it was written, this is not the same as understanding and being able to exploit the knowledge contained therein. Vitruvius himself wrote at the beginning of his manuscript:
The architect should be equipped with knowledge of many branches of
study and varied kinds of learning, for it is by his judgment that all
work done by the other arts is put to test. This knowledge is the
child of practice and theory...
Cited in: The Oxford Handbook of Greek and Roman Art and Architecture
A certain degree of basic technical knowledge is certainly helpful. More importantly, skilled craftsmen would be needed to do the actual building. These were lost, along with Roman concrete. As Wikipedia on Roman Technology surmises,
Roman technology was largely based on a system of crafts. Technical
skills and knowledge were contained within the particular trade, such
as stonemasons. In this sense, knowledge was generally passed down
from a tradesman master to a tradesman apprentice. Being that there is
only a few sources from which to draw upon for technical information,
it is theorized that tradesmen kept their knowledge a secret.
Further, the kinds of building projects for which the Romans are famed (for example, aqueducts) require enormous resources. In the wake of the fall of the Western Empire and the economic decline that went with it, few states had such resources. Note that the most prominent structures erected in the Middle Ages - churches and castles - were built by those who could command the wealth and human resources to do it.
Note also that Roman architecture did not completely disappear; for example, the basilica was used in churches by the Merovingians and the Carolingians.
Joachim Henning (ed), 'Post-Roman Towns, Trade and Settlement in Europe and Byzantium: Byzantium, Pliska, and the Balkans' (2007)