The decline of the "Muslim Golden Age" was due to multiple factors that are commonly associated with the decline of most major empires. Much of what was to blame for the decline was a condition referred to as imperial overreach. This is a condition where an empire which has undergone rapid expansion is unable to maintain control over the various territories it quickly acquired. This is typically because the political state in question has been unable to complete the long and costly process of incorporating a new group of peoples with their own culture and political customs.
This is definitely true of the Islamic empire which rapidly expanded during the reign of the first caliphs to succeed Muhammad, the Rashidun or "rightly guided caliphs". The meteoric rise of the Islamic empires can be seen in just the rule of one caliph, Umar ibn al-Khattab, who in a mere decade managed to conquer all of the Sassanid empire and two-thirds of the Byzantine empire. The very quick incorporation of sub-Saharan Africa, Arabia, Persia, and much of Spain left little time or ability for the caliphates to incorporate much of the very diverse territories they converted or conquered.
Another unique problem with the Islamic caliphates were their reliance on the figure of Muhammad. The farther removed from the prophet temporally, the less united the ummah was. This lead to a multiplication of sects such as the Shia Muslims, whose loyalty to the prophets offspring rather than the accepted caliphate (to whom the sunni, or traditionalists faction supported) only compounded the pressure on the caliphate by the strain of overreach. This overreach was also visible in its more traditional forms by the rise of the Mamluk slave armies to political prominence and the fracture of the Abassid caliphate into multiple dynasties of varying degrees of independence.