If you get rich in a business you soon find that you have no reasonable 'line of work' for your money, or in other word: capital. So you get essentially too rich for meaningful expansion in your core business. And now you can start either to waste it around for personal luxuries or other consumption — or you throw your money around as an 'investor' or money lender if you still want to make even more money. They diversified. Not all cloth merchants went into banking and there were other sources of accumulated wealth. The Welser family had a big stake in lucrative cloth trade but was founded on silver mining in Tyrol.
This is the story from the Italian merchants, the Fugger onwards to today's petro-dollars — it doesn't matter whether the base came from cloth, grains, oil or selling books over the internet, stealing an operating system or creating a search engine, high demand and control of prices make a very nice profit:
Cloth merchant: In the Middle Ages or 16th and 17th centuries, a cloth merchant was one who owned or ran a cloth (often wool) manufacturing or wholesale import or export business. A cloth merchant might additionally have owned a number of draper's shops. Cloth was extremely expensive and cloth merchants were often very wealthy. A number of Europe's leading banking dynasties such as Medici and Berenberg built their original fortunes as cloth merchants.
And
Merchant banks were in fact the first modern banks. They emerged in the Middle Ages from the Italian grain and cloth merchants community and started to develop in the 11th century during the large European fair of St. Giles (England), then at the Champagne fairs (France). As the Lombardy merchants and bankers grew in stature based on the strength of the Lombard plains cereal crops, many displaced Jews fleeing Spanish persecution were attracted to the trade. The Florentine merchant banking community was exceptionally active and propagated new finance practices all over Europe. Both Jews and Florentine merchants perfected ancient practices used in the Middle East trade routes and the Far East silk routes. Originally intended for the finance of long trading journeys, these methods were applied to finance the medieval "commercial revolution".
The most important point to observe here is that neither wealth as such nor certainly not any cloth has anything to do with 'founding a bank'. How delicious and perishable is cloth, says the ever as hungry as a capitalist moth. The key is that an assumed and believed amount of money exists, and that business partners believe that the key player will predictably honour his obligations. The repeated use of 'believe' gives the essential hint: believe in Latin is credere, and what we see is another transformation of the debt system into the credit system we still use to account.
This is demonstrable with undermining the premise that 'one needed to be a cloth merchant to become a merchant banker':
Italian banking houses such as the Bardi, Peruzzi, and Medici did much better. In banking history, the Italians are most famous for their complex joint-stock organization and for spearheading the use of Islamic-style bills of exchange.(Graeber/Debt)
The Bardi started with just general trading, Peruzzi started with wheat trading and went into cloth before banking, with cloth being the point they knitted an international trading post system. Acciaiolis also started with general merchandise in Cremona (or Brescia), while the Solaros seem to have gained enough capital before reliable historical information about them came about to appear as jump-heading into Lombard banking style.
Counter examples to cloth merchant status needed are also the Gondi and Strozzi families:
Strozzi is the name of an ancient (later noble) Florentine family, who like their great rivals the Medici family, began in banking before moving into politics. Until its exile from Florence in 1434, the Strozzi family was by far the richest in the city, and was rivaled only by the Medici family, who ultimately took control of the government and ruined the Strozzi both financially and politically. […]
And indeed, already Rosso Arduino Strozzi and Pietro Bueno Strozzi were rich enough to recreate their own past as 'being enobled by Charlemagne' and with the Gondis claimed such ancient descent as to hold offices in this system as patricians to be already named by Dante as "ancient", and thus rich…
Going back to the two examples mentioned in the question: both were quite late to the game. Metzler has the first traceable pure financial transactions only in 1728, while its founder Benjamin was indeed mainly in the cloth trade. But Berenberg is much more colourful. Hans and Paul were indeed sons of a Belgian cloth trader, and they dealt also in cloth with Merchant Adventurers. But an even larger part of their trade was dyes, cereals, fruit, salt, and spices like ginger and pepper. Being Hamburg based and especially well connected to the Netherlands, Baltics, Russia, Portugal and England: They were proper 'pepper sacks', like their other Hanseatic or Augsburg and Nuremberg based Upper-German counterparts, before concentrating on money-making alone.
The other spice traders called peperzaks were then the people of the Dutch East India Company VOC:
The VOC was a driving force behind the rise of Amsterdam as the first modern model of international financial centres that now dominate the global financial system. With their political independence, huge maritime and financial power, Republican-period Amsterdam and other Dutch cities – unlike their Southern Netherlandish cousins and predecessors such as Burgundian-rule Bruges and Habsburg-rule Antwerp – could control crucial resources and markets directly, sending their combined fleets to almost all quarters of the globe.
During the 17th century and most of the 18th century, Amsterdam had been the most influential financial centre of the world. The VOC also played a major role in the creation of the world's first fully functioning financial market, with the birth of a fully fledged capital market. The Dutch were also the first who effectively used a fully-fledged capital market (including the bond market and the stock market) to finance companies (such as the VOC and the WIC). It was in the 17th-century Dutch Republic that the global securities market began to take on its modern form.
And it was in Amsterdam that the important institutional innovations such as publicly traded companies, transnational corporations, capital markets (including bond markets and stock markets), central banking system, investment banking system, and investment funds (mutual funds) were systematically operated for the first time in history. In 1602 the VOC established an exchange in Amsterdam where VOC stock and bonds could be traded in a secondary market. The VOC undertook the world's first recorded IPO in the same year. The Amsterdam Stock Exchange (Amsterdamsche Beurs or Beurs van Hendrick de Keyser in Dutch) was also the world's first fully-fledged stock exchange. While the Italian city-states produced first formal bond markets, they did not develop the other ingredient necessary to produce a fully fledged capital market: the formal stock market.
The Dutch East India Company (VOC) became the first company to offer shares of stock. The dividend averaged around 18% of capital over the course of the Company's 200-year existence. The launch of the Amsterdam Stock Exchange by the VOC in the early 1600s, has long been recognised as the origin of 'modern' stock exchanges that specialise in creating and sustaining secondary markets in the securities (such as bonds and shares of stock) issued by corporations. Dutch investors were the first to trade their shares at a regular stock exchange. The process of buying and selling these shares of stock in the VOC became the basis of the first official (formal) stock market in history. It was in the Dutch Republic that the early techniques of stock-market manipulation were developed. The Dutch pioneered stock futures, stock options, short selling, bear raids, debt-equity swaps, and other speculative instruments.
Finally the Höchstetters of Donauwörth/Augsburg again are evidence for going from weaver/taylor to then head of corporation, only then going into cloth merchant, getting rich enough for small aristocracy, then pooling their funds to start trading with spices, then cloth and minerals like silver, iron ores. When they went 'bank' they became fantastically rich in a very short time, and knew how to get richer: same as before, not only through trade but by usury and monopolies. Just like the other merchants – of all kinds – they tried to get some kind of a monopoly. With Fugger it was local dominance in copper and silver, and Höchstetter tried to get a world wide monopoly in mercury.
The pattern to observe is that cloth isn't necessary for this, but the profits made from international long distance trade – that might involve cloth – forms a nice base to transform profits into capital. By coincidence, in England typical merchant bankers like Child started from (being a weaver's son in one case) as goldsmiths. Another way to go for riches is of course drugs. In this case the well known family of Merck.
And if taken together we see this answered as the example in 'original accumulation of capital'.
How well positioned as a group cloth merchants emerging from their late medieval standing were to bring together local dominance, monopolies and trade from, into, and around 'protected' markets – with local lines of credit already present to expand on in the form of their trading outposts throughout Europe – may be analysed with classical means.
— John Munro: "The Monetary Origins of the ‘Price Revolution’ : South German Silver Mining, Merchant-Banking, and Venetian Commerce, 1470-1540'", Department of Economics, University of Toronto, Working Paper No. 8, 2003.
— Meir Kohn: "Merchant Banking in the Medieval and Early Modern Economy"
Working Paper 99-05, Department of Economics, Dartmouth College, 1999.
— Peter Kriedte: "Peasants, Landlords And Merchant Capitalists. Europe and the World Economy, 1500-1800", Berg: Warwickshire, 1983.