In addition to the "patrician" Optimates and "plebeian" Populares, there was also a third group - the Equites - which today we might think of as a kind of 'upper middle class'. To complicate matters still further, by no means all members of the Patrician class were associated with the Optimates faction. Perhaps the best-known example is Julius Caesar, who belonged to the Patrician class, but was politically aligned with the Populares (although quite how far he shared the Populares agenda remains a matter for debate).
You are absolutely right that the three groupings were not remotely like political parties in the modern sense. In modern political parties, people tend to give their loyalty to the party and follow a party line. The groupings in ancient Rome were rather shifting groups of alliances between smaller sub-groups of wealthy and powerful people.
Political support in Rome was primarily mobilised by patronage or clientela - the relationship between wealthy, influential patrons and their poorer but free dependants. The concept is complex to modern eyes (and I suspect it was not much less complex to ancient Roman eyes). It had some elements that we might think of as the more modern notion of noblesse-oblige. Other aspects we might associate with the techniques employed by modern drug lords or Mafia kingpins. These could involve acts of charity and patronage or through the use of enforcers with clubs and brass knuckles. If a patron changed sides, he would expect his clientele to come with him.
The shifting allegiances could give rise to some very strange outcomes (at least to modern eyes). Consider the election of Crassus and Pompey as co-consuls in 70BC (they hated each other) or, even more bizarre to modern eyes, the election of Caesar and Bibulus in 62BC!
It may be easier to understand the way that the system worked if you remember that most votes were controlled by this network of patronage. Individual patrons were often themselves caught up in the network of patronage of more important figures. It would not be at all unusual for someone to repay a favour to one patron by calling their clients to support a particular candidate (say from the Optimates) for election as a Praetor, while simultaneously also agreeing with another patron to support a candidate from the rival camp (in this example, the Populares) for election as an Aedile at the same time.