Confederate general Nathan Bedford Forrest famously said, "I git there first with the most men," often misquoted as "I git there firstest with the mostest." A Union cavalry general, John Buford, reportedly said, "A horse is just a form of transportation."
The main advantage of cavalry is that it could get to places, and at speeds and timetables not available to infantry. The disadvantage was that horsemen had to dismount to fight properly against infantry (in an era of rifles, especially "repeating" rifles introduced late in the war). Every fourth cavalryman held the horses of three others, so the fighting strength of the cavalry unit was effectively reduced by one fourth.
Confederates like Forrest understood that it was better to arrive first at a critical location with say, 2,000 men, of which only 1,500 could fight if it came to that. Especially when they were on a "raiding" mission, "hitting and running" would be much better than arriving with 2,000 infantry "later." The northern generals (other than southern-born Buford), were more concerned about the one-quarter reduction in fighting strength. A difference of philosophies leading to a difference in tactics.
This tactic was used less in subsequent wars as the spread of "repeating" rifles further increased the advantage of infantry over cavalry. Cavalry was most effective during the era of single-shot weapons such as muskets during which "contact" weapons such as lances or sabers were most effective.