The size of the British Army varied a lot through the 63 years of the Victorian era. The Wikipedia article on the British Army includes a table of personnel figures from 1710 to 2015. If we look at the section for 1801 - 1921, which includes the Victorian period, we can see that the army establishment was at its lowest at the beginning of Victoria's reign, with the Army numbering 130,000 in 1840, and peaked at the end of her reign when the Army numbered some 275,000 in 1900.
As you have already found, the Wikipedia page on the British Army during the Victorian Era provides some figures for the numbers of cavalry and infantry regiments at this time.
An infantry regiment would be commanded by a Colonel. The number of men in a regiment would depend on the number of battalions that made up the regiment, and the relative strength of those battalions. As a very rough guide (see below) you might say that a foot regiment's establishment numbered about 1000 men for most of the Victorian period.
A cavalry regiment might consist of up to between 600 and 900 troopers.
A regiment in the British Army consists of one or more battalions, each battalion consisting of a number of companies. The number of battalions in a regiment has never been fixed, nor has the number of companies that make up a battalion. Even the size of companies changed over time as requirements changed!
To illustrate, consider the examples of the 71st (Highland) Regiment of Foot and the 74th (Highland) Regiment of Foot (the later famously involved in the 1852 Birkenhead disaster which gave us the Birkenhead Drill of "Women and children first".
The 71st (Highland) Regiment of Foot was raised in 1777/78 and initially consisted of a single battalion of 1032 Officers and Men of all ranks. A second battalion was raised the following year, but disbanded in 1783. The 2nd Battalion was restored in 1804, and disbanded again in 1815.
The Regiment then remained as a single battalion until 1881. In 1818, the establishment of the Regiment was reduced from 810 to 650 rank-and-file, which was further reduced to 576 rank-and-file in 1821. Companies were then added and disbanded as required until 1881 meaning that the strength of the regiment varied between about 600 and 1100 men over that period.
The 74th (Highland) Regiment of Foot was raised in 1787. It consisted of a single battalion, made up of 10 companies totalling 902 Officers and Men of all ranks. In 1821, at the end of the Napoleonic Wars, the regiment was reduced to 8 companies each nominally of 72 rank-and-file (about 500-600 men in total).
Companies were added and removed as required until the regiment was amalgamated with the 71st to form the Highland Light Infantry as part of the Childers' reorganisation of the infantry regiments of the Army on 1 July 1881 (the 71st became the 1st Battalion, HLI, and the 74th became the 2nd Battalion, HLI).
The strength of the regiment thus varied between about 600 and 1000 men over the early part of Victoria's reign.
Regiments consisting of only a single battalion for most of Victoria's reign prior to the Childers' reforms of 1881 was by no means unusual (I just chose the 71st and the 74th because they happen to be the two that know best at this period). Most of the regiments that were amalgamated by Childers' reforms followed this pattern.
So, as a (very) rough guide, you might say that for most of the Victorian era - at least until 1881 - an infantry regiment would number up to about 1,000 men when at full establishment (but be aware that this generalisation masks a lot of variation).
In addition to the infantry and cavalry, the British Army employed a number of specialist units, including the Royal Engineers, Royal Artillery, Army Works Corps (a precursor of the Royal Pioneer Corps, established during the Crimean War), etc.
In 1862, the total strength of the Royal Artillery was:
- 29 horse batteries,
- 73 field batteries, and
- 88 heavy batteries
having been bolstered by the artillery units of the Honourable East India Company (see below).
If you are interested in specific regiments of infantry or cavalry, it is worth noting that Wikipedia has pages dedicated to each:
Another point to remember is that, under the provisions of the Government of India Act 1858, the UK government took over the Presidency armies (the armed forces of the Honourable East India Company). However, strictly speaking, most of these troops then formed the 'Indian Army', rather than being counted as part of the 'British Army' (an exception here were the 21 horse batteries and 48 field batteries of the Honourable East India Company, which were amalgamated directly into the Royal Artillery in 1862).
If you're interested in the British Army in the later part of Victoria's reign, I can recommend The Late Victorian Army, 1868-1902, by Edward M. Spiers (Manchester University Press, 1992).