Artillery comprises large, heavy engines throwing large, heavy missiles. Size is important in an artillery crew because a larger man can perform the same tasks of loading and aiming the engine faster, and longer without fatigue, than a smaller man. These are plain physical attributes of the technology being discussed.
Artillery propels missiles on a ballistic arc; a parabola if you will, though in the strictest sense that is not quite accurate. In order to be effective the missile must hit a target at a given distance, and thus it is necessary to calculate a launch angle that will result in the missile's arc intersecting the target's motion, at a moment in time when the target is occupying said location. This truly is the rocket science of the pre-calculus era. It must be calculated correctly, quickly, and under all the stresses of being in a dire life-and-death situation; battle. Yes, one smart small man can coordinate the efforts of a team of larger men; until that smaller man becomes a casualty. If one wants a reliable and robust artillery corps, every man on every crew must be capable of stepping up to the next most complicated role, at some reasonable level of efficiency; and then be able to step again at some level of efficiency.
In consequence of these basic facts, one can only have a reliable, accurate, fast-firing, and robust artillery corps if every crew member is in training, at some level, to eventually be crew-captain, section lead, and then battery commander in sequence. Not all will make it of course, but that is the career progression because no-one can predict who will be a battlefield casualty.
But, you say, just pre-calculate a table of ranges and elevation angles for each weight of missile. That sounds fine, until one remembers that this is an era when less than 5% of the population is capable of reading simple text, let alone complicated and concise artillery tables; and even fewer are numerate. There is no printing press, so each crew lead would likely have been required to make his own copy of the tables during his training. As part of learning how to read these tables, and of copying them, and of how to correct and then calculate them, he is by definition entering the intellectual elite of your society. As paper and ink and tutors and training time are expensive, any sane commander will only recruit individuals for such training who can demonstrate true talent for the work, and who are useful during training as effective crew members.
How does one properly select artillery crew then? One takes the largest 1/3 or 1/4 of the available men, and then takes the smartest 1/3 or 1/4 of those.
There is no other sane means to do it.
So frankly, any ruler who tried to form an artillery corps from other than brawny brainy soldiers would have quickly been proven unfit to rule by his peers, and have even more promptly disappeared from history.
Even more frankly, a military incompetent like Hitler can only rise to command armies of millions in an environment where an extremely effective general staff, and even more talented general officers, exists to hide his mistakes for several years. To the best of my knowledge and research, no such combination ever existed prior to the formation of the Prussian General Staff in the 1800's.
Update - Requirements to qualify for artillery training in U.S. Military:
To ... "manhandle" the standard round for the 155 Howitzer, a shell weighing 117 pounds, not simply lifting it once, but repeatedly, over a long or even an indefinite period.
Strength Requirement: Very Heavy (Lift on an occasional basis over 100 pounds with frequent or constant lifting in excess of 50 pounds.)
ASVAB Score Required: 93 in aptitude area FA (Field Artillery): Arithmetic Reasoning, Coding Skill, Mathematics Knowledge, and Mechanical Comprehension.
