See also my answer to the homosexuality question and the Wikipedia articles on Pederasty and Age of Consent.
[Disclaimer: I hate to sound unprofessional, but I do not wish to appear to be condoning paedophilia: I find the idea revolting. Nevertheless we must be able to talk about history in a scientific, distanced manner.]
[Edited:] The answer to your question depends on your definition. In ancient Athens and Rome, the modern concept of paedophilia did not exist as such, so the question would be meaningless to an Athenian if you asked him. If by paedophilia you mean the modern psychiatric definition, which pertains to sex with pre-pubescent children, that was considered abnormal and probably as rare as now.
If you use the popular modern definition, which includes all children that cannot be considered adults, so up to 16 or 18 years old, that was not uncommon in Greece and Rome. Flirtation and sex between a bearded man and an unbearded, pubescent boy was acceptable or even common, but only in certain contexts, in certain circles, in certain cities, at certain times. This answer is about sex between grown men and pubescent children. [End of edit.]
In Antiquity, unmarried sex usually did not impact a man or a boy's reputation; but girls were supposed to remain virgins until marriage. So attitudes towards sex were tied to marriage for girls. The discrepancy was probably connected with the common situation of a younger girl marrying a (somewhat or much) older man. Another relevant factor was the fact that men can procreate at a late age, while women have a fairly limited timeframe.
Pubescent girls could be married off to adult men, which is still common practice in many parts of the world. Until ca. 1800, a very common minimum age of marriage in Europe was 12 years old. Only after 1800 was this gradually raised in the West. This probably reflected social attitudes. The modern Western boundary of 16 or 18 years (which means after puberty) for marriage is relatively new; while I believe the average age of marriage has been increasing for centuries, 16–18 probably only became universal in the West somewhere during the 20th century.
That doesn't mean girls were mostly married off during puberty in the Middle Ages and Antiquity; but it does mean that marriage was often legally possible at 12. One can imagine that consummation was often (though not always) to happen soon after. Sometimes sons and daughters who were supposed to forge an important alliance were even married at 9 or younger; but then marriage was often only allowed to be consummated later.
A famous example of this widespread (perhaps nearly universal) practice was Mohammed. From Wikipedia:
Traditional sources dictate that Aisha was six or seven years old when betrothed to Muhammad, but the marriage was not consummated until she was nine or ten years old. While the majority of traditional sources indicate Aisha was 9 (and therefore a virgin) at the time of marriage, a small number of more recent writers have variously estimated her age at 15 to 24.
Even if the traditional story is not true, this shows that the consummation of a marriage at 9 years old, which was perhaps rare and would seem inconceivable in modern society, was at least not absurd then. Nor was it in Rome or Greece.