The Wikipedia reference on February omits quite a bit of information. Simply put, the Roman religious year ended on 23 February (a.d. VII Kal. Mart. in the Roman counting backward and inclusively way...i.e., the 7th [6th in our counting] day before the first of March), with the Feast of Terminalia. (Before the Julian calendar, if they inserted an Intercalary month, February was in fact cut short at 23 days and the extra month then inserted between February and March.) Thus, when the Julian calendar was introduced, the natural place to insert a day was right after that. So in leap years, instead of the 24th day of February being a.d. VI Kal. Mart., the 25th day of February was a.d. VI Kal. Mart. and the 24th day was Bis a.d. VI Kal. Mart. (the second sixth day from the Kalends of March....of course, it precedes the first sixth day before since we are counting backward). Note "second sixth" hence the alternate name for leap year: "bissextile year".
In the Middle Ages, they kept the Roman numbering of days, so what we now call the 24th of February really was the leap day. Even after modern day-of-month notation came into vogue, the Catholic Church kept feast days that were on 24-28 February in normal years shifted by one day to 25-29 February in leap years, and this was not generally changed until 1970.
Note that saying Terminalia was the last day of the Roman sacred year does not mean that the next day was the first day of the new year. The sacred new year really did start on March 1 (as opposed to the political new year which had been January 1 since the mid-2nd century BC). But the last 5 days of February were the precursor to the new year celebration, and not part of the previous year. That's why intercalary months cut February short by 5 days (and then the entire intercalary month was a run-up to the new year with the festivals that precede the new year being celebrated on its last five days).