Yes, the secession of the Confederate states was illegal.
It's important to understand that secession was not strictly, or merely, a “legalistic” matter. Secession was a hotly contested political issue that divided the electorate. It could not be settled by a judge. Any ruling in one direction or the other would be rejected by half of the population. See the last section here, esp the concluding two paragraphs, for a fuller discussion of this.
However, to the extent that secession was a LEGAL question, the “legalistic” aspect is very straightforward. In three sections: the law of the land, the arguments in support of the legality of secession, and why those arguments fail. Then two follow-up sections, on historiography and later legal commentators.
The Law of the Land
Unilateral secession has been illegal since the Articles of Confederation. No one ever mentions this, but the full title of that document was the "Articles of Confederation and Perpetual Union Between the States of [list]". (Emphasis added, obviously.) Article 13 laid out a possible mechanism for secession. It said:
"And the Articles of this Confederation shall be inviolably observed
by every State, and the union shall be perpetual; nor shall any
alteration at any time hereafter be made in any of them, unless such
alteration be agreed to in a congress of the united states, and be
afterwards confirmed by the legislatures of every state."
That language is awkward for modern ears to parse. I bolded the word "them" above. If "them" refers to the Articles, then this is a sentence about amending the Articles. But if "them" refers to the States, then this could be construed as a sentence about changing the composition of the States or changing the membership of the Union. So here is a potential legal way out of the Union: propose it in Congress, win passage there, and then get it ratified by every state. We might call that a "consensual", cooperative, amicable secession. But there's no support in the Articles for unilateral secession, where a state just decides on its own to leave.
The Articles created a much weaker national government, than the Constitution did. It follows that the stronger and "more perfect Union" created by the Consitution is, if possible, even more perpetual than the Perpetual Union created by the Ariticles. Certainly not conditional, or "less perpetual". That's the exact reasoning used by the Supreme Court in Texas v White. The Constitution says it is the supreme law of the land: there's no room for a state government to suspend or nullify it.
The debates at the Constitutional Convention don't have much to say about secession – probably because the issue was already settled by the "perpetual" nature of the existing Union. There is a letter from Madison to Hamilton on the topic, from the Ratification era. Madison says "a reservation of the right to withdraw... is a conditional ratification... it does not make N. York a member of the New Union... she could not be received on that plan." (Letter to Hamilton)
Buchanan (of all people!) made a similar argument. He pointed out some pretty strong negative evidence: in the long fight for Constitutional ratification, no one ever tried to persuade reluctant states by arguing that "the moment that any state felt herself aggrieved she might secede from the Union. ... What a crushing argument would this have proved against those who dreaded that the rights of the States would be endangered by the Constitution!" (From Buchanan's 4th annual message.) Generally the absence of counter-evidence is not the same thing as evidence; but this actually is a strong point. Constitutional ratification was protracted and difficult, with extensive discussion by the leading political minds of the day. They didn't just FORGET to mention a right to withdraw, if one existed.
The argument in support of secession
The basic argument is that the states agreed to enter into the US Constitution, like a contract, so they could decide to leave (terminate). But by itself that argument is severely flawed. Either party can opt NOT TO ENTER a contract; entering into a contract is voluntary. But once a contract is entered into, it takes BOTH parties to dissolve it. One party can't dissolve a contract unilaterally.
To support unilateral secession, supporters invoke what is called "Compact Theory": the Union is not a national government, but instead a compact of independent sovereigns, somewhat like a League of Nations. Nations can unilaterally rescind treaties; so, under Compact Theory, could the states unilaterally rescind their membership in the US. Compact Theory had a long and respectable pedigree in American politics. Jefferson was a proponent, and so was Madison (at least for a while).
This is really the entire argument. The states are not bound by anything like contract law. The states are sovereign, they entered into a compact, and they can withdraw from at will.
Why the argument fails
Compact Theory had already been rejected by the Supreme Court in 1816, 44 full years before South Carolina seceded. The case was Martin v. Hunter's Lessee (1816). By the way, 4 of the 6 Justices who participated in that ruling were Southern (one of them was George Washington's nephew). The ruling was that the Constitution was not an agreement between the states at all; rather, as the preamble says, it was ordained and established directly by the people. Justice Story wrote for the majority:
“The Constitution was for a new Government, organized with new
substantive powers, and not a mere supplementary charter to a
Government already existing. The Confederation was a compact between
States, and its structure and powers were wholly unlike those of the
National Government. The Constitution was an act of the people of the
United States to supersede the Confederation...”
The status of Compact Theory was reiterated in McCulloch v. Maryland (1819); still 41 years before secession. 5 of the 7 sitting Justices in that case were Southern (still including the nephew); and the decision was unanimous. McCulloch extended Martin, saying that "the constitution and the laws made in pursuance thereof are supreme ... they are binding on the states and cannot be controlled by them." This directly rejects Compact Theory, which would hold that the federal government is a creation of the states, where the states maintain superiority.
Both cases draw a distinction between the Articles of Confederation and the Constitution. In Martin the Court says that the Articles were a "Compact", but the new government is not. In McCulloch the Court says that there is no phrase in the Constitution, unlike the Articles, that "excludes incidental or implied powers". You can actually go back to Chisholm v. Georgia (1793) to see compact theory get its first beating from the Court. This was the original Court appointed by George Washington. That ruling was that the People directly established "a Constitution by which it was their will that the State governments should be bound." Supreme or sovereign power was retained by citizens themselves, not by the "artificial person" of the State of Georgia. 1793! The 11th amendment undid part of Chisholm, making it impossible for citizens to sue other states in federal court; but it conferred Sovereign Immunity, not Sovereignty. No support for Compact Theory there.
It's very straightforward. By 1820, Southern majorities of the Supreme Court had already rejected the only argument in support of the legality of secession, THREE TIMES. That's well before secession, and Texas v White.
If you're serious in asking the question (I suspect you are not), it's important to understand that in the historiography of the Civil War, this question is one of the Southern Apologist arguments. Saying that secession was legal until the Court issued its 1869 ruling in Texas v White, implies that there was no legal basis for the Court's ruling, ie it's pure judicial fiat. That's a pure Southern Apologist argument. The rest of the logic goes: "secession was not illegal until 1869, therefore secession was LEGAL up until 1869, therefore Lincoln's use of armed force to put down the slaveholders rebellion was illegal." The line of argument is used to paint Lincoln as a criminal, an aggressor, a dictator.
And it's not true. Texas v White was pure Stare Decisis: the opposite ruling would have violated previously stated law. The law underpinning the decision had been settled back in 1816.
Later Legal Commentary
There is some discussion about whether later legal commentary contradicts the argument from the 1816 and 1819 rulings. At least three times since 2000, judges have stated that the question of a state's ability to secede was unsettled prior to the Civil War. That has occurred in a Supreme Court ruling, in a Supreme Court justice's correspondence, and in a state supreme court ruling.
"At the time, the phrase 'one Nation indivisible' had special meaning
because the question whether a State could secede from the Union had
been intensely debated and was unresolved prior to the Civil War." (
Elk Grove Unified Sch. Dist. v. Newdow, 542 U.S. 1, 6 n. 1, 124 S.Ct.
2301, 159 L.Ed.2d 98 (2004). )
"If there was any constitutional issue resolved by the Civil War, it
is that there is no right to secede."
- In 2010 the Alaska state supreme court wrote that:
"While a state's ability to secede was an unsettled question before
the end of the Civil War, subsequent United States Supreme Court
opinions have concluded that secession is clearly unconstitutional,
and Lincoln's belief in a perpetual Union is reflected in what we have
described as 'a plenitude of Supreme Court cases holding as completely
null' the acts of secession by Confederate states."
Is there a contradiction here?
No. It's important to note that none of these jurists (except for the second half of Alaska, after the word "subsequent") reference any legal action or court rulings. SCOTUS 2004 does NOT say, "the question whether a state could secede was unresolved prior to Texas v White." Scalia does not write, "the constitutional issue resolved by Texas v White is that there is no right to secede." The Alaska court does not say, "a state's ability to secede was unsettled before Texas v White." What they all mention is the war. All of these jurists are saying that the Civil War itself settled the question. Not any court ruling: the actual WAR.
And that's the truth. Secession was not a legalistic question that the courts could settle, by examining the Constitution and accumulated Supreme Court precedent. A good comparison is the Dred Scott ruling. There, Justice Taney intended to settle the question of slavery in the territories, and of negro citizenship. The ruling did not come anywhere close to settling that question. If anything, the ruling inflamed the controversy. It is routinely cited as one of the incidents that escalated sectional tension, in the long lead-up to the war.
Likewise secession. Secession was a political (and military!) question. As late as the Summer of 1864, it was still an open question whether the Federal govt had the political support to defeat secession. Imagine if Lincoln had lost the 1864 election, it's easy to envision the winning "Peace Democrats" negotiating a peace, leaving the Confederacy in place. In that alternate hypothetical, a later SCOTUS would likely have to acknowledge that although secession might be de jure illegal, it had de facto occurred. That hypothetical Court might conclude that the US citizens had not shown a determination to maintain the Union; obviously it was possible to secede. It would be the opposite ruling not because the underlying law & precedents would have changed, but because the facts would have changed.
It took the WAR to settle the question of secession. That's what we see in history, and that's what the 21st century judges are commenting on.