Regular banknotes, issued in "normal times" and made from paper are practically all rectangular and without holes. But there were a couple of times when people remembered that anything can be "money", if they just believe in it.
Below are some examples of holes in notes and odd shapes. These examples were chosen to fit the question as originally framed. Some examples still fit the updated question: Zimbabwe's hole was officially re-issued, although not designed that way. Some examples of Notgeld were designed and issued in more or less round form. The last example even being completely irregular from the start as the material used was small and intended to be used for another purpose…
Having a hole like coins?
Hole Punched Notes
In 1997, Zaire’s (now the Democratic Republic of the Congo) dictator Josepeh Sese Seko Mobutu was finally overthrown. As his face featured on the country’s bank notes, the new government simply punched out Mobutu’s face out of their existing currency and continued to use it until new currency could be printed.
Hole Punched Notes, UKcredit, MoneyLife, 2017
If this is not that strict about "not a rectangle but still valid, legal tender", then:
Then the Portuguese did something weird to the edges of the Real:
Portugal's first paper money was introduced in 1797 by the government.5 Denominations issued until 1807 included 1200, 2400, 5000, 6400, 10,000, 12,000 and 20,000 réis. Some of these notes were revalidated for continued use during the War of the Two Brothers (1828 to 1834).5
From the 1820s, several private banks issued paper money. The most extensive issues were by the Banco de Lisboa, whose notes were denominated in both réis and moedas, worth 4800 réis. This bank issued notes for 1200 and 2400 réis, 1, 4, 10, 20, 50 and 100 moedas. The Banco Commercial de Braga, Banco Commercial do Porto, Banco de Guimaraes and Banco Industrial do Porto also issued notes, with bearer cheques issued by a number of other banks between 1833 and 1887.
In 1847, the Banco de Portugal introduced notes for 10,000 and 20,000 réis.6 5000 réis notes were issued from 1883, followed by 50,000 réis in 1886. In 1891, the Casa de Moeda introduced notes for 50 and 100 réis,7 and the Banco de Portugal introduced notes for 200, 500, 1000 and 2500 réis, followed by 100,000 réis notes in 1894.
And early American banknotes are weird as well.
Continental Currency. 1776
$1/6 Plate C Serial Number: 145,707 CC 02/17/76
Signer: Robert Tuckniss (in red ink).
Size: 80 x 60mm (front border design: the vertical dimension is 78mm while the horizontal border is trimmed on our example; back border design: 74 x 55mm).
Comments: Numbered and signed in red ink. The sundial with the "FUGIO" legend and "MIND YOUR BUSINESS" motto appear on the right center of the front. In this fractional denomination, one ornament appears in the upper right corner of the sundial frame; this is keyed to the denomination as each mark equals 1/6th of a dollar. In the right border cut "CURRENCEY" is misspelled. The back shows the thirteen linked rings representing the colonies and the legends "WE ARE ONE" and "AMERICAN CONGRESS". Paper contains blue threads and mica flakes.
Provenance: Purchased through the Robert H. Gore, Jr. Numismatic Endowment from the EANA mail bid auction of 1/13/96, lot 278.
(click to enlarge)
From a Swiss auction of "Banknoten und Notgeld", SINCONA, 2015, note that in that catalogue many more such examples are found. Sometimes the "holes" just look like ones as they are really a look-through window for the watermark. But often in times of crisis or other sudden changes they are really holes and the money retains its, well, 'a' value. :
If you insist on "designed and issued that way" (instead of 'remained in circulation') then I guess only the types of Mobuto and Soviet holes count (they were re-issued in that way), and only the paper-like Notgeld shapes count. Round and oval from the start, sometimes. That would make 1914–1923 pretty much the timeframe where to look for more examples (if we do not go much further back).
But really, anything can be declared money, even if it is inherently worthless. For example cut up playing cards, issued in 1914 in Lopischewo, East Prussia:
"Notgeld: Der schöne Schein", Spiegel, 2008.
And from previous threads here on HS:E:
Although the material is in both cases not even paper, the first was worth and intended to stand in for paper.
Just for entertainment, larger versions of the Notgeld, click to enlarge: