tl;dr
Yes, arguably. There were several repeated assurances that NATO would not expand, given only orally or in letters, public newspaper articles, but no official treaty signed over this. This was heard from the Soviet side and is still interpreted by the Russian side today as "a promise". But it was a mere prerequisite for opening talks with the Soviets about the Germany merger in the first place and later left aside from both sides, for different reasons.
And No, arguably. The view that this promise was "broken" might be seen as an over-interpretation since in the meantime to admitting new NATO-members several changes occurred, among them the political development between the Soviet Union successor states, former Warsaw Pact members and Nato that ended in signing a treaty with Moscow in 1997 that made membership for Poland etc possible. The last treaty might be seen as the Russians renouncing the promise of non-expansion of NATO (leaving out certain key elements that lead up to this treaty and its "promises") — and therefore the West as released from a promise that the West never considered legally binding anyway.
This is no easy story.
Promises made leading up to West-German expansion ("re-unification")
Gorbachev insisted for quite a long time that the West – that is politicians and diplomats of various ranks from a number of NATO member states, chief among them Baker, Kohl and Genscher – indeed "promised" to the Soviets that NATO would not expand any further Eastward after the GDR was absorbed into West-Germany. In his autobiography and later in interviews he repeatedly told that this was the case:
The Americans promised that Nato wouldn't move beyond the boundaries of Germany after the Cold War but now half of central and eastern Europe are members, so what happened to their promises? It shows they cannot be trusted.
(From: Gorbachev: US could start new Cold War (2008))
Some Western analysts now come to the same conclusion:
Russia's got a point: The U.S. broke a NATO promise (LA-Times 2016).
A German magazine with a generally strong anti-Russian leaning in recent years writes:
After speaking with many of those involved and examining previously classified British and German documents in detail, SPIEGEL has concluded that there was no doubt that the West did everything it could to give the Soviets the impression that NATO membership was out of the question for countries like Poland, Hungary or Czechoslovakia.
This stream of assurances started and was necessary to get the opposing Soviet side to accept German unification. That is, before, during and after the 2+4-Talks it was negotiated that the GDR would be absorbed into the NATO-member West-Germany and the so enlarged (West-)Germany would still remain a regular member of the Western military alliance:
Earlier in the conversation, Baker had given assurances to Gorbachev that were to play an important role several years later in the vehement Russian opposition to any further eastward expansion of NATO – to include former members of the Warsaw Pact and even former Soviet republics like the Baltic States. If Germany were to remain part of NATO, Baker said, “there would be no extension of NATO's jurisdiction for forces of NATO one inch to the east.”[…]
In the (deliberately created?) atmosphere of ambiguity the stage for the formal consent was finally set. By mid-July, in a plethora of private talks and the Two Plus Four meetings at the foreign ministers' level, clarification was achieved as to the form that NATO's first eastward expansion could take:
- Non-integrated German units could be stationed in the former GDR immediately after Germany regained full sovereignty, and German NATO-integrated forces after the withdrawal of Soviet troops, but no allied forces.
- Germany would not produce or possess nuclear, bacteriological or chemical weapons.
- NATO would transform its structure and its role in Europe, emphasize its political role.
(From: Hannes Adomeit: "Gorbachev’s Consent to Unified Germany’s Membership in NATO", Paper Delivered to the Conference on “Europe and the End of the Cold War,” at the Université de Sorbonne, Paris, June 15.17, 2006.)
This is all about the interpretation of words. And it is also about the reframing of history. A formal agreement to not accept any states from Warsaw Pact into NATO was never written out in a treaty. That is true. But it's not only Russia's current view on the matter that concludes that the leaders of the Soviet Union were "tricked". Subsequent aims and offers from the Russian side to improve relations and co-operation were often just ignored. Western archives support this view to a certain extent:
U.S. Secretary of State James Baker’s famous “not one inch eastward” assurance about NATO expansion in his meeting with Soviet leader Mikhail Gorbachev on February 9, 1990, was part of a cascade of assurances about Soviet security given by Western leaders to Gorbachev and other Soviet officials throughout the process of German unification in 1990 and on into 1991, according to declassified U.S., Soviet, German, British and French documents posted today by the National Security Archive at George Washington University.
The documents show that multiple national leaders were considering and rejecting Central and Eastern European membership in NATO as of early 1990 and through 1991, that discussions of NATO in the context of German unification negotiations in 1990 were not at all narrowly limited to the status of East German territory, and that subsequent Soviet and Russian complaints about being misled about NATO expansion were founded in written contemporaneous memcons and telcons at the highest levels.
The documents reinforce former CIA Director Robert Gates’s criticism of “pressing ahead with expansion of NATO eastward [in the 1990s], when Gorbachev and others were led to believe that wouldn’t happen.” The key phrase, buttressed by the documents, is “led to believe.”
(From NATO Expansion: What Gorbachev Heard – Declassified documents show security assurances against NATO expansion to Soviet leaders from Baker, Bush, Genscher, Kohl, Gates, Mitterrand, Thatcher, Hurd, Major, and Woerner
National Security Archive at the George Washington University)
Several newspaper articles from that era and the following years confirm these recent "discoveries":
New York Times 1990: Upheaval In The East: Soviet Union; Kohl Says Moscow Agrees Unity Issue
Der Spiegel 1993: „Unser Traum wird früher Wirklichkeit“
The relevant parts of that German magazine article are almost repeated in English here:
New York Times 2009: Enlarging NATO, Expanding Confusion:
What would Mr. Gorbachev demand in return? To learn the answer, Mr. Baker and Mr. Kohl journeyed to Moscow within a day of each other. On Feb. 9, 1990, Mr. Baker asked Mr. Gorbachev, “Would you prefer to see a unified Germany outside of NATO, independent and with no U.S. forces or would you prefer a unified Germany to be tied to NATO, with assurances that NATO’s jurisdiction would not shift one inch eastward from its present position?” Mr. Gorbachev, according to Mr. Baker, answered that “any extension of the zone of NATO would be unacceptable.” Their meeting ended without any final deals made. Mr. Baker left behind a secret letter, detailing what he had said, for Mr. Kohl in Moscow.
While Mr. Baker was in Moscow, though, members of the National Security Council back in Washington were worrying about his comment that NATO would not move eastward. To undo the damage they felt Mr. Baker had caused, they drafted a letter that President Bush sent to Mr. Kohl later that day.
The presidential letter included language that differed in a subtle but significant way from the language offered by the secretary of state. Instead of a pledge about NATO’s borders, Mr. Bush suggested that East German territory be given a “special military status” within NATO. What that status would consist of was to be negotiated later, but the core assumption was clear. NATO would grow and former East German areas would have a special status within the alliance as it did so.
A foreign leader can see daylight between a president and his secretary of state from the other side of the world, and Mr. Kohl did not have to look that far. He just had to read the differing phrasings used by Mr. Bush and Mr. Baker to notice it. So whose language did Mr. Kohl echo in his own talks with Mr. Gorbachev the next day, Feb. 10 — the president’s or the secretary’s?
Mr. Kohl chose to echo Mr. Baker, not Mr. Bush. The chancellor assured Mr. Gorbachev, as Mr. Baker had done, that “naturally NATO could not expand its territory” into East Germany. The documents available do not record Mr. Kohl using the presidential phrase — “special military status” — that the National Security Council had rushed over to him. Mr. Kohl’s foreign minister, Hans-Dietrich Genscher, visiting the Kremlin as well, assured his Soviet counterpart, Eduard Shevardnadze, that “for us, it stands firm: NATO will not expand itself to the East.”
Did NATO 'promise' not to expand?
Yes. Countless times from 1989–1993 representatives of member states did just that. To Gorbachev and the SU as well as his successors. Just not in writing; that is: in the form of a signed treaty. Politicians and others, regardless of affiliation, claiming the opposite today, are just lying. Even the German parliament disseminates official "factsheets" to its members, detailing for example that Baker's later denials are counterfactual. (Just look at this video, showing Baker standing right next to Genscher when Genscher repeats this non-expansion promise, declaring even assimilation/annexation of the GDR to be out of the question. [Youtube, snippet from a show on the largest German TV station, rebroadcast 2014, 2:28min])
It is not believable that those proponents of this view just do not know any better. Time has progressed and times have changed and the lack of a signed treaty on this, that went through all steps of formalisation, ratification etc may be interpreted as somewhere between "no promise" or "long ago"/"the view at the time", "necessary ruse", or "irrelevant today".
While Moscow was never a real fan of NATO's eastward expansion, the current interpretation from the Russian side favours this view of outspoken promises, that were not kept afterwards or really broken in 1997/99.
One note concerning how the question is currently phrased: Whether anything "required NATO to reject any membership applications from the nations formerly part of the Warsaw Pact" is a tricky phrase, and can be easily answered with a resounding "No". "Required" and "reject" are words that were indeed never used in public discourse. It was always something with less precision, like "move", "expansion", without naming names of any other states.
Assurances after the fact — and this time: treaties signed
From: LeMo NATO-Erweiterungen
On the other side of the debate, NATO expansion was seen internally as a largely anathematic idea from 1989 to 1993. Too risky and certainly not open for open debate, whatever think tanks and single politicians produced in the meantime. This changed only in 1993 when German minister of defence Volker Rühe named the elephant in the room during the security summit (IISS) and opened a debate that was met with great skepticism after the years of taboo. This coincided somewhat with the still perceived a threat of Soviet/Russian forces still stationed on German soil. Once this concept got more traction after Bill Clinton adopted it:
Initially, it was not something that most people wanted to discuss, let alone support. Many said it could not be done. It was too hard, too ambitious. It was suggested that the American people were not willing to expand their commitments in Europe after the end of the Cold War. Some predicted a policy train wreck with Moscow and the derailment of Russian reform. (Senator Lugar, Arora p126.)
The following years were filled with attempts and initiatives to reduce Russian opposition for enlargement. This time hinting at the possible future membership of Russia itself, since the rhetoric in the day was to bring "Eastern Europe countries back into Europe". Both sides proposed several possible concrete forms of co-operation, of which for example the Partnership for Peace did materialise, ineffective as it was. The Russian leadership under Yeltsin gradually lowered its guards in hope of future cooperation and:
On 27 May 1997, Russia and the Alliance signed the “Founding Act on Mutual Relations” at the Elysée Palace in Paris. With the signing of the Act, NATO had now won Russian acceptance of its right to enlarge. (Arora, p195.)
From: Chaya Arora: "Germany’s Civilian Power Diplomacy. NATO Expansion and the Art of Communicative Action", Palgrave Macmillan: New York, Basingstoke, 2006.
Conclusion
There were assurances given by NATO representatives, and deliberately left ambiguous, and kept out of any treaties at the time of German unification. The Russian side was led to believe and did believe these words as a promise. Whether it was caused just by the speed of unfolding events, the general political turmoil across Eastern Europe or short sighted trust misguided through misinterpretations, r even personal ineptitudes, in the end the Soviets had no signed papers about this promise to show for.
In the following years of imagined peace dividend a process unfolded that more often than not alluded to a very close partnership or even a possible membership of Russia in a very much enlarged NATO, that might then be more accurately described as a political organisation and less of military alliance. In the light of these developments Russia then formally agreed to NATO enlargement. Due to the fact that neither membership nor even partnership enfolded in the way Russia envisioned or hoped for, but produced only several dysfunctional paper tigers, led to the conclusion in Moscow that the West had betrayed Russia all the time. Both sides have valid points in interpreting the political or moral obligations that might follow from the words exchanged, especially in late 89 and early 90. Morally or compared to an interpersonal level, NATO's argumentatio is weak and low, legally, from a perspective of international law, their position might be called "correct". The later denials from some of those involved at the time on the Western side are often walking very wobbly the line between re-interpretation of history and blatant lies surrounding the events.