Erler first became a member of the German Bundestag in the 1987 national elections. Following the 1994 national elections, he joined the SPD parliamentary group’s leadership around chairman Rudolf Scharping. From 1994 until 1998, he served as the chairman of the Subcommittee on Disarmament, Arms Control and Non-Proliferation.
In the negotiations to form a coalition government with the Christian Democrats under Chancellor Angela Merkel following the 2005 federal elections, Erler led the SPD delegation in the working group on foreign affairs, his counterpart from the CDU/CSU was Friedbert Pflüger. Under the leadership of minister Frank-Walter Steinmeier, he subsequently served as Minister of State (alongside Günter Gloser) at the Federal Foreign Office from 2005 to 2009. When the SPD returned into the opposition from 2009 until 2013, Erler served as deputy chairman of his parliamentary group, again under Steinmeier’s leadership.
From 2014 until 2017, Erler served as the German government’s Coordinator for Inter-Societal Cooperation with Russia, Central Asia and the Eastern Partnership Countries at the Federal Foreign Office. From 2015 until 2016, he assumed the position of Special Representative of Germany for the country’s chairmanship of the Organization for Security and Co-operation in Europe (OSCE) in 2016, in addition to his other responsibilities.