Ebrahim joined the liberation movement as a youth activist in 1952, and participated in the Congress of the People Campaign, that drew up and adopted the Freedom Charter in 1955. He was active in all the campaigns of the 1950s, and after the banning of the ANC in 1960, Ebrahim joined its armed wing, Umkhonto We Sizwe in 1961. He was arrested in 1963 and charged under the sabotage act with 18 other accused in the Pietermaritzburg Sabotage Trial. He was sentenced to 15 years on Robben Island. He was released in 1979, was banned and restricted to his home town in Durban. In 1980, as per instruction of the ANC, he went into exile. He operated from the frontline states bordering South Africa and was responsible for the political underground.
In December 1986, he was kidnapped from Swaziland by the South African Security Forces and detained in South Africa where he was severely tortured. He was charged for high treason and sentenced to 20 years imprisonment on Robben Island. In 1991, the appeal court ruled that his kidnapping from a foreign country was illegal, and that the South African court had no jurisdiction to try him. He was subsequently released from prison in early 1991.
In July 1991, he was elected to the NEC of the ANC and also became a member of the National Working Committee. During the CODESA negotiations, Ebrahim was tasked by the ANC to establish the Patriotic Front, which consisted of over 93 organisations. Its aim was to achieve consensus on both the negotiation process and the final outcome of the negotiations that led to the establishment of a democratic South Africa.
Ebrahim was elected a member of the National Assembly of Parliament in 1994. In August 1997, he was elected Chairperson of the Foreign Affairs Committee and also became a member of the Joint Select Committee on Intelligence. Ebrahim resigned from Parliament in July 2002, to take up the position of the Senior Political and Economic Advisor to the Deputy President of South Africa.
Since 2002 Ebrahim has been actively involved in international conflict resolution efforts. In May 2009 he was appointed as Deputy Minister of International Relations and Cooperation.