The recent shakeup in Yemen has effectively ended former president Saleh’s influence in the country and is a welcome development, but it is far from sufficient to end his legacy of 33 years of misrule, mismanagement and corruption. With much support from Saudi Arabia, the GCC countries, the United States and the UN Security Council, President Abd Rabbu Mansur Hadi has stripped Saleh’s son, Ahmad, and his nephews of their military fiefdoms and appointed them to diplomatic posts overseas. Saleh himself has left for Saudi Arabia. Another pillar of the Saleh regime, General Ali Muhsin Al-Ahmar, the former commander of the 1st Armored Division, has also been stripped of his power.

The Yemeni military, which for long has been a law unto itself and has dominated the country’s economy, is now being restructured, and hopefully it can be turned into a force that stays permanently out of politics and becomes obedient to civilian authority. This reform process is far from over, and even if it succeeds there are other political forces in Yemen that have to be contended with because they constitute a serious threat to the long-term stability of the country.

The first among these is the Al-Ahmar family, the paramount sheikhs of the Hashid tribal confederation whose leading member is Sheikh Hamid Al-Ahmar. The Al-Ahmar sheikhs were full partners with Ali Abdullah Saleh during his long period of rule and benefited considerably from this relationship. They amassed great wealth and power and only split from President Saleh several months into the uprising that led to his resignation. That break happened because Saleh was preparing to hand power over to his son, Ahmad, as if Yemen were a monarchy. The political party that the Al-Ahmar sheikhs control is Al-Tajammu Al-Yamani li-l-Islah, or simply Al-Islah, the Yemeni branch of the Muslim Brotherhood. As a political phenomenon, Al-Islah represents a unique combination of tribal leadership with an Islamist ideology and has played an important role in seizing control of the Yemeni uprising at the expense of the young people who led the demonstrations in “Change Square” in Sana’a and elsewhere in the country.

Ahmar, who effectively leads Al-Islah, is worth several billion dollars because of his control of Yemen’s cell phone network, and he hopes to win the presidential election scheduled for 2014. Other than his intimate association with the much-hated Saleh regime, Ahmar represents a divisive force in Yemeni society. He is hated by many from South Yemen, who feel they have been occupied and exploited by Northerners, like former president Saleh or Ahmar himself. Similarly, the powerful Zaydi movement in the Northern provinces of Yemen, the Huthis, equally hate Ahmar because he was part of a sustained military campaign against them, as well as being an ally of Saleh.

The central problem with Yemen today is that it is a highly fragmented and divided country, with no national leadership that can unite a majority of the population around a vision or program for the future. Many, if not most, Yemenis in the South want to secede from the country, which was united in 1990 by the joining of the northern Yemen Arab Republic with the southern Peoples’ Democratic Republic of Yemen. Any visitor to the South today will see the Southern secessionist flag painted on many houses and public buildings and squares. The Southern secessionists refuse to be involved in the National Dialogue conference that was initiated in March to resolve the problematic political legacy that Saleh left behind. Many, if not most, Southerners feel they have no future with the Northerners, whose leaders have systematically expropriated their land and stripped them of their jobs since the civil war of 1994.

The Huthis in the north have similar grievances about systematic discrimination by the government in Sana’a, and have no confidence that men like Ahmar represent a new beginning for the country. Left to themselves, the Yemenis will not be able to resolve their differences and civil war cannot be excluded as a possibility.

Furthermore, President Hadi has no substantial political base and cannot stabilize the country on his own, which is why we have seen him relying so heavily on outside powers like Saudi Arabia, the GCC, the US and the UN.

While the steps that President Hadi has taken are noteworthy, much more needs to be done in order to prevent Yemen from becoming a failed state. The Huthis and Southerners have to be brought into the political system and given a stake in the future of the country. For this to happen, Saudi Arabia, the GCC and the United States have to become even more involved in the reconciliation process. The Huthis are viewed as proxies of Iran, but this is not an inescapable relationship—they can be easily loosed from Tehran’s grip if given certain guarantees of religious and political autonomy.

Similarly, the Southern Yemenis must be given greater control over their own region and no longer be at the mercy of the rulers in Sana’a who have acted as predators. In other words, a new political structure has to be devised for Yemen, one that breaks with the corrupt and brutal leaders of the past and that also grants Yemenis a measure of local autonomy. Without these two developments, it is hard to imagine Yemenis believing in the future of their country.