In OPEC case, when cartel firms gain profit, they reinvest profit in new capacity by exploring new oil mining or investing in the new machine which able to produce higher level of output. When the demand for oil is relatively constant, reinvestment in new capacity leads to a higher excess capacity. The capacity to produce oil is underutilized and each country that joins Organization of Petroleum Exporting Countries (OPEC) has a higher excess capacity.
Underutilization makes an incentive for cartel members to use the excess capacity. In order to prevent that, the cartel makes side payments for the members. A side payment is made by one or more parties in a cartel agreement to other parties to induce them to join the cartel. If a country in OPEC produces oil by using its fully utilized capacity, it will increase supply of oil in the market and if the demand unchanged, the oil price will go down. If the price goes down, the profit of oil producers will decrease and it will make all members become worse off.
In order to maintain the price level of oil, the oil producers should maintain its level of output as well. Basically, each country wants to get higher profit by selling high level of output from its own plant but the country can be induced to agree to a plan which includes side payments from the member based on production share. Side payments can compensate the profit opportunity that cannot be captured because of the restriction of output.
As the excess capacity increases, the side payments that are needed to compensate OPEC members also increase. The country that has larger OPEC share have to pay a larger portion of side payments as well. The diagram below represents OPEC share countries.
From the diagram above, the fact that Venezuela and Saudi Arabia have a big capacity to produce oil implies that it is costly more to these counties because they have to pay larger side payments to compensate other countries.
In addition to that, the higher excess capacity of OPEC countries is tempting OPEC member countries to cheat from the cartel agreement and increase the oil production. By doing this, the countries can experience economies of scale which leads to a decrease in average cost. With lower cost, the countries can get higher profit. If countries who cheat on the agreement has a relatively large capacity than the other, for instance Saudi Arabia, it can produce with the full capacity production, sell a lower price and do predatory pricing to other members.
High burden of side payments and temptation to cheat on cartel agreement makes the cartel become more vulnerable and more likely to break apart. The other explanation regarding the destruction of cartel is the increase of competition from non-OPEC countries such as United States and Russia.
Source: Smart Planet Bulletin 2012
Increase in competition can drag the oil price down because of increase in supply. One of the factors of a high level of oil production from non OPEC countries is the increase level of oil exploration in those areas. The decrease in oil price internationally will reduce OPEC price as well in order to be able to compete with non OPEC countries. At the end, there is no further incentive to keep the cartel agreement.
There will be some impacts toward the long run viability of the cartel. In the long run, the cartel is unstable and difficult to sustain because there is a higher probability to breakdown. Whether members of the cartel choose to cheat on the agreement depends on whether the short run returns to cheating outweigh the long term losses from the possible breakdown of the cartel. The relative size of these new factors depends on how difficult it is for countries to monitor whether the agreement is being followed by other country. If monitoring is difficult, then the cartel members are more likely to cheat and the cartel will more unstable.
Several factors that make the monitoring become more difficult and thus make cartel to break down are as follow: a greater number of members, differentiated products, difference in production cost structure, fluctuation of market demand and low frequency of sales with high value contract.
Ceteris Paribus 
The excess capacity of the large members of a cartel serves purpose of a threat for smaller members that if anybody cheats than the cartel leader can drag down the price enough to make cheating actually causing a loss instead of profit.