As of now, no explanation has been provided. There are guesses; even conspiracy theory has been evocated. However, the real issue is briefly explained in this quotation from Robert Reich.
…the nation’s and the world’s capital markets have become a vast out-of-control casino in which fortunes can be made or lost in an instant — which would be fine except for the fact that most of us have put our life savings there. Pension funds, mutual funds, school endowments — the value of all of this depends on a mechanism that can lose a trillion dollars in minutes without anyone having a clear idea why. So much of the market now depends on computer programs and mathematical models that no one fully understands…
Algorithmic trading, flash trading and high frequency algorithmic trading are practices made possible by the role of software in the stock market. As stock trading and other financial transactions became more and more digitalized there was a window of opportunity for automated strategies in dealing with finance.
October, 19th of 1987 was the first time the world, at large, became aware of the effects of program trading, which is, trading performed by software. As a result, the NYSE introduced the circuit breaker, a forced halt to avoid a deadlock state on a selling spree.
Since 1987, financial organizations have built teams of gifted young Ph.D.s who specialized on implementing different sorts of strategies, based on mathematics and statistics, using advances on data structures, algorithms and hardware. These people are known as quants. Quants do not necessarily have a degree in Computer Science, but do write algorithms or work closely with people who does.
The use of software over the time span of computer´s networks opened a series of new possibilities in the trading business, where a combination of a huge amount of data and speed played a crucial role bringing new possibilities on volume and price volatility. Exploring this is more akin to gamming than to real trading.
However, the crucial point is that by mixing different sorts of strategies and relying on software over a complex network is a risk business. Of course those institutions are aware of the risks, but we are not sure of how much software engineering knowledge is being used in these systems as to avoid losses.
Notwithstanding, if the market is seen as a game, it is hard to know if you win by luck (an error on the other trading party) or by a fair strategy.
On the other hand, society must have some way of monitoring the quality of trading as to avoid huge mistakes, as per the 1987 crisis and the May, 6th incident. It is seems that the solution used in 1987, halting the market, did not work in 2010, maybe because the speed is different, or maybe because the transactions provided a way of working around the circuit breaker. The point is that there is a need for other type of policy to avoid such problems.
We believe that transparency is the best way to do it. In the specific case, the idea of Software Transparency, that is software has to be transparent to the people who may be affected by it, seems one to explore.
Read more about Software Transparency on a recent paper at the BISE journal (here).