Risk Markets And Politics

Sunday, May 21, 2006

Crytpophilia

When of the first things most people realize when they get into systematic trading or "technical analysis" (if they are rigorous and honest with themselves, at least) is the danger of trading rules that are excessively tailored to the vagaries of past data. Whether this methodological pit-fall of data-mining is called "data-snooping" or "back-fitting", the idea is the same: the specific prices and desired result of testing had too much influence on model selection, and out-of-sample predictiveness may be lacking. Over-complicated trading signals that produced excellent returns in the past will tend to fare worse going forward, and in markets that weren't included in their development. Even apparently significant and elegant signals are more likely to fall to this sort of mean-reversion if they were developed through an intensive search. Likewise, John Holland in Hidden Order, while outlining his model of complex adaptive systems, favors simplicity in order to avoid what he calls the "unwrapping" problem. The mark of a good scientific model/theory is its capacity to make additional predictions outside of the immediate questions it was designed to address.

Considering this background, it is perennially amusing to hear about decoded bible messages and the like, in which there is little doubt that the cipher was the message, and many ciphers had been tried on a large body of text in order to produce a handful of messages. Sure enough, it's another version of the "uncertainty principle". Paranoia, both the everyday variety and the clinical condition, follows similar a pattern. Namely, projecting meaning into possibly random circumstances (and, additionally, overestimating small probabilities). Ron Howard's portrayal of John Nash comes to mind.

In light of how misleading code interpretation can be, what is its source of fascination? It is a decidedly literal version of proverbial "search for meaning", and also carries a subversive attraction. On one hand it can be an appeal to authority, and on the other, an act of resistance. Intentional codes, including metaphor and other tropes, can disguise the transmission of knowledge to kindred spirits in a possibly hostile environment. The word "code" can likewise refer to the ordering principles of a group, as in a secret society. In the end, everyone wants to be an "insider".

In today's political climate, is it surprising that during the "war on terror" and related controversies regarding privacy, the right to encrypt has not faced a greater challenge? Probably not. It would be difficult to enforce, and intelligence agencies might feel, as is often (amusingly) the case in the corporate world, that an email's sender and recipients can be just as telling as its content. Insofar as an encrypted message would then in itself constitute a "red flag", steganography would become more important anyway.

In the 1980s, David Chaum of MIT produced some crytpographic research with a potentially more intimate bearing on politics. Namely, a cryptographic protocol for online elections. The special problem that online elections represent (from a cryptographic standpoint, at least) is as follows. The election results must be verifiable and correct. Only registered voters should be permitted to vote and only once. However, voting must remain anonymous. It should not be possible to determine who voted for whom. Chaum overcame this tension through an ingenious use of public key cryptography and blind signatures.

While the specific cryptographic problem was nicely solved, many obstacles still remain for the implementation of such a protocol. It is more work to forge physical identification, and so online voting might encourage the buying and selling of votes, or the identity theft of non-voters. Some actually argue that allowing votes to trade freely might be a good idea, but in any case, the more serious obstacle to online voting likely involves more gross and obvious methods of falsifying results outside of the encrypted transmissions, as one hears with regard to the Diebold systems in the 2004 presidential election. These systems were laughable in their security features, and fell far short of Chaum's work, regardless of the partisan circumstances. Clearly, traditional physical voting systems are also subject to fraud. Online elections might actually be more secure as data could be stored in multiple locations and later cross-verified to detect tampering by local agents. When I implemented such a protocol in 1997 as part of an undergraduate project, I also noted that online elections might have interesting effects insofar as they reduce the "cost" of physical voting, and that such technology could be used for "less serious" opinion collection where participants might also wish anonymity, as with television ratings and viewing habits. In a way, this is reverse data-mining: aggregating information while obscuring certain relations, with the aim of maintaining privacy.

2 Comments:

  • In data mining people refer to the problem as "overfitting". There are many methods for tackling this problem, such as cross-validation (where the predictions are checked on cases that were not used for model-building), bootstrap (perturbing the input data), Bayesian priors, regularization, and so on. The cruel reality is that picking such a technique may itself be an act of overfitting. In contrast to overfitting, the pitfall of the bold, underfitting is the reluctance to make valid predictions, the excessive timidity.

    There has been some work on privacy-preserving data mining, with some interesting mathematics. A good starting point is the Agrawal&Srikan paper.

    By Anonymous Aleks Jakulin, at 6:28 PM  

  • For a list of online futures trading brokers and a profile of an online
    futures broker recommended by others visit futures trading

    Futures trading and commodity links. Features tools, indicators and charts
    future trading

    Futures Truth Top 10 Trading Systems
    future trading system

    By Anonymous futures trading, at 9:22 PM  

Post a Comment

<< Home