Quantitative trading series | Part 4 – Commitment of Traders
As I promised last time, I will finally delve into my quant beginnings. I consider the work in COT analysis to be my truly first quantitative achievement. COT, or the Commitment of Traders, is the report published by the CFTC (US federal agency for the supervision of commodity markets). The report is actually a huge table with positions of various market participants. Traders with futures positions above certain limit must report their positions to the CFTC every week. These positions are then summed up and categorized into groups. First major group are hedgers – i.e. subjects dealing with the physical commodity. Their goal is to get rid of the risk of unexpected price movements. They make money from their business operations, not markets. Second major group are large speculators such as investment banks or hedge funds. These institutions take over the risk of price movements from hedgers. Their reward for doing this is the chance for profit. It’s not the purpose of this article to explain the basics of COT analysis. The point is the COT data provides valuable insight into the structure of the market. People who are able to interpret the data correctly get a huge advantage in trading commodity futures or spreads.
However the interpretation of COT data isn’t easy. Raw data are stored in a CSV table with more than 100 columns. It’s nearly impossible to find important values and understand their evolution over a certain period of time. Therefore COT data are always visualized in charts. As you can imagine, downloading the huge tables and selecting relevant columns in Excel is very lengthy. However, this is exactly the way I was doing COT analysis in my beginnings. It was during the years 2007 – 2009. At that time, I was using COT to trade ETFs on precious metals, so I had to manually sort the data only for gold and silver. Even so, it took me about 15 minutes every week. The idea that I might visialize data for most commodities in this way was absolutely unreal. Then came the end of 2009 and the CFTC began publishing COT data in a new, disaggregované form. Practically, this meant separation of swap dealers into a separate group. During the bull market in oil between 2007 and 2008 some banks disguised speculative positions of big funds for as swaps for clients, which are not subject to such strict position limits. Basically the data in new format better reflect the activity of the hedgers in the market.
In the early 2010 I had a dilemma how to incorporate disaggregated data in my analysis. It was so fresh that even paid services hadn’t been offering it. Of course, I could remake my Excel spreadsheet, but I knew it doesn’t really make sense in the long term to analyze COT this way. That was the moment when I decided to create my own program for the COT analysis. During the summer of 2010, I created a set of scripts in Bash (Unix shell scripting language). Gnuplot commands were used for generating charts. Suddenly I got the ability to create charts of the distribution of positions for dozens of commodities within a minute.
I used to send my COT analyses to several fellow traders. Because sending dozens of charts was impractical, I decided in late summer 2010 to move calculations on a standalone server and create a website where the charts would be displayed. I attach here an example of two such charts for gold (pale blue drawings are mine):
Why do I write about all this? It’s because it demonstrates one of the fundamental advantages of quantitative trader. Sometimes you find yourself in a situation, where you need to analyze data in a way that the trading platform or any other service doesn’t offer. I was exactly in this situation in 2010. I needed to visualize COT data in a way nobody was offering. The only option left for me was to do it myself. Thanks to my own coding abilities I created probably the first website in the world where you could find COT analyses of disaggregated data.