May 18, 2024

Model 5: Pioneering Technological Advancements through Precise Backtesting

Introduction

We are excited to share the latest progress update on our Model 5 research and development efforts. The Model 5 represents a significant leap forward in technology. With improved accurate backtesting techniques, testing different strategies, and we have made remarkable progress in Model 5 development.

We would like to start with some background knowlege in our Model 4. You may be wondering why we are developing Model 5. Model 4 was a great success, but we found that it was not as safe as we wanted it to be. We wanted to make sure that our Model 5 is as safe as possible, as we will be starting to put more funds into our Model 5. Additionally, we can diverify our assset under management (AUM) into different strategies, which will help us to reduce the risk of our investment. Hence, we are developing Model 5.

Model 4 used the backtesting and live trading architecture we developed a year ago. There was huge effort put into designing the architecture, and we are proud of the results. However, by the end of developing Model 4, we noticed few problems that we wanted to address in Model 5.

1. Accuracy (gap between backtesting and live trading)
2. Speed
3. Performance

Keeping this in mind, let's dive into the latest advancements in Model 5.

Latest Innovations

For Model 5, we wanted, safe and consistent performance such that we could deploy more AUM to the system with less risk.

To do so, we conducted huge amounts of time and resources on backtesting numerous strategies with different hypotheses. This led to the breakthrough of Model 5 that dramatically improved the performance with safety and consistency—with a little bit of problems.

For Model 5, we developed completely new architecture. For major parts, we had to come up with new backtesting and live modules, and use some of the modules from Model 4. Based on characteristic of Model 5, we needed extremely short term anomalies detection and rapid execution.

When we first tried the Model 5 live, it was terrible in terms of accuracy, speed, and performance. Iteration of continuous fixes and testing led to a huge blow of cash. What we thought would work on backtesting did not work on live testing. We saw slight improvements each fixes, but another major breakthrough was needed—which we did.

We improved the speed and optimized our Model 5 such that, even with slight delay, it is able to maintain its performance. Along with this, there were many other factors that was playing a role with this gap of backtesting and live trading. We will talk about this more in depth, in “Fixing Fetch, Brokerage API, and Architecture” section. We want to point out that this took about as same, if not, even more time than researching Model 5. What was even more painful was, we had to burn lots of cash in order to test this expensive model. This was certainly very tricky task than we ever imagined. The amount of engineering that was put on to Model 5 was insane.

We added slight layor of safety protection in Model 5 that does not cause huge performance drawdown. However, there are still concerns with the safety that we want to improve on.

Precise Backtesting

One of the most important improvement was very in-depth precise backtesting with extreme details. Whenever we backtest, or live test, performance and the statistics didn’t provide huge insights on how the model was doing. Once we integrated huge precision and details into backtesting, we were able to improve the model with much better precision.

Shown below, are some of our performance graphs of Model 5. During the research and development period, we tracked various results of backtesting. We shortened the period of data for better precision and added intraday precision for tracking the results. We could test various factors that make a difference in live trading like delays and conditional orders.

Unfortunately, we deleted some of the sensitive information and blurred the graphs.

Extreme precision allowed us to consider every factor that causes a difference in live trading vs backtesting. Delay and bid-ask spread are the most problematic factors we are still improving on though, to further increase performance.

Fixing Fetch, Brokerage API, and Architecture

To start with, fetch, brokerage API, and architecture are bugs that led to accuracy and speed problem, which caused lots of cash to test Model 5.

Due to Model 5's characteristics of extreme short term trading period, backtesting was not enough. We had to test the Model 5 live.

Not surprisingly, backtesting and live trading are two different things. They do not always match. This was bit of a concern for Model 4, since it relies somewhat on accurate data. Sometimes live trading did not work great while it did on backtesting. It did not follow the backtesting model correctly. At the same time, we really needed speed. Fetching the data took so long that we had to increase the speed of fetching data. So in our previous development of Model 4, we increased the speed of live trading of fetching data from 1 minute 30 seconds to 20 seconds. But it appears that the data accuracy has declined.

We naively thought that accuracy and speed comes with trade-offs. But after working on Model 5 and digging deeper, we were wrong. Very wrong by a large mile. We can have both. It appeared that the accuracy of data was due to the way we implemented the fetching of the data quickly. It was hard to detect the problem as this cause was mixed with speed delays of other bugs. For our Model 4, we developed a brokerage API from scratch as it was not offered. But this custom API had poor designs that caused speed. So, accuracy and speed were independent. We could have both. We just needed to fix the bugs. Along with fixing fetching data and brokerage API, we fixed the architecture of live trading. This was a crucial bug. For our Model 5, we came up with new architecture that was built on top of Model 4 architecture. But a lot has changed. We made crucial mistakes in the time measurement system. We only realized this after blowing lots of cash on live testing. This is an embarrassing mistake, but thank god we fixed it.

You may ask, why we conducted live trading instead of paper trading. Well, even with paper trading, those bugs were hard to discover and we did need live testing at some point to get all details right. They are slightly different. We did, later, add paper trading feature that take account into delays.

There were many times we almost gave up because of this unknown bug. We even tried different types of strategies built on top of Model 5. We had Model 5-A and Model 5-B we attempted. We came back to the original Model 5 after having the same issue of accuracy. We thought this was due to speed. While it was but also other factors involved.

We did not give up because it was so obvious our backtesting indicated promising results and we could not be wrong with backtesting.

Lesson here is to never give up if numbers indicate you are right.

After fixing our architecture and brokerage API, we were able to achieve both accuracy and speed. There were many times, after fixing bugs, that we thought we cracked everything. But performance did not fall into false hopefulness and there were obviously bugs leftover. We had this cycle of hopefulness and frustration numerous times. In the end, we did not give up and we won the battle. These bugs were the final puzzle in the development of Model 5.

What we are grateful for is, assuming we had no choice with delays, desperately, we improved Model 5 a lot from the early version of Model 5.

After working tirelessly to address these issues, we are pleased to report that Model 5 has shown significant improvements in accuracy and speed.

Improved Performance

One of the key highlights of our research is the significant performance improvement. Through rigorous testing and optimization, we have achieved around a 50% return, 70% in peak, compared to early versions of Model 5’s 30% return in 3 months, and better stability than Model 4. Long term backtesting in early R&D with reduced precision of early versions also indicates steady performance compared to Model 4, showing better consistency.

Future Research and Development

To further improve the Model 5, we will focus on bid-ask spread, gathering large precise data, long term backtesting to accurately compare performance against Model 4 and to get more statistics.

Obstacles

Clearly, the major obstacle was to narrow the gap between backtesting and live trading. In recent testing, our data indicates live trading in large factor, accurately following the Model as expected. We already have and will continue improving on Model 5’s downsides,

Conclusion

In conclusion, our ongoing research and development efforts with Model 5 have yielded promising results. We are committed to pushing the boundaries of technology and look forward to sharing further advancements and enabling access to the Model 5 fund.