3 keys to algo crypto backtesting

A lot of articles on algorithmic crypto trading oversimplify the reality of building a trading rig and backtesting code. Here we’ll review 3 aspects of backtesting that are often glossed over but are key to achieving results.

Let’s get right to it, the 3 keys to backtesting:

• candle low values for stops
• managing execution prices
• iteratively comparing backtest to actuals

This means strong backtesting is a lot of effort (no surprise!) but let’s dig into this further.

Many backtesting rigs use candle close value for each tick. In live trading with stop-loss orders it is the low value in a tick that will stop your position.

Let’s look at an example. In the chart below for BTC let’s imagine we execute a BUY order at the blue line on the lower green candle, roughly midway from low to high for that tick.

The next tick’s low could trigger a stop if it was a tight stop-loss. But if your backtest code assumes a stop-loss SELL at the Close, then that is not the case and the position would extend forward in your simulation. Look closely at the red candle following the green entry tick (cross), its close and low values are very different. In this specific case the latter would show a larger profit.

Thus the impact of a BUY at a higher price than the open may have serious consequences to your simulation and backtesting. It is not the difference between your executed price and the tick open, instead it can be a big difference in the way the position extends and gains profit.

The execution price can have a cascading effect on a trade’s profit.

Consider 2 positions, one entered at 20.00 (to use round numbers), the second at 20.50, a slightly higher (market buy) price. This [relatively small] difference of 0.50 units can have a cascading effect on the position’s profit over time. Why? Because in a subsequent tick the 20.50 position might get stopped as a Low value could be lower than its stop-loss setting. Meanwhile the first position continues for many more ticks to a profit. Big difference!

To deal with this we need to estimate our executed prices…

Important to keep in mind that backtesting’s job is to estimate returns from a model/strategy when run in the past.

Your backtesting code does not know what price a trade received (past tense) on BUY or SELL side.

When looking into past historical market data, an actual order execution price is not knowable. Indeed you may have a log showing actual price of a position but the backtrader is looking at general market data not individual trade logs.

Using the open price for a tick on entry is not realistic, neither is using the close price. Why does this make a big difference? Because, as discussed in prior section, an entry at a higher execution price could easily cause a limit sell order to trigger sooner than in the backtest simulation.

You want your backtesting results to be slightly pessimistic.

There is no real answer to where to estimate execution prices should be precisely, it’s important to understand that you cannot get this exactly right in your simulations. Assuming an OPEN price for your BUY orders is generally optimistic and not the ideal approach. Similarly assuming a CLOSE price for your SELL orders is likely to deviate strongly from actual results.

BUY at market

Here are some options to choose from for BUY market orders:

a) estimate an execution price midway (or some predefined distance) between a tick’s OPEN and CLOSE

b) estimate an execution price midway (or some predefined distance) between a tick’s LOW and HIGH

c) chose a random price between OPEN or LOW and CLOSE or HIGH

While market action tends to be random in nature, I’m not a fan of (c) because your backtest code will not be idempotent; it will return different results each time it is run on the same data. This is messy.

Option (b) highlights a more volatile market, while option (a) less so. This is up to you but a choice must be made. You could, for example use a volatility index such as CVI and then choose according to that value in the period you are backtesting.

Profit taking SELL limit orders

Here are some options to choose from for profit taking SELL limit orders:

a) estimate an execution price midway (or some predefined distance) between a tick’s OPEN and HIGH

b) estimate an execution price midway between a tick’s CLOSE and HIGH

c) chose a random price between the above candle values

Stop-loss SELL limit orders

Here are some options to choose from for stop-loss SELL limit orders:

a) estimate an execution price midway between a tick’s OPEN and CLOSE

b) estimate an execution price midway between a tick’s CLOSE and LOW

c) chose a random price between the above candle values

Notice the use of LOW vs. HIGH values depending on the order’s stance. A profit taking limit sell order is dealing with a rising price while a stop-loss preventative limit sell order is dealing with a decreasing price. Both are likely to face gaps that jump over the limit values!

Dealing with gap-down past stop-loss limit orders

A very common way for your position losses to expand is to have a price gap-down past your stop-loss limit order. Your trading rig needs to manage this by identifying the position below its stop-loss limit order and executing a market sell order on that before it goes further into the red.

Remember that many trading exchanges to not support stop-loss market orders, eg. Binance, so you are forced to use stop-loss limit orders to protect your positions. A stop-loss limit order has a ‘stop price’ and a ‘limit price’, the former triggers a limit sell on the price of the latter.

Many exchanges such as Binance do not support stop-loss limit orders.

What does this mean for your backtest algorithm? To have a conservative/pessimistic view it should take a sell price somewhere between the stop’s limit price and the tick low. The lower the ratio the more conservative the result. It is the case that a significant percentage of these stop-loss limit orders will execute at their limit price, but others will execute below the limit price and this must be accounted for.

Accounting for profit taking sell orders

If your trading rig supports profit-taking models (it should), whereby a position is closed at a profit once reaching an exit level, these orders should be managed aggressively.

In this scenario a market order will leave profits on the table, better to make a limit sell order at a specific price.

A strong approach when looking to take profits on a position is to get the top of the order book (see above) and set the SELL limit price to one of these prices. This will likely fill at a higher price (profit) than a market order at this time.

What does this mean for your backtest algorithm? To have a conservative/pessimistic view it should assume a price between the profit-taking exit level and tick high. An actual sell price would depend on order book spread at the time of a trade but of course there is no ‘order book’ in a backtest simulator so we need to estimate from the data available.

There’s no way to improve backtest algorithms unless they’re compared to actual results iteratively.

Unless you iteratively compare your backtest results to actual trading results you cannot be confident in your backtesting.

Here’s a concrete example of how to achieve this:

• run your live trading rig for a period of time, eg. 24hours
• log the results: each trade, each exit, each stop-loss
• after this period, run your backtest code on that same period
• compare the results of the backtest with your actual results!

Compare each trade in detail, what assumptions did your backtest code make that are not representative of actual live trading?

What are the differences?

• is your backtest optimistic? If so where? how? why?
• is your backtest overly pessimistic?

You want your backtest to consistently show a slightly pessimistic result; conservative.

After doing this a few times, turn your trading rig off and evolve both the backtest code and your trading rig, then iterate.

Because your live trading rig can run with relatively small order size, it doesn’t cost much to run it as a test.

Of course during any period of time there will be unexpected action in the market, the point here is to identify areas where your backtest is clearly not estimating what your actuals revealed.

Some examples of what I’ve found when doing this:

• the backtest was overly optimistic about LIMIT SELL prices, in actual trading there were often gap-down situations that cut profits
• the backtest was overly optimistic about BUY execution prices, in actual trading it was rare to get an tick open price executed
• the backtest was not capturing STOP-LOSS sell orders on low values within a tick, in actual trading this was ending positions early

This is the only way to earn profits from your crypto trading rig. Evolve your backtesting over time and iterate over actual results to improve it.

Backtesting your algorithm trading model is hard work! This should come as no surprise. There is no ‘easy path’ here.

Source: https://medium.com/@gk_/3-keys-to-algo-crypto-backtesting-99f006cfd390?source=rss——-8—————–cryptocurrency