Almost every trading bot I am shown looks brilliant on the chart. A smooth rising line, few losses, a strategy that "just works". Then real money goes in, and within weeks the results look nothing like the test. In most cases the idea was not the problem. The backtest was. In this article I explain what a rule-based bot in Python is made of, what an honest backtest has to include, and the kind of checks a strategy should pass before it touches real money.
One thing first, so there is no misunderstanding: this is not financial advice, and no bot or backtest can promise profit. Markets change, and a strategy that worked in the past can lose money tomorrow. The goal of a good backtest is not to prove you will win. It is to find out, cheaply and early, that a strategy will probably lose. The strategy and all numbers below are illustrative.
What a trading bot really is
Strip away the marketing and a trading bot is four simple parts: data (prices, and maybe volume), rules (when to buy, when to sell), risk limits (how much to put in each trade and when to stop) and execution (sending orders to a broker or exchange). Python is a good fit because the same code can test the rules on history and later run them live. The hard part is not the code. It is being honest with yourself about what the test really shows.
Step 1: write rules a machine can follow
"Buy when the trend looks strong" is not a rule. A rule is exact enough that two people, or a computer, would make the same decision every time. A classic teaching example is a moving-average crossover: hold the asset when the short average is above the long one, stay out when it is below. It is not a good strategy by itself, and real strategies need much more than this, but it shows what "exact" means.
Step 2: make the backtest honest
Most beautiful backtests are wrong in one of two quiet ways. The first is look-ahead: the test uses today's closing price to decide a trade that, in real life, you could only place after the close. The second is free trading: no fees, no spread, no slippage. Both make results look far better than reality. The fix for the first is to let the bot act only on information it would really have had at that moment; the second needs a realistic cost on every trade.
In the illustrative test in the chart above, the same rules showed +34% over three years without costs. With a realistic cost per trade, that fell to +16%. Nothing about the idea changed; only the honesty of the test did.
| Illustrative test of the same strategy | Return | Worst drawdown |
|---|---|---|
| Backtest, no costs, same-bar trading | +34% | −12% |
| Backtest with fees and slippage, next-bar trading | +16% | −15% |
| Out-of-sample year the strategy never saw | −3% | −19% |
Step 3: the overfitting trap
Look at the last row of the table. This is where most bots die. If you try enough settings, different lengths, thresholds and stops, one of them will always look great on the past, purely by luck. That is overfitting: the strategy has learned the noise of one period, not a real pattern. The basic defence is to keep part of the history aside and never touch it while you tune. How you split it, and how you judge the result, is where most of the real work is. If the result falls apart there, the strategy was probably never real.
Step 4: paper trading before real money
A strategy that survives the backtest still has to survive the real world: delayed data, orders that fill at worse prices, a connection that drops at the wrong moment. Run the bot for a few weeks with a paper or demo account, and compare every paper trade with what the backtest says should have happened. The gap between the two is often the most useful number you will get.
Step 5: risk limits are part of the bot
Risk limits are not a setting you add later; they belong in the code from day one. At minimum: a maximum size for each position, a daily or weekly loss limit after which the bot stops, and a manual "kill switch". A bot without limits does not know when it is broken. It just keeps trading.
The check that decides whether I trust a strategy
Everything above is the basic hygiene, and it already removes most bad strategies. But there is one more check I always run before I would let any strategy near real money, and over the years it has rejected more "great" strategies than all the other tests together. It is not complicated, but how you set it up depends on the market, the timeframe and the data you have, and set up the wrong way it gives false comfort. That is why I prefer to go through it on a real strategy with the person who built it, rather than as a general recipe.
Common mistakes
- Testing without costs. Fees, spread and slippage belong in every backtest.
- Using data that has not arrived yet. In real life you act after the data arrives, not before.
- Tuning until it looks perfect. The more settings you try, the more likely the winner is luck.
- Skipping paper trading. A few weeks of demo trading costs nothing compared with a live surprise.
- No stop for the bot itself. Every bot needs loss limits and a way to switch it off.
The same discipline shows up in other work I have written about: a go/no-go feasibility study for any investment, cleaning data before you trust it, and practical Python on real business data.
Have a strategy or a bot you want to test before risking real money?
Send me a short description: which market and timeframe, what the rules are, and how you have tested it so far. I will tell you where the backtest may be fooling you, and we can go through the checks that matter for your strategy, including the one above.
Contact me Read: go/no-go feasibility study Read: Python on business data
Sources
- Look-ahead bias, transaction costs and overfitting are well-known pitfalls in backtesting trading strategies; holding out data that is not used for tuning, and paper trading before going live, are standard ways to reduce them.
- Past performance, in a backtest or in live trading, does not guarantee future results.
- The strategy and all numbers in this article are illustrative only. This article is not financial advice.