Beware the Backtest

Backtesting is a popular tool amongst traders and marketers. Traders use it to experiment and uncover new strategies. When one looks promising, the marketers go to work! Creating fancy charts and graphs to sell you on hypothetical performance. But are strong backtests indicative of strong future results?

The world of "Smart-Beta" is filled with promising backtests. Unlike plain-vanilla ETFs that simply track an index, Smart-Beta aims to outperform by adjusting their holdings - sometimes greatly deviating from their index.

A recent study called the Smart Beta Mirage analyzed the performance of these "smarter" funds from 2000 - 2018. Sadly, it found that outperformance only lived in the data.

Smart-beta in backtests showed an above-market returns of 2.77%, but when put into action, they underperformed by 0.44%.

There are several reasons for the disappointing results. The most obvious one, no one can predict the future. The oldest adage in investing is "Past performance is no guarantee of future results". Things change, what worked in the past might not work in the future.

Another reason is backtesting is flawed. It's susceptible to bias and data mining. ETF providers are incentivized to attract as many investors as possible. A great way to do this is to show great performance would've been, even if it sacrifices the integrity of testing.

While backtests are interesting and a great tool if used correctly, like star ratings and analyst targets, they do little in predicting the future. Don't count on them.

Comments

Today's Special: Humble Pie

You champion a project, fight for an idea, and then...reality sets in. That churning in your stomach isn't butterflies, it's the realization you've missed the mark. Pride will puff up your chest, and kick in the "defend at all costs" instinct. But arguing with the umpire never changed a call. Admitting you're wrong isn't a sign of weakness. It can strengthen your professional standing. In a world obsessed with the illusion of infallibility, the courage to adjust course is a breath of fresh air. It shows you're confident enough to be wrong, and adaptable enough to learn from it. Do your research, think critically, and stand behind your decisions. But when the data whispers (or screams) otherwise, don't be afraid to swallow that slice of humble pie. Be the first to acknowledge. Don't wait for someone to point out your mistake. Be open, take responsibility, and most importantly, focus on what you're going to do to address it. Don't dwell ...

The Art of Giving Feedback

Constructive feedback is an awkward affair. You don't want hurt feelings, but recognize the importance of honesty. You've tried the classic "hoping things will get better on its own" and unfortunately it hasn't played out. When giving feedback, here are a few things that I try to keep it mind. Start with empathy. Step into their shoes and understand their story. If you don't know, ask. Be genuinely curious. Feedback is a dynamic affair. Shared communication with a shared goal towards progress. Take the emotion out of it. Focus on the situation, not the person. Focusing on the person adds unnecessary weight to an already emotionally-bloated event. Be specific. Give clear examples. Vague feedback equals dismissed feedback. Doing above won't de-awkward things fully, but it will dampen it and increase the chance of better outcomes.

Negative Feedback, Positive Lessons

In the battle against plastic bags, a five-cent tax was shown to be much more successful at deterring usage than a five-cent credit for bringing your own bags. Carrots satisfy but sticks sting, and they sting hard. So we default to the less painful choice of avoiding loss. Loss aversion impacts the way we process information. A 2019 study invited participants to learn through a series of multiple choice questions. Each question only had two options to choose from. Whether guessing correctly or not, they would still learn the right answer. Despite the identical learning opportunity, participants were much more successful at recalling the answers they guessed correctly than those they got wrong. "You're right!" feels good. We savour the moment, analyzing every detail. "You're wrong!" stings. We want to quickly forget, dismiss, and move on. When we succumb to loss aversion, we miss opportunities to learn. Failure is part of the process. We'll experie...

Herman Ye

Search