Beware the Backtest

Backtesting is a popular tool amongst traders and marketers. Traders use it to experiment and uncover new strategies. When one looks promising, the marketers go to work! Creating fancy charts and graphs to sell you on hypothetical performance. But are strong backtests indicative of strong future results?

The world of "Smart-Beta" is filled with promising backtests. Unlike plain-vanilla ETFs that simply track an index, Smart-Beta aims to outperform by adjusting their holdings - sometimes greatly deviating from their index.

A recent study called the Smart Beta Mirage analyzed the performance of these "smarter" funds from 2000 - 2018. Sadly, it found that outperformance only lived in the data.

Smart-beta in backtests showed an above-market returns of 2.77%, but when put into action, they underperformed by 0.44%.

There are several reasons for the disappointing results. The most obvious one, no one can predict the future. The oldest adage in investing is "Past performance is no guarantee of future results". Things change, what worked in the past might not work in the future.

Another reason is backtesting is flawed. It's susceptible to bias and data mining. ETF providers are incentivized to attract as many investors as possible. A great way to do this is to show great performance would've been, even if it sacrifices the integrity of testing.

While backtests are interesting and a great tool if used correctly, like star ratings and analyst targets, they do little in predicting the future. Don't count on them.

Comments

Today's Special: Humble Pie

You champion a project, fight for an idea, and then...reality sets in. That churning in your stomach isn't butterflies, it's the realization you've missed the mark. Pride will puff up your chest, and kick in the "defend at all costs" instinct. But arguing with the umpire never changed a call. Admitting you're wrong isn't a sign of weakness. It can strengthen your professional standing. In a world obsessed with the illusion of infallibility, the courage to adjust course is a breath of fresh air. It shows you're confident enough to be wrong, and adaptable enough to learn from it. Do your research, think critically, and stand behind your decisions. But when the data whispers (or screams) otherwise, don't be afraid to swallow that slice of humble pie. Be the first to acknowledge. Don't wait for someone to point out your mistake. Be open, take responsibility, and most importantly, focus on what you're going to do to address it. Don't dwell ...

When Perfect Becomes a Problem: The iCar Story

Let's talk about Apple's iCar, or rather, the ghost of it. A decade. Ten billion dollars. Poof. Gone. Like a puff of smoke from a dream that never quite woke up. They wanted to launch a revolution, a fully-formed, flawless chariot. But revolutions aren't born in secret labs; they're forged in the messy, chaotic crucible of the real world. You don't build a movement by hiding in the shadows. You don't create a product people love by ignoring them. You don't change the world by waiting for perfection. It's about the minimum viable. It's about shipping early, shipping often, and listening—really listening—to the people you're trying to serve. Apple built a cathedral of secrecy. A monument to what might have been. And then, they tore it down. They spent billions on a dream, while ignoring the simple truth: the market doesn't care about your dreams. It cares about solutions. It cares about things that work. So, here's the lesson: stop chasing...

Why We Shouldn't Be Afraid of Ambiguity

Ambiguity. That fuzzy monster that chases us down darkened hallways, whispering doubts about our roadmap and feature sets. You know the feeling. You constantly wrestle with unknowns: Will users like this? Is this the right direction? Frankly, if you had a nickel for every time the answer wasn't crystal clear, well, you might actually want to chase that ambiguity down the hall. But here's the thing: ambiguity isn't your enemy. It's your dance partner. Innovation rarely happens in a land of perfect clarity. Sure, there's a time for well-defined processes. But when you're creating something new, there are bound to be more questions than answers. The key is to learn to waltz with the unknown . Embrace the experiment. Don't be afraid to throw some spaghetti at the wall and see what sticks. Focus on outcomes, not outputs. Don't get hung up on features. What problem are you trying to solve? How will you measure success? Get comfortable with "go...

Herman Ye

Search