WEBVTT

1
00:00:00.040 --> 00:00:00.240
Hello.

2
00:00:00.521 --> 00:00:02.643
Welcome back to Papers with Backtest podcast.

3
00:00:02.723 --> 00:00:05.867
Today we dive into another algo trading research paper.

4
00:00:05.965 --> 00:00:06.145
Yep.

5
00:00:06.387 --> 00:00:09.172
Another journey into the world of systematic trading.

6
00:00:09.492 --> 00:00:10.609
Algorithmic trading research,

7
00:00:10.633 --> 00:00:15.531
it's really all about trying to find those systematic edges in the market,

8
00:00:15.875 --> 00:00:18.141
moving beyond just gut feelings,

9
00:00:18.203 --> 00:00:18.344
right?

10
00:00:18.345 --> 00:00:18.844
Precisely.

11
00:00:19.422 --> 00:00:19.672
I mean,

12
00:00:19.673 --> 00:00:20.484
in today's markets,

13
00:00:20.500 --> 00:00:22.344
you really need that systematic analysis.

14
00:00:22.875 --> 00:00:24.109
Looking at historical data,

15
00:00:24.641 --> 00:00:27.594
that's key to finding trading rules that might actually work.

16
00:00:27.904 --> 00:00:29.386
And that's exactly our mission today.

17
00:00:29.486 --> 00:00:33.128
We're diving deep into the trading rules and maybe more importantly,

18
00:00:33.449 --> 00:00:36.156
the backtest results from the book Algorithmic Trading,

19
00:00:36.554 --> 00:00:39.093
Winning Strategies and Their Rationale.

20
00:00:39.094 --> 00:00:39.218
Yeah,

21
00:00:39.234 --> 00:00:41.195
the aim is to pull out those practical insights.

22
00:00:41.196 --> 00:00:41.460
You know,

23
00:00:41.820 --> 00:00:43.742
how do you actually build and check these strategies?

24
00:00:44.226 --> 00:00:46.585
We're leaning on the author's real world experience here.

25
00:00:46.679 --> 00:00:48.429
The book covers a lot of ground stocks,

26
00:00:48.648 --> 00:00:49.304
ETFs,

27
00:00:49.523 --> 00:00:50.226
futures,

28
00:00:50.367 --> 00:00:51.163
currencies.

29
00:00:51.367 --> 00:00:51.523
Uh-huh.

30
00:00:51.773 --> 00:00:52.882
And different styles too,

31
00:00:52.992 --> 00:00:54.195
like mean reversion,

32
00:00:54.288 --> 00:00:54.710
momentum.

33
00:00:55.084 --> 00:00:58.547
But we'll keep our focus tight on the actual rules they used and what the backtest showed.

34
00:00:58.809 --> 00:00:59.207
Definitely.

35
00:00:59.309 --> 00:01:05.856
And it's worth remembering the author managed hedge funds through some pretty major market events like the 08 crisis.

36
00:01:06.113 --> 00:01:06.293
Right.

37
00:01:06.332 --> 00:01:07.613
That experience adds weight.

38
00:01:07.660 --> 00:01:09.582
It tells you something about the real challenges,

39
00:01:10.520 --> 00:01:12.520
the bumps in the road you hit with algo trading.

40
00:01:13.098 --> 00:01:13.223
OK,

41
00:01:13.363 --> 00:01:14.223
let's kick things off.

42
00:01:14.582 --> 00:01:17.770
The book hammers home the importance of backtesting,

43
00:01:18.410 --> 00:01:19.660
foundational stuff.

44
00:01:19.895 --> 00:01:20.316
Absolutely.

45
00:01:20.332 --> 00:01:20.441
Now,

46
00:01:20.504 --> 00:01:22.363
just for anyone maybe newer to this,

47
00:01:22.613 --> 00:01:24.379
backtesting is basically a...

48
00:01:24.952 --> 00:01:28.076
Taking your strategy and running it on old market data.

49
00:01:28.376 --> 00:01:28.537
Yeah.

50
00:01:28.796 --> 00:01:30.037
To see how it would have done.

51
00:01:30.279 --> 00:01:31.177
That's the gist of it.

52
00:01:31.298 --> 00:01:31.439
Yeah.

53
00:01:31.497 --> 00:01:32.220
The hope is,

54
00:01:32.400 --> 00:01:32.681
you know,

55
00:01:32.841 --> 00:01:33.798
if it worked in the past,

56
00:01:33.861 --> 00:01:35.181
maybe it's got a shot in the future.

57
00:01:35.923 --> 00:01:37.126
But this is crucial.

58
00:01:37.462 --> 00:01:41.267
The book really stresses doing your own independent backtesting,

59
00:01:41.829 --> 00:01:43.751
even for strategies that are already published.

60
00:01:43.752 --> 00:01:43.892
Okay.

61
00:01:44.017 --> 00:01:44.329
Why?

62
00:01:44.595 --> 00:01:47.079
Why retest something if someone's already shared the results?

63
00:01:47.298 --> 00:01:48.095
Seems like extra work.

64
00:01:48.329 --> 00:01:48.501
Well,

65
00:01:48.720 --> 00:01:49.533
several big reasons.

66
00:01:49.861 --> 00:01:50.251
First off,

67
00:01:50.644 --> 00:01:55.508
Strategies can be incredibly sensitive to the real nitty-gritty details of how you implement them.

68
00:01:55.911 --> 00:01:56.891
Nitty-gritty like what?

69
00:01:57.172 --> 00:01:57.290
Oh,

70
00:01:57.493 --> 00:01:59.251
things you might not even think about initially.

71
00:01:59.571 --> 00:01:59.751
Like,

72
00:01:59.993 --> 00:02:01.696
what exact order type do you use?

73
00:02:02.055 --> 00:02:02.837
Market on open.

74
00:02:03.219 --> 00:02:05.555
Or maybe a market order a few seconds after the open.

75
00:02:05.977 --> 00:02:06.243
Ah,

76
00:02:06.438 --> 00:02:06.696
I see.

77
00:02:06.743 --> 00:02:07.899
Those could give different prices.

78
00:02:08.258 --> 00:02:08.727
Big time.

79
00:02:09.508 --> 00:02:09.665
Or,

80
00:02:10.008 --> 00:02:10.680
for futures,

81
00:02:11.212 --> 00:02:13.227
what closing time are you using in your test?

82
00:02:13.602 --> 00:02:16.587
Does it match the stock market close or the specific futures close?

83
00:02:16.946 --> 00:02:17.493
It matters.

84
00:02:18.133 --> 00:02:19.087
And maybe the biggest one?

85
00:02:19.258 --> 00:02:19.790
What price?

86
00:02:19.944 --> 00:02:20.724
triggers your trade.

87
00:02:21.045 --> 00:02:21.465
The bid.

88
00:02:21.945 --> 00:02:22.445
The ask.

89
00:02:23.304 --> 00:02:24.084
The last trade.

90
00:02:24.443 --> 00:02:24.787
Wow,

91
00:02:24.947 --> 00:02:25.205
okay.

92
00:02:25.724 --> 00:02:27.744
Those details probably aren't always spelled out in papers,

93
00:02:27.787 --> 00:02:28.029
are they?

94
00:02:28.263 --> 00:02:29.068
Often not fully,

95
00:02:29.849 --> 00:02:31.787
but they're absolutely vital when you go live.

96
00:02:32.045 --> 00:02:33.810
They can totally change your profitability.

97
00:02:34.209 --> 00:02:38.591
So doing our own backtest lets us nail down those specifics for our setup.

98
00:02:38.873 --> 00:02:39.326
Exactly.

99
00:02:39.795 --> 00:02:40.529
And ideally,

100
00:02:40.795 --> 00:02:41.732
like the book points out,

101
00:02:41.982 --> 00:02:45.107
your backtesting setup should be close to your live execution system.

102
00:02:45.482 --> 00:02:47.154
You want to test what you'll actually trade.

103
00:02:47.545 --> 00:02:48.091
Makes sense.

104
00:02:48.652 --> 00:02:52.515
What else does independent testing uncover besides implementation details?

105
00:02:52.635 --> 00:02:55.119
It lets you really scrutinize things for hidden flaws,

106
00:02:55.279 --> 00:02:57.181
like biases in the data you're using,

107
00:02:57.463 --> 00:02:59.861
or maybe flaws in the strategy logic itself.

108
00:03:00.306 --> 00:03:01.603
And importantly,

109
00:03:02.244 --> 00:03:04.306
you can do true out-of-sample testing.

110
00:03:05.150 --> 00:03:07.791
Use data that came out after the strategy was published.

111
00:03:08.166 --> 00:03:10.088
If it suddenly falls apart on that new data,

112
00:03:10.791 --> 00:03:10.931
well,

113
00:03:10.994 --> 00:03:11.791
that's a big red flag.

114
00:03:11.869 --> 00:03:12.088
Right.

115
00:03:12.197 --> 00:03:14.994
Maybe just worked on that specific historical period by luck.

116
00:03:15.556 --> 00:03:16.041
Precisely.

117
00:03:16.181 --> 00:03:16.385
Okay,

118
00:03:16.463 --> 00:03:16.572
so.

119
00:03:17.128 --> 00:03:18.389
Backtesting is non-negotiable.

120
00:03:18.509 --> 00:03:19.871
Let's talk pitfalls.

121
00:03:20.271 --> 00:03:21.332
The book mentions several.

122
00:03:22.293 --> 00:03:25.215
Data snooping bias sounds ominous.

123
00:03:25.277 --> 00:03:26.175
It is in a way.

124
00:03:26.394 --> 00:03:27.379
It's also called overfitting.

125
00:03:27.535 --> 00:03:29.558
Basically it happens when your model is too flexible,

126
00:03:29.582 --> 00:03:30.683
too many knobs to tweak.

127
00:03:31.082 --> 00:03:31.222
Yeah.

128
00:03:31.238 --> 00:03:34.285
You fiddle with it until it looks amazing on your historical data.

129
00:03:34.722 --> 00:03:35.910
But you might just be fitting noise,

130
00:03:36.019 --> 00:03:37.254
random patterns that won't repeat.

131
00:03:37.847 --> 00:03:39.191
Then it performs poorly live.

132
00:03:39.222 --> 00:03:42.519
Like finding shilps and clouds and expecting them to predict the weather.

133
00:03:42.754 --> 00:03:43.582
That's a good analogy,

134
00:03:43.754 --> 00:03:43.941
yeah.

135
00:03:44.472 --> 00:03:45.441
And a common trap,

136
00:03:45.519 --> 00:03:46.160
the book notes,

137
00:03:46.620 --> 00:03:48.962
is trying to validate on out-of-sample data,

138
00:03:49.144 --> 00:03:51.546
but then tweaking the model again based on that data.

139
00:03:51.827 --> 00:03:52.046
Ah,

140
00:03:52.444 --> 00:03:54.046
so you contaminate your test set.

141
00:03:54.128 --> 00:03:54.347
You do.

142
00:03:54.749 --> 00:03:57.913
You've basically turned your out-of-sample data into more in-sample data,

143
00:03:58.132 --> 00:03:59.296
just reinforcing the bias.

144
00:03:59.671 --> 00:04:00.155
Sneaky.

145
00:04:00.374 --> 00:04:00.554
Okay,

146
00:04:00.757 --> 00:04:01.155
another one.

147
00:04:01.898 --> 00:04:03.163
Survivorship bias,

148
00:04:04.179 --> 00:04:05.413
especially with stocks.

149
00:04:05.741 --> 00:04:06.476
What's that about?

150
00:04:06.835 --> 00:04:07.054
Right.

151
00:04:07.398 --> 00:04:11.491
So many stock databases only include companies still trading today.

152
00:04:12.179 --> 00:04:14.554
They leave out the ones that went bust or got bought out.

153
00:04:14.612 --> 00:04:15.293
The losers,

154
00:04:15.373 --> 00:04:15.893
basically.

155
00:04:15.894 --> 00:04:16.934
The ones that didn't survive,

156
00:04:16.955 --> 00:04:17.135
yeah.

157
00:04:17.215 --> 00:04:17.776
Yeah.

158
00:04:17.975 --> 00:04:20.498
This can make backtests look way better than reality,

159
00:04:20.819 --> 00:04:21.440
especially for,

160
00:04:21.580 --> 00:04:21.779
say,

161
00:04:22.358 --> 00:04:24.178
long-only mean reversion strategies.

162
00:04:24.381 --> 00:04:25.420
Why those specifically?

163
00:04:25.764 --> 00:04:28.225
Because those strategies often buy stocks that have dropped,

164
00:04:28.709 --> 00:04:29.764
hoping they'll bounce back.

165
00:04:30.490 --> 00:04:33.264
If your data only shows the stocks that did bounce back,

166
00:04:33.295 --> 00:04:34.108
the survivors,

167
00:04:34.654 --> 00:04:35.670
your results look great.

168
00:04:35.889 --> 00:04:38.592
But you wouldn't know in real time which ones would survive.

169
00:04:38.733 --> 00:04:39.201
Exactly.

170
00:04:39.295 --> 00:04:41.545
You'd buy some that just keep falling and get delisted.

171
00:04:42.076 --> 00:04:42.639
Interestingly,

172
00:04:42.873 --> 00:04:44.217
for short-selling strategies,

173
00:04:44.652 --> 00:04:50.701
It can sometimes make results look worse because the failing stocks you would have profited from shorting are missing.

174
00:04:51.666 --> 00:04:54.588
So it cuts both ways but usually inflates long strategies.

175
00:04:54.689 --> 00:04:54.947
Often,

176
00:04:55.072 --> 00:04:55.314
yes.

177
00:04:55.892 --> 00:04:58.095
The bias on the long side tends to be stronger.

178
00:04:58.761 --> 00:05:02.084
It's a reminder that you might only be seeing the winners in historical data.

179
00:05:02.204 --> 00:05:02.844
Really important.

180
00:05:03.047 --> 00:05:04.766
Any other data gremlins to watch out for?

181
00:05:05.149 --> 00:05:07.649
Definitely need to adjust for stock splits and dividends.

182
00:05:08.032 --> 00:05:08.508
If you don't,

183
00:05:08.813 --> 00:05:11.891
a price drop on the ex-dividend date looks like a crash,

184
00:05:12.196 --> 00:05:13.297
triggering fall signals.

185
00:05:13.438 --> 00:05:13.657
Right.

186
00:05:13.938 --> 00:05:14.539
Simple error,

187
00:05:14.750 --> 00:05:15.375
big impact.

188
00:05:15.875 --> 00:05:17.860
And for things like pair trading across markets,

189
00:05:18.157 --> 00:05:20.063
make sure your prices are simultaneous,

190
00:05:20.547 --> 00:05:21.579
or as close as possible.

191
00:05:21.922 --> 00:05:23.641
You need prices from the same point in time.

192
00:05:24.079 --> 00:05:26.125
What about short selling and back tests?

193
00:05:26.282 --> 00:05:27.672
Any special issues there?

194
00:05:28.457 --> 00:05:28.617
Yeah,

195
00:05:28.757 --> 00:05:29.818
shorting isn't always easy.

196
00:05:30.579 --> 00:05:31.720
Stocks can be hard to borrow.

197
00:05:32.120 --> 00:05:35.966
There might be rules like the old uptick rule restricting when you can short.

198
00:05:36.286 --> 00:05:38.743
If your backtest assumes you can always short anything instantly.

199
00:05:39.149 --> 00:05:40.110
It's unrealistic.

200
00:05:40.267 --> 00:05:40.509
Very.

201
00:05:40.626 --> 00:05:43.173
It can seriously overstate profits for short strategies.

202
00:05:43.392 --> 00:05:43.634
Okay,

203
00:05:43.751 --> 00:05:44.806
moving beyond stocks.

204
00:05:45.540 --> 00:05:46.134
Currencies,

205
00:05:46.915 --> 00:05:47.509
futures,

206
00:05:47.993 --> 00:05:49.149
different data headaches.

207
00:05:49.415 --> 00:05:50.056
For currencies,

208
00:05:50.103 --> 00:05:52.978
the price you get can really depend on your broker or platform.

209
00:05:53.478 --> 00:05:53.962
Spreads vary.

210
00:05:53.978 --> 00:05:57.899
So using actual bid-ask data in your backtest is much more realistic.

211
00:05:58.033 --> 00:05:58.353
Got it.

212
00:05:58.654 --> 00:05:59.374
And futures.

213
00:05:59.615 --> 00:06:01.957
Constructing those continuous contracts is tricky.

214
00:06:02.739 --> 00:06:05.661
Stitching together expiring contracts needs care.

215
00:06:06.418 --> 00:06:12.770
How you back adjust for price level or for returns can create artificial jumps or mess up P&L,

216
00:06:13.371 --> 00:06:16.004
especially if you're trading spreads between contracts.

217
00:06:16.598 --> 00:06:17.817
You got to choose the right method.

218
00:06:17.989 --> 00:06:18.192
Man,

219
00:06:18.207 --> 00:06:20.254
it really is all about the details and backtesting.

220
00:06:20.379 --> 00:06:20.520
Yeah.

221
00:06:20.832 --> 00:06:21.036
Okay.

222
00:06:21.176 --> 00:06:24.114
Let's shift gears to some strategy examples from the book.

223
00:06:24.661 --> 00:06:25.067
First,

224
00:06:25.145 --> 00:06:27.582
a linear factor model for ranking stocks.

225
00:06:27.933 --> 00:06:28.814
What's the core idea?

226
00:06:29.014 --> 00:06:29.135
OK.

227
00:06:29.295 --> 00:06:31.218
So this uses statistical factors,

228
00:06:31.356 --> 00:06:33.557
things derived from fundamentals or market data.

229
00:06:34.260 --> 00:06:35.081
You normalize them,

230
00:06:35.202 --> 00:06:40.042
put them on a common scale and then see how they historically related to stock returns.

231
00:06:40.128 --> 00:06:41.167
To rank stocks.

232
00:06:41.268 --> 00:06:41.448
Yeah,

233
00:06:41.449 --> 00:06:41.948
to rank them.

234
00:06:42.284 --> 00:06:44.885
The goal isn't necessarily to predict exact returns,

235
00:06:44.948 --> 00:06:46.151
but more relative returns.

236
00:06:46.526 --> 00:06:50.229
Which stocks are likely to do better or worse than others based on these factors?

237
00:06:50.620 --> 00:06:51.135
Then you build,

238
00:06:51.198 --> 00:06:51.370
say,

239
00:06:51.557 --> 00:06:52.745
a long-short portfolio.

240
00:06:52.969 --> 00:06:56.353
And the book mentions Joel Greenblatt's magic formula as an example.

241
00:06:56.392 --> 00:06:56.533
Right.

242
00:06:56.534 --> 00:06:58.416
A classic example of a simple factor approach.

243
00:06:58.814 --> 00:07:00.635
It ranks stocks using just two things.

244
00:07:01.096 --> 00:07:01.916
Return on capital,

245
00:07:02.119 --> 00:07:03.721
how well the company uses its money.

246
00:07:03.900 --> 00:07:04.424
Efficiency.

247
00:07:04.760 --> 00:07:05.838
And earnings yield,

248
00:07:05.900 --> 00:07:06.385
basically,

249
00:07:06.603 --> 00:07:08.822
how cheap the stock is relative to its earnings.

250
00:07:08.885 --> 00:07:09.807
Value and quality.

251
00:07:09.885 --> 00:07:10.291
Exactly.

252
00:07:10.650 --> 00:07:11.228
Rank on both,

253
00:07:11.603 --> 00:07:12.322
combine the ranks.

254
00:07:12.713 --> 00:07:13.182
By the top,

255
00:07:13.291 --> 00:07:13.494
say,

256
00:07:13.932 --> 00:07:15.775
30 stocks hold for a year.

257
00:07:16.713 --> 00:07:19.291
The book quotes a pretty impressive back-tested return.

258
00:07:19.777 --> 00:07:22.439
like 30.8 percent annually from 88 to 04,

259
00:07:22.900 --> 00:07:24.623
much better than the S&P 500 then.

260
00:07:25.084 --> 00:07:27.443
Shows the power of simple rational factors.

261
00:07:27.564 --> 00:07:28.049
Impressive.

262
00:07:28.103 --> 00:07:28.283
Okay,

263
00:07:28.385 --> 00:07:29.283
mean reversion.

264
00:07:29.689 --> 00:07:32.470
The book has a conceptual linear strategy.

265
00:07:32.853 --> 00:07:32.970
Yeah,

266
00:07:32.994 --> 00:07:34.135
this one's more illustrative.

267
00:07:34.431 --> 00:07:35.650
The core idea is simple.

268
00:07:35.994 --> 00:07:38.853
Prices stray from their average but tend to snap back.

269
00:07:39.041 --> 00:07:39.838
Revert to the mean.

270
00:07:40.119 --> 00:07:40.322
Right.

271
00:07:40.416 --> 00:07:44.447
So this strategy trades when a price deviates significantly from its moving average.

272
00:07:44.947 --> 00:07:46.369
The size of the trade is scaled.

273
00:07:46.760 --> 00:07:47.385
You trade more,

274
00:07:47.386 --> 00:07:48.369
the further away it gets,

275
00:07:48.745 --> 00:07:51.326
often using standard deviation to measure that deviation.

276
00:07:51.365 --> 00:07:52.545
And the look-back period.

277
00:07:52.887 --> 00:07:55.605
Often tied to the estimated half-life of mean reversion,

278
00:07:55.688 --> 00:07:57.969
how long it typically takes to revert halfway back.

279
00:07:58.523 --> 00:08:00.109
It's presented as a fairly simple model,

280
00:08:00.484 --> 00:08:03.086
less prone to overfitting because it doesn't have tons of parameters.

281
00:08:03.266 --> 00:08:06.125
Then there's a Bollinger Band example on the GLD-USO spread,

282
00:08:06.344 --> 00:08:07.812
gold versus oil ETFs.

283
00:08:08.125 --> 00:08:08.328
Uh-huh.

284
00:08:08.625 --> 00:08:11.469
Bollinger Bands create those dynamic channels around a moving average.

285
00:08:11.906 --> 00:08:12.172
Here,

286
00:08:12.453 --> 00:08:15.312
they applied it to the price difference between GLD and USO.

287
00:08:15.516 --> 00:08:16.266
The spread.

288
00:08:16.516 --> 00:08:16.625
Yeah.

289
00:08:17.153 --> 00:08:18.875
And they used a dynamic hedge ratio,

290
00:08:19.415 --> 00:08:20.897
adjusting the shares over time.

291
00:08:21.538 --> 00:08:25.198
Entry was when the spread hit a Z-score of one standard deviation away.

292
00:08:25.925 --> 00:08:27.284
Exit was when it came back to zero,

293
00:08:27.542 --> 00:08:28.104
the average.

294
00:08:28.167 --> 00:08:28.885
And did it work,

295
00:08:29.003 --> 00:08:30.370
even though gold and oil aren't,

296
00:08:30.643 --> 00:08:30.823
like,

297
00:08:31.128 --> 00:08:32.385
perfectly linked long-term?

298
00:08:32.604 --> 00:08:33.010
Apparently,

299
00:08:33.065 --> 00:08:33.307
yes.

300
00:08:33.635 --> 00:08:36.745
The book reported positive returns and sharp ratio in the backtest,

301
00:08:37.276 --> 00:08:38.714
even without strong coin aggression.

302
00:08:39.229 --> 00:08:42.995
Suggests you can sometimes find short-term mean reversion in spreads,

303
00:08:43.464 --> 00:08:45.276
even if the assets wander apart eventually.

304
00:08:45.737 --> 00:08:46.277
Interesting.

305
00:08:46.337 --> 00:08:46.537
Okay,

306
00:08:46.638 --> 00:08:47.117
momentum.

307
00:08:47.317 --> 00:08:49.457
Example using TU Futures Treasury notes.

308
00:08:49.597 --> 00:08:49.879
Yeah,

309
00:08:49.880 --> 00:08:51.258
a time series momentum strategy.

310
00:08:51.637 --> 00:08:52.398
Super simple rule.

311
00:08:52.699 --> 00:08:54.515
If the price went up over the last 12 months,

312
00:08:54.898 --> 00:08:55.140
buy.

313
00:08:55.461 --> 00:08:56.000
If it went down,

314
00:08:56.101 --> 00:08:56.617
sell short.

315
00:08:56.797 --> 00:08:57.562
Hold for one month.

316
00:08:57.656 --> 00:08:58.344
And the logic?

317
00:08:58.859 --> 00:09:02.523
Based on finding a statistically significant positive link between the

318
00:09:02.867 --> 00:09:06.226
12-month return and the next month's return for those futures historically.

319
00:09:06.789 --> 00:09:09.726
They also mentioned a tweak involving daily incremental investments.

320
00:09:10.179 --> 00:09:10.570
Simple,

321
00:09:10.976 --> 00:09:11.758
but data-driven.

322
00:09:11.883 --> 00:09:13.804
What about cross-sectional momentum?

323
00:09:14.065 --> 00:09:15.206
commodities example.

324
00:09:15.286 --> 00:09:15.427
Right.

325
00:09:15.446 --> 00:09:16.968
This looked across a bunch of commodities,

326
00:09:16.987 --> 00:09:17.849
like 52 of them.

327
00:09:18.310 --> 00:09:18.751
Each month,

328
00:09:19.071 --> 00:09:20.591
rank them by their 12-month return.

329
00:09:20.751 --> 00:09:21.013
Okay.

330
00:09:21.032 --> 00:09:22.591
Then by the top performer's future,

331
00:09:22.872 --> 00:09:24.474
short the bottom performer's future,

332
00:09:25.255 --> 00:09:25.919
hold for a month,

333
00:09:26.177 --> 00:09:26.661
re-rank,

334
00:09:27.021 --> 00:09:27.458
repeat.

335
00:09:27.919 --> 00:09:28.622
And the results?

336
00:09:28.700 --> 00:09:31.958
Excellent backtest results reported for mid-2005 to late-2007.

337
00:09:33.177 --> 00:09:33.911
High APR,

338
00:09:34.302 --> 00:09:35.286
good sharp ratio.

339
00:09:35.661 --> 00:09:35.786
Yeah.

340
00:09:35.942 --> 00:09:36.130
But,

341
00:09:36.161 --> 00:09:36.942
and this is a huge,

342
00:09:37.067 --> 00:09:37.849
but the book notes,

343
00:09:37.989 --> 00:09:39.911
it got hammered during the 2008 crisis.

344
00:09:40.255 --> 00:09:40.411
Ah,

345
00:09:40.989 --> 00:09:41.974
a momentum crash.

346
00:09:42.067 --> 00:09:42.599
Exactly.

347
00:09:42.925 --> 00:09:47.838
A stark reminder that what works in one market regime can fail spectacularly in another.

348
00:09:48.279 --> 00:09:50.142
Fact tests don't always capture that fragility.

349
00:09:50.224 --> 00:09:50.763
Very true.

350
00:09:51.296 --> 00:09:51.656
Lastly,

351
00:09:51.857 --> 00:09:53.018
news sentiment strategies.

352
00:09:53.218 --> 00:09:53.598
Briefly,

353
00:09:53.838 --> 00:09:54.360
what's the idea?

354
00:09:54.699 --> 00:09:56.119
This uses tech to read news,

355
00:09:56.221 --> 00:09:56.740
social media,

356
00:09:56.803 --> 00:09:56.920
etc.,

357
00:09:57.701 --> 00:09:58.944
and assign sentiment scores,

358
00:09:59.022 --> 00:09:59.983
positive or negative,

359
00:10:00.045 --> 00:10:01.069
to companies or assets.

360
00:10:01.365 --> 00:10:02.569
So quantifies the buzz.

361
00:10:02.748 --> 00:10:03.029
Sort of,

362
00:10:03.030 --> 00:10:03.186
yeah.

363
00:10:03.686 --> 00:10:04.631
Then you build strategies,

364
00:10:04.772 --> 00:10:06.834
like buying stocks with improving positive sentiment,

365
00:10:07.092 --> 00:10:08.756
shorting ones with rising negative sentiment.

366
00:10:09.100 --> 00:10:11.819
The book mentions reports of high returns in sharps and back tests.

367
00:10:11.944 --> 00:10:12.115
Okay.

368
00:10:12.240 --> 00:10:13.350
But often before costs.

369
00:10:13.537 --> 00:10:15.475
Costs of trading and the data feed itself,

370
00:10:15.553 --> 00:10:16.037
presumably.

371
00:10:16.319 --> 00:10:16.522
Right.

372
00:10:16.787 --> 00:10:18.006
Those can eat into profits quickly.

373
00:10:18.256 --> 00:10:20.350
These examples really show the variety.

374
00:10:20.804 --> 00:10:26.044
but also highlight that you have to look closely at the rules and the backtest context critically.

375
00:10:26.345 --> 00:10:26.787
Absolutely.

376
00:10:27.126 --> 00:10:28.087
And like the book stresses,

377
00:10:28.665 --> 00:10:30.044
even if a backtest looks good,

378
00:10:30.810 --> 00:10:34.263
is it statistically significant or just luck?

379
00:10:34.607 --> 00:10:35.505
How do you test for that?

380
00:10:35.669 --> 00:10:37.333
The book touches on a few ways.

381
00:10:37.880 --> 00:10:40.333
Comparing your Sharpe ratio to certain thresholds,

382
00:10:41.130 --> 00:10:42.693
using Monte Carlo simulations,

383
00:10:43.005 --> 00:10:47.990
basically generating random price paths to see how often sheer luck produces similar results.

384
00:10:48.083 --> 00:10:48.333
Okay.

385
00:10:48.632 --> 00:10:54.177
or even just randomizing your actual trades across history to see if timing luck was the key factor.

386
00:10:54.560 --> 00:10:55.220
Different methods,

387
00:10:55.259 --> 00:10:56.677
different assumptions about randomness.

388
00:10:56.904 --> 00:10:58.021
So they might give different answers.

389
00:10:58.083 --> 00:10:58.700
It's complex.

390
00:10:58.865 --> 00:11:00.966
The book also mentions market regime shifts,

391
00:11:01.083 --> 00:11:02.146
structural changes.

392
00:11:02.544 --> 00:11:03.685
How did those mess things up?

393
00:11:03.950 --> 00:11:04.427
Big time.

394
00:11:04.849 --> 00:11:06.474
Think about decimalization in U.S.

395
00:11:06.536 --> 00:11:08.177
stocks back in 2001.

396
00:11:08.833 --> 00:11:09.786
Spreads got way tighter.

397
00:11:09.974 --> 00:11:11.349
Killed some Stadarb strategies,

398
00:11:11.396 --> 00:11:11.630
right?

399
00:11:11.911 --> 00:11:12.333
Exactly.

400
00:11:12.693 --> 00:11:16.193
Strategies relying on tiny bid-ask spreads suddenly struggled.

401
00:11:16.615 --> 00:11:18.099
Or the 2008 crisis.

402
00:11:18.352 --> 00:11:19.734
Massive volatility spikes,

403
00:11:19.853 --> 00:11:20.714
volume changes.

404
00:11:20.994 --> 00:11:25.498
That completely changed the game for many mean reversion and even momentum strategies.

405
00:11:25.560 --> 00:11:27.482
So the market itself changes the rules.

406
00:11:27.584 --> 00:11:28.123
Constantly.

407
00:11:28.443 --> 00:11:31.545
The author's own crisis survival stories highlight key lessons.

408
00:11:31.982 --> 00:11:33.966
Don't manually override your system in panic.

409
00:11:34.341 --> 00:11:35.388
Stay under leveraged.

410
00:11:35.763 --> 00:11:38.513
Accept that strategy performance waxes and wanes.

411
00:11:39.138 --> 00:11:40.513
And beware of overconfidence.

412
00:11:41.170 --> 00:11:41.888
It's dangerous.

413
00:11:41.935 --> 00:11:42.623
Humility is key.

414
00:11:43.216 --> 00:11:43.404
Okay,

415
00:11:43.513 --> 00:11:44.060
last point.

416
00:11:44.482 --> 00:11:45.623
Software and platforms.

417
00:11:45.795 --> 00:11:46.701
What does the book cover?

418
00:11:46.948 --> 00:11:48.130
It mentions a few categories.

419
00:11:48.269 --> 00:11:49.731
Scripting languages like Python,

420
00:11:49.770 --> 00:11:50.331
MATLAB,

421
00:11:50.731 --> 00:11:52.915
very flexible but steeper learning curve maybe.

422
00:11:53.075 --> 00:11:54.056
For custom builds.

423
00:11:54.157 --> 00:11:54.274
Yeah.

424
00:11:54.614 --> 00:11:55.696
Then Excel with VBA,

425
00:11:56.259 --> 00:11:58.641
easier entry for some but might choke on complex stuff.

426
00:11:59.157 --> 00:12:00.821
And specialized platforms like FXO,

427
00:12:00.985 --> 00:12:01.462
MetaTrader,

428
00:12:01.524 --> 00:12:04.462
TradeStation often have built-in backtesting and broker links.

429
00:12:04.712 --> 00:12:07.274
Trade-offs between flexibility and ease of use.

430
00:12:07.415 --> 00:12:07.806
Pretty much.

431
00:12:08.009 --> 00:12:10.665
And it touches on collocation for high-frequency trading,

432
00:12:10.712 --> 00:12:13.634
putting your servers right next to the exchange's servers to cut down latency.

433
00:12:14.134 --> 00:12:15.056
Milliseconds matter there.

434
00:12:15.352 --> 00:12:31.393
requires direct data feeds too speed is everything at that level okay this has been a really insightful look at the trading rules and back test realities from algorithmic trading really drives home the complexities it does and for anyone listening who's into this you've got to remember the challenges data

435
00:12:31.455 --> 00:12:44.783
quality those biases we talked about implementation details market changes it's a minefield if you're not careful and that crucial takeaway a great back test is not a crystal ball It needs rigorous out-of-sample validation.

436
00:12:45.172 --> 00:12:47.175
a real understanding of why it works,

437
00:12:47.233 --> 00:12:48.735
and a healthy dose of skepticism.

438
00:12:48.835 --> 00:12:49.475
Precisely.

439
00:12:49.737 --> 00:12:50.956
It's about continuous learning,

440
00:12:51.499 --> 00:12:52.217
critical thinking,

441
00:12:52.378 --> 00:12:52.839
and yeah,

442
00:12:52.960 --> 00:12:55.940
staying humble about the market's ability to surprise you.

443
00:12:56.081 --> 00:12:58.464
Thank you for tuning in to Papers with Backtests podcast.

444
00:12:58.706 --> 00:13:00.807
We hope today's episode gave you useful insights.

445
00:13:01.026 --> 00:13:03.034
Join us next time as we break down more research.

446
00:13:03.284 --> 00:13:04.753
And for more papers and backtests,

447
00:13:04.815 --> 00:13:08.018
find us at https.paperswithbacktests.com.

448
00:13:08.831 --> 00:13:09.409
Happy trading.