One of the main points of the Collins piece is that overly simple models can understate risk and lull retirees into a false sense of security and that more sophisticated models can do a better job of evaluating risk in anticipation of strategy formulation and decision making (this is often said with a straight face by someone with a really expensive model or service to sell you). But, as he points out, that begs the question (not really satisfyingly answered) of how far do you go. Returns too normal? make them less so with other distributions (say: Beta, Extreme, Gamma, Laplace, Logistic, Lognormal, Pert, Rayleigh, Wakeby, and Weibull) or methods (two-state regimes, autoregression vectors, etc) . Inflation too static? Randomize it. Same for longevity and make sure it's a match to SOA data not the SS life table. Fees? Add them of course. Then maybe tier them. Taxes? Add, and then try your mighty best to get more accurate per individualized client. Spending? add rules, shocks, trends, jumps, regimes, etc. You get the idea.
But is it worth it? and how far do you go? Before I get to that (opinion), Collins points out two other things I hadn't thought about: 1) Bonini's paradox (that was new to me: more complete models get less understandable; the more accurate, the more it is as difficult to understand as the real-world process being modeled), and 2) How, exactly, do you rank heterogeneous outcomes amongst many sophisticated models? Good points.
My opinion on this is that that there are diminishing returns to the complexity and even if there weren't -- which there kind have to be, right? -- retirement is a game of whack-a-mole. Solve one problem and create another. Even Collins suggests something to that effect in discussing a paper by Athavale and Goebel (2011) where they used all those distribution types above to test the 4% rule. Their takeaway? "They conclude that a 4% withdrawal rate tested in non-normal distributions generally results in a lower sustainability rate when compared to test results using a simple Monte Carlo method that assumes distributional normality. " At the risk of being rude that might be a "Duh" on someone's lips other than mine. That's a point I've made before. Find more risk through more sophisticated analysis and what are you going to do when you can't make new money out of thin air? You are going to do things like spend less and allocate more carefully. And then if you don't superannuate you are still going to leave money (even more than before) on the table than you otherwise would have. So, problem not really solved since you have not succeeded in a god-like prediction of the future.
Let's look at it like this. In one of those two dimensional thingys my co-consultants loved when I consulted we might put sophistication (simple to complex, but maybe not as far as replicating reality) on the X axis. On the Y you could put anything (more in a sec); and note here this is not real, just my opinion "napkin:"
A-->B could be anything you like such as:
- risk assessment (most probably lower to higher, as Collins shows)
- can I understand it (yes to less so)
- are the assumptions transparent (yes to no)
- modeler biases present (probably low to high)
- usable by mortals (high to maybe)
- can I afford it (yes to probably no)
- processing and turnaround time? (low to likely high)
- can I self manage (yes to no)
- is it even accessible or is it locked-up IP (yes, accessible, to maybe not-so-much)
- and so on...
Let's pause at #1. If the result of the sophistication is just to find higher risk, it's as I mentioned above. There are only so many ways to cut the retirement pie. So, unless the cool model finds me new money out of thin air the rational-response levers in the face of high risk are still the same: spending, allocation, risk pooling, work longer...or again, and lotteries[1] (I'd argue that risk pooling actually does find new money out of thin air in a way). At least, I guess, it's good we found the increased risk, it's just that we haven't eliminated the risk of underspending and over-committing to legacy yet. By all means discover new or underappreciated risk but after that, what? If we recast the drawing above as one that looks at the usefulness of tools for marginal changes in complexity it might look like this where the gap between the line and the dotted line is maybe best called modeling risk:
I wish I could prove that but I can't. Mostly it's still just my opinion that a) the complexity of models is great and important and useful to the extent that it finds worse situations than we had anticipated and so that we can rationally react to it (do we care much about the better?) but that b) the trade-offs involved will suppress usefulness the farther we go, and c) it'll never predict the future so the unknowable trade-off between future and present will remain. Too bad no one know where that "too far" begins. [2]
On the other hand I will point out that this is the second "paper" or post I've read this month that has made the same interesting point about modeling and simulation. Both make the case that there probably is some usefulness to one area of modeling sophistication once we have gotten to a place where we have done as much as we can on things like fees, taxation, longevity, or spending. This is in the area of return modeling when it comes to autocorrelation: trends and reversion. Returns aren't just not normal, they also trend and mean revert which doesn't happen so much when returns are simplified to iid (independent and identically distributed) in an MC sim where there are also no dynamic changes in volatility. I don't know this kind of math/method very well so I'll just quickly point out that Collins makes the case that using a dynamic two state return regime-switching model has some lift when it comes to forecasting. Based on his own and the work of others that he lists in the paper he concludes that the two state model is a good way to approximate the range of future portfolio values and that "Unlike simple Monte Carlo simulations, return series produced in a regime switching engine exhibit all of the following empirical asset price behaviors: skewness, fat-tails, autocorrelation, volatility clustering, and dynamic correlations. " He goes on to analytically demonstrate the purported effectiveness at finding "reasonable" new risk vs. just more risk. But I also think he is selling a little bit, so who knows.
The other place I ran into this recently was at earlyretirementnow.com in The Ultimate Guide to Safe Withdrawal Rates – Part 20: More Thoughts on Equity Glidepaths where he discusses allocation in a glidepath context and comes up with different (better?) conclusions than those of a paper by Kitces and Pfau on the same topic. He gets different results because unlike MC simulation he uses a historical data approach that factors in short term mean reversion, long term mean reversion, and correlations. His point is that the difference can have profound and advantageous effects on decision making as retirement unfolds in different market scenarios and that in MC sims "you also lose all the interesting return dynamics that are due to equity valuations occasionally deviating and then returning to economic fundamentals." ERN, as I've mentioned before, like this blog as well, has "skin in the game" and is not selling. Or at least I know for sure that I am not selling. Yet.
Me? I used to think that MC sims, with enough iterations, will find the paths similar to mean reversion and trends but maybe I don't think that so much anymore since those paths would be dominated by the sheer number of paths that by design don't trend. So, the stuff from Collins and ERN (I know it's been done elsewhere too) above probably is an area where more complexity helps [Lee 2013 indirectly disagrees; see note 3]. I've thought about ways to add these kinds of things (e.g., two-state regime switches or ERN-style historical method) to my own modeling but for right now it is a step or two beyond me and I have to console myself with the idea that maybe the incremental utility coming from the additional work I'd have to do would be likely much smaller than the incremental utility that comes from the fact that I can simulate anything in the first place.
But what about the question of evaluating heterogeneous outcomes amongst many sophisticated models? That feels like open territory to me. Where does one even start? It also feels like a calibration issue. There is nothing to calibrate against except the future which isn't going to happen. This part of the topic is probably better left for another post but I will say that for my own planning purposes, in the absence of some "best" modeling calibration, I use a triangulation approach. That means I use many models and methods to try to come to a judgement about where I am and where I am going. I also re-ask the question pretty frequently and I tend to focus on the worst-case side of the output rather than the full dispersion of outcomes. This approach helps me diversify away some of the idiosyncratic modeling risk that may exist inside the assumptions and techniques of different models and modelers. It is also vaguely akin to the phenomenological approach used in my Religion B.A. (yeah, you read that right) where, though there is a more formal definition of the methodology that borders on incomprehensible or maybe better said as "you can't get there from here," in practice it often meant using multiple methods of inquiry to come to some type of understanding of any particular phenomenon being studied when there is no objective prior "law" or comparative "thing" or whatever to provide calibration and context. Same idea here. Few people can avoid liking the feeling of being confirmed in their own biases and I was confirmed here in mine by Collins. Or actually he used a reference to Nobel Laureate Thomas Sargent; even better! In one of his footnotes he writes: "Thomas J. Sargent’s Noble Prize winning research deals with how investors make decisions when they doubt the accuracy of their model. When confronted with ambiguity, they tend to use a family of models and to over-weight bad outcomes as a mechanism for exercising caution: see Hansen & Sargent (2008) for example." Confirmed. I like it. But I don't think I answered my reader's original question with any degree of precision or helpfulness.
Other Links -- Here is a short list of links I read or thought about in doing this. There are others but I can't find them right now.
How Risky is Your Retirement Income Risk Model? Collins et al. 2015 SSRN
Stress Testing Monte Carlo Assumptions, Marlena Lee 2013 SSRN
It’s Time to Retire Ruin (Probabilities), Milevsky 2016 CFApubs
The Power and Limitations of Monte Carlo Simulations, Blanchett and Pfau 2014 advisorperspectives.com
---------------------
[1] that is only partly tongue in cheek. Allocation optimization can show in some models and methods that low wealth can sometimes have very high allocations to risk as a form of lottery ticket that may be the only way out.
[2] I'm probably being overly dramatic in here. I think if a model were able to see the near future a little better with a narrower band of outcomes where there is maybe less downside risk than expected and maybe even a little less upside I'd both be on board and willing to spend a little to see it.
[3] in Stress Testing Monte Carlo Assumptions (2013), Marlena Lee comes to this conclusion: "Monte Carlo simulations incorporate many assumptions that simplify reality. These assumptions are not perfect descriptions of the world, but they appear to be decent approximations for some purposes. Moreover, simulation methods that better reflect historical returns do not dramatically impact results in our setting. Bootstrapping returns to account for extreme tail returns has little impact on the simulations relative to a simple assumption that returns are normal. And although expected returns on equities do vary through time, it seems reasonable to simply assume that expected returns are constant through time. One important assumption that does have a critical impact when using Monte Carlo simulations to project absolute future wealth is the long-run expected rate of return on equities. Changing expected return assumptions dwarfs differences that arise from all other assumptions examined in this study. When using Monte Carlo to project future wealth, no tool, no matter how many bells and whistles, can escape this fundamental problem. Expected future returns are unobservable and are incredibly difficult to estimate precisely."
No comments:
Post a Comment