Jun 24, 2017

Some Differences Between Open-ended MC Simulation and Historical "Closed" Sims

One of the smartest early retirement blogs around these days is earlyretirementnow.com.  This is a guy with skin in the game because he wants to retire early.  He has also put some of his prodigious PhD skills to use in doing the homework necessary to get ready.  He does stuff that I can't touch or sometimes even understand.  I tried to play around with (abuse might be a better word) some of his math here because I liked the elegance of the approach although it took me a while to figure it out.

Because he likes to share what he has learned in his ER journey, ERN now has a spreadsheet that takes some of his math that I was playing with (link above) and that he had originally worked into some GNU Octave code (which is like matlab according to ERN; I have to imagine it is not too different than R). The spreadsheet does what I want to call a type of historical rolling simulation (not exactly, I think, like what others mean when they use that term but very close) starting with data from 1871.  The tool also allows one to look at subset "runs" such as "since 1950" or CAPE <=20 or CAPE >30, etc.



I think this is a useful and constructive addition to the literature and tools of ER finance and I have added his spreadsheet to my own "triangulation tool box."  On the other hand, because it is a spreadsheet it almost by definition has some limitations.  ERN puts it this way: "In Octave, we can calculate a large number of simulations and calculate safe withdrawal rates over a wide range of parameter value assumptions. Millions and millions of SWRs over many different combinations of parameter values (retirement horizons, final asset value target, equity shares, other withdrawal assumptions). That would have been cumbersome, probably even impossible to implement in Excel. But a quick snapshot on how one single set of SWR parameters would have performed over time? That’s actually quite easy to do, even though there are 1,700+ different retirement cohorts between 1871 and 2015."

While it is quite a bit better (my opinion) than other free online historical rolling simulators because of things like his unique approach (using different and pretty slick math and using monthly rather than annual data[1]) and while it is no doubt more than "good enough" it also shares some of the downside of simple online or excel tools.  The main thing I am thinking of here is that it lacks one of the features that is the main reason people use simulators (with all their own faults[2]) in the first place: it under-imagines the possible range of alternative futures because it sticks close to the historical record.  I realize that being too conservative after factoring weird alternative futures into a sim means that one can (gasp!) underspend, but the thing is that no one (for me anyway) will be there if it doesn't work out, certainly not a Ret-fin blogger I don't know.  That's why I like to: a) use more open ended style simulation, and b) triangulate among many tools that all have different embedded assumptions (and methods unique to the analysts that created each different tool). If I underspend for a few years, so be it. I'm ok with a moderate lifestyle and ok with my kids getting a little if I mis-analyzed.

I didn't have time to do an exhaustive study or comparison but I did run his spreadsheet through some simple paces and compared it to some vaguely similar assumptions in my simulator to see what would happen[3].  I noted first that his spreadsheet chart starts at 4% because most input parameter configurations -- due to the historical approach -- result in very low or zero fail rates below a 4% spend rate.  As I mentioned above I think that that's a little too sanguine given what I have read and studied in the past so I modified his spreadsheet a bit to look at spend rates between 2.5% to 6%. I also knew my MC sim would have decidedly nonzero fail rates below a 4% withdrawal rate.  When I line up the two tools, ERN and an MC sim, for a hodge-podge of input variables it looks like this if I got everything right and the got assumptions lined up close enough:

 

Key:
1. ERN - 60/40, all years, no suppression
2. ERN - 50/50, all years, no suppression
3. ERN - 50/50, all years, 10Y return suppression
4. ERN - 40/60, all years, no suppression
5. MC sim - 50/50, no return suppression
6. ERN - 50/50, 10Y return suppression, scenario: CAPE between 20 and 30
7. MC sim - 50/50, 2% portfolio return suppression first 10 years

Conclusions:

What does this tell us and which approach is "right?"  I'm inclined to say neither and/or both are right.  The chart merely highlights in a mini-form that all models are endowed with the explicit and implicit (opaquely embedded) assumptions that are unique to both a style of analysis as well as the biases and capabilities of the modeler.  It also highlights that a simulator will by its nature have a wider universe of outcomes than one limited to history and thus will more than likely have a higher guess at a "fail rate."  While both approaches are right and wrong at the same time you can see why I might get cautious about using a 4% spendrate at an early age when looking at the output of both tools. 

My personal opinion is that one needs to take all models with a big grain of salt. This is why I like to triangulate by using many tools.  That way I am not leaning too hard on any one set of assumptions or world-views.  While the result of the triangulation process will likely, if not necessarily, be more conservative than one particular historical sim result, I'm ok with that because I will do it all over again next year and I may have different tools and different results. I will also have a non-zero probability of changing my mind about my path at that time, too.  


-----------------------------------------------------------------------------------------

[1] it is also faster than simulation which has its benefits.

[2] the extreme and ahistorical sim outcomes can perhaps lead to a little bit of chicken-little syndrome sometimes.

[3] For ERN I used several simple middle of the road allocations 40/60 50/50 and 60/40 (note that he leans towards an 80 or 90% equity allocation and also that he has allocation options for cash and gold).  I also assumed a 444 month horizon (age 58-95), 1% fee, 5% return for stocks and 2% for  bonds or alternatively 3% and .5% for the first 10 years, 20K SS at age 70, etc.  For the sim I used 50/50, age 58-95, misc spend rates, 1M endowment, 1% fee, 20k SS at 70, etc. Note that the MC sim uses a return distribution based on a historical data table starting, if I recall correctly, in 1926.



No comments:

Post a Comment