Some Observations on my Machine Learning Project

The original post and all the associated links and references, for context, are here:

• https://rivershedge.blogspot.com/p/machine-spending.html

A recent question from a correspondent, one that I asked myself in the last post was:

"Why bother to run a sloppy, slow, imprecise machine when one can access the insight directly, accurately and faster by other means?" [paraphrased h/t to David Cantor]

Good question; cuts to the core of my project.  Let's see if I can rationalize what I call "riding a bike in first gear:" a lot of motion and heat for very little forward progress.


1. I learned a little bit.

This was the original goal. Learn something new. Plus I had a challenge goal of getting a program to teach itself something, anything. Think I hit that goal. Plus RL seems to be a hot topic in the trade. 

2. I beefed up some tech skills

Because the machine training is so slow I had to learn some new stuff not to tear out what's left of my hair.  For example, I learned how to set up and run RStudio (for a fee) on Amazon Web Services.  That was more complex than I thought and took a while to unravel. But now I know and that may be useful later. 

Also, the early designs were clunky and inefficient.  One instantiation was cut into too few discrete chunks to be useful and another made a massive mis-use of memory.  I was forced to confront all that and learned some new coding skills. And kept what's left of my hair. 

3. I taught myself something I already knew for a reason

Because this was a novice effort, I used retirement finance since it was something I already knew. I also pointed the machine at questions and answers I already knew in that domain so I'd be able to evaluate what the machine taught itself. Spending is my interest so spending optima were a desideratum. I had other means to see those optima which gave the project its decent evaluative context. Whether that knowledge biased the design or execution in any way is another good question.

4. Might have a new tool for the future. 

So, I used answers I knew to build a machine just to find them again. That sounds like an exercise in pointlessness. Maybe it is. But at least I know the RL method now even if only in a reduced simplistic way. My guess is that this approach might be more useful for intractable problems or problems where answers are not known. Also it seems like the nature of ai to find answers to questions you didn't ask...to the extent you trust and understand the machine.  Maybe there will be some application of this method to something else in the future. 

5. I learned I would not do this the same way again for the same problem.

In order to construct my benchmark in the last post it dawned on me, a little late, that like Dorothy in the wizard of oz, I always had the power to go home. The ruby slippers here, metaphorically speaking, are/is the life-consumption utility simulator I built last year.  I can either let the machine come to its conclusions, which are a little fuzzy and imprecise, or I can do it directly via another method. By "directly via another method" I mean for some given set-up I could tell that code to systematically survey -- at each age and wealth level -- the life-consumption utility across intervals of both spend rates and asset allocations. There is generally only one critical point for each age-wealth combo so: it'd be easy. I mean it would take a while but probably less time than training a machine...maybe.  The machine doesn't systematically survey and choose (kind of like a grid search), it randomly explores and discovers (like a serendipity process).  I think they are doing the same thing, it's just that one is not carefully supervised. 

That systematic max-search could even be built into a machine but that might just be like using a sledge hammer to open a door when the key is in your pocket.  TBD, but I'm not re-coding anything anytime soon.  

6. I learned a bit more about payoffs

In trading, gaming and something like pari-mutuel horse betting, it's not just about probabilities, it's also about payoffs or consequences. I'd hazard an opinion here that this point is often under appreciated in retirement finance. Maybe some get it but not enough. 

The cleanest example of this principle is in trading, my trading. I was consistently "wrong" 70% of the time (i.e., that's the probability, and a terrible probability at that) but I also had profitable trading because I strictly managed return:risk to 3:1 or more (there is the payoff or consequence). I've been at least seeing more of this consideration in the lit over the last couple years. It's why you'll often see critics of "risk of ruin" metrics (i.e., the probability) talk more now about the missing consideration of "magnitude in terms of years ruined" (i.e., the payoff).

The convexity baked into utility theory, with some manifest objections by some I might add, captures at least this idea (payoffs) well.  In my recent project I ran into it a couple times. First was in something mentioned in the last post where by "sampling" less (see the link for what I mean) it looks like I lost the payoff consequences of the spend-choice to some degree and therefore got a more traditional finance answer to my question. When I sampled more, the steeply penalizing consequences then intruded more consistently and gave me a different answer that was more consistent with economics.

Second, in most of my machine-related posts, in order to be consistent, I was presenting constant-wealth[1] answers. When looking at other wealth states at some age (let's say 75), however, the answers, to me, were a little counter-intuitive. For example, for $500,000, say, the spend policy was higher than at 1M and for 3,000,000 the discovered spend policy was lower.  My interpretation was that the consumption fail from x% of 3M down to to hardscrabble subsistence level is penalized way more than a fall from x% of .5M due to the convexity of the utility payoffs. Looks like spending policy is reined in by the machine when the consequences are hard.  Easy enough to test that directly but I haven't. 

-----------------------------------------

[1] in other words I was looking at spend rates at each age as if we were to have $1M each of those ages. It'd be a little like lining up a 60yo and an 80yo at a starting gate, each with 1M, and then telling them to figure out what they can spend. This is a different question entirely than asking a 60yo w $1M what she thinks she can spend at 80. It depends on the state at that age.  For all we know she'll be destitute or super rich at 80.