Feb 7, 2018

I solved my recent EM Gaussian mix problem, but...

Ok, so I solved my recent EM (expectation maximization) and Gaussian mix quantitative finance problem...but now I am irritated. And it wasn't easy for me.  But let's get this out of the way first.  This was my purpose:

1. I just wanted to learn something new and complex for the f*** of it because that is what I do. We all need our strange hobbies, and

2. Being able to describe non-normal distributions mathematically enhances my ability to do financial modeling in life-cycle simulation and some other things I do by quite a bit.

But here is my irritation.  I am 60 and my brain and eyes are decaying daily.  Searching and reading 1000 false leads online, which might have been fun and easy at 25 is no longer fun or easy.  I can't read screens for long periods of time and even if I do I have kids that are hungry or need laundry done or forgot to tell me they need poster-board for a project tomorrow or need a ride to a friend's house or, god forbid, are sick. Then, even if I have their cooperation and I have good eyes and time, I do not have access to work colleagues or professors or computer scientists or whatever else. This is what I have: I have my cats; I have the checkout people at the grocery store; I have my daughter at Stanford; I have reader David C in NY; and that is about it.  When I want to figure this stuff out I have to beg strangers in the world by email (usually no response) or send requests to yahoo answers or Ask Dr. Math (totally hit and miss) or, most often, figure it out on my own but with the constraints described above. 



This time around I punted and got the easiest thing I could think of: "Quantitative Finance for Dummies." But even that guy threw me off.  You'd think a "dummies" book would hold your hand and spoon feed you but that is not the case.  This guy still hid behind the obscure notation and had no workable examples or data or spreadsheets or code snippets.  I can get that kind of obfuscation from the academics.  I wanted Dummy! 

The way to beat this, I discovered, is totally counter-intuitive. And it is also upside down and backwards.  To solve this I had to actually do two things:

a. Upside down -- I had to go to more complex and comprehensive sources to get the full set of equations and methods. That was not fun or desired. Then I had to work my way back to the simple stuff (Wtf and why!? This is not humble brag. I am actually pretty annoyed),

b. Backwards -- I had to steal Excel and R-script code off the internet and then actually code the algorithm (spreadsheet and program) until it worked even though I had no real idea what I was doing and then I had to go back and reread the miscellaneous web sources (not dummies) to figure out what I had done.  I did the same thing last year when I decoded the finite differences process for working through a PDE (and I do not know differential equations at all). 

Honestly what the hell is wrong with the Dummies series.  Also, and I don't mean to pick on the Brits, but the constant use of the word "maths" I personally found jarring but that may just be me and I realize it is more correct than math for some reason and it is a cultural difference that has no real importance anyway. Glad I got that off my chest. 

Here is an example of the EM thing and I promise I am not going into the details...since I am a rookie anyway.  If I have two distributions, one with mean -1 and sd of .5 and the other with mean 1 and sd .5 and the data is 75% weighted towards the latter and 25% to the former, I can construct a combined distribution.  That's cool but uninteresting. More interesting is having the same combined distribution in real data but then having no idea what underlying [normal] distributions may possibly describe it. I personally want to figure that out so that I can then combine them mathematically using software to create a custom distribution or describe some odd real distribution I run into. I realize this is kindergarten for quant-fins but this is a productivity leap for me. 

Look at this distribution. It's like a freaky, extremely exaggerated version of real market data.  This is the combined distribution I described above...


Now if I wanted to figure out how to describe that in a formula for a simulation, I can. I just decode it algorithmic-ally and then I use the results to mix distributions programmatically.  That is a nice step.  For me anyway. No thanks to "Dummies."












No comments:

Post a Comment