Wednesday, October 29, 2014

Expected value

 I wanted to write this post partly because I haven't written in a long long time which is bad for health and partly because I've been thinking quite a lot about the state of my well being in the past few tumultuous months.

This summer was my introduction to a world of pain and despair, namely doing Bayesian Models. In the past, I've often congratulated myself on not having chosen fields of study that required patience, grit and the ability to keep growing cultures day after day, waiting for a better tomorrow.  The readers of this blog are probably familiar with my views on instant gratification. However, in the realm of Rev.Thomas Bayes, there isn't such a thing as gratification, let alone instant. This is the first time in my life I have run code over 32 core processors with 1 TB RAM only to find that Bayesian models take just about 7 days to tell me that I am absolutely wrong. The next seven days would go into figuring out a subsection of the problem which would then cause something else to break in the next seven days. Basically after three months, I had nothing to show except the fact that I had crashed the CMU server a few times (which felt oddly satisfying) and that I had written code like I have never done before.

In my undergrad, when people talked about memory efficient programming, I've often wondered why on earth would anyone need to do that considering all that memory in the computers. During this summer, the happiest days were when I managed to reduce the time for one iteration by three seconds. This is 3x80000 seconds that we are talking about - a time that would otherwise be spent with nerve wracking anticipation and bags of candy consumption. Heck, forget the processors, this is the kind of waiting that my genes are not programmed to handle. 

Basically, my respect for people who do Bayesian Statistics as their main area of research just soared up. The amount of Math and programming these guys did, in what seems to be the days of yore (which is essentially pre-Google), just makes me gape in awe and wonder. These people would probably click their tongues at me and shake their heads collectively on how spoiled kids these days are with their fancy Stackoverflow and parallel processing. 

Anyway, after this three months of masochism, I went back to school as a changed woman. Between homework and midterms,  I moped so much that there was a sufficient cloud of gloom wherever I went. When the situation calls for my special brand of relentless narcissistic pessimism, I always step up and gleefully spread the depression all around, even in a place filled with despondent grad students. My adviser, however, cheerily reassured me that this existential crisis was nothing but expected out of a newly minted second year PhD student.

This brings me to the next phase. Yesterday, I was sitting in a seminar and when I took a quick look at the room from my vantage point, it suddenly dawned on me that I was sitting in a seminar at Carnegie Mellon listening to a professor talking about a complex Econometrics model. The room was filled with so many accomplished Statisticians, Economists, Criminologists and Mathematicians. It suddenly hit me that this is the bunch of people who trusted me enough to let me in and give me an opportunity to do things that I would have otherwise never ever done in my life - not even in my wildest dreams. Here I am, sitting in the same room with them and hey, some of the stuff on the slides even looked familiar that I didn't have to pretend like I understood!

This gave me a chance to think about just how far I along I have come in life and how I never really acknowledge that. As a screwed up kid, whenever I thought about future, life beyond a decent engineering college always tapered off into uncertainty. It struck me that given the kind of circumstances I grew up in and given the kind of background I am from, I really have exceeded every expectation I had for myself. Everything beyond this point is just a bonus in so many aspects. 

It is actually tremendously liberating to think about my PhD this way and it helps me calm down to focus on the nitty gritty of research that I usually wave my hand about.  All I need to do now is cut myself a lot of slack, dig into the tubs of ice cream and then get back to doing the next assignment.

Each day at grad school brings on a new set of challenges and in this pleasant randomness lies my hope, which in turn, springs the eternal. After all, man never is, but always to be blessed.

P.S: I owe the pun in the title to the three hours of Advance Econometrics problem sets, but felt appropriate to the context.