In the long run

It’s midnight, you’re at your favorite bar and the girl or guy of your dreams walks through the door. You know you need to talk to them, but immediately wonder how.  You can use that pick up line you planned a million times, you can fumble over some new line you just learned, or you throw a hail Mary and just say “Hi, I’m [insert your name nervously], and I think we should grab drinks.”

To all you young statisticians out there: don’t try any of these. The problem with statistics is, we always think about the long run; never the short run.

In statistics, we use this term “the long run,” stemming from a really nerdy concept called ”The Central Limit Theorem.”  I’ll leave the work of proving the theorem for our readers; suffice to say it basically states that if we do something enough, we will eventually get the actual permanent outcome.  

To some extent, our lives follow the central limit theorem in that our daily routines tell us what we actually do.  Think about your daily wake-up time: you set your alarm for 6:30am, but rarely actually wake up at exactly that time; some mornings you wake up at 6:35am and other mornings 6:25am.  If we took all those various wake-up times and graphed them, though, they would wind up normally distributed with the mean at 6:30am.  I’m stretching the realities of statistics slightly, but this is the basic concept.

The point here is this: our expectations are build around what is going to happen in the long run. But as the nervous person at the bar, there’s no such thing as a long run; because of time constraints this evening, a really short run is all we have.  

Some life events and decisions – like careers, friendships, and pick of our favorite foods – give us plenty of time to play out.  The sample size of these events is large, so we know if our expected value is going to return positive or negative results.  Other things, however, don’t have a long run.  It’s the championship game and your team needs 3 yards for a first down; if not, you lose the game (and your bet!).  In the long run, the average NFL team goes 2 ½ yards on that play (believe me; I’m a statistician).  But you need 3 yards, not 2 ½.  There is no long run; the whole season comes down to this short run.

I was recently at a casino, but needed to get back home for a friend’s birthday party.  Time was limited, so I couldn’t play the long run.  I took my $20, marched up to the roulette table, put it on my birthday (12 and 6), and hoped the odds would be in my favor.  If I had time to sit down and gamble, I may have hit those numbers in the long run – but I needed to leave.  Lucky for me, I wound up winning on the one hand I played!

When we have time to think about things and make informed decision, we should do what the long run dictates; when we don’t have time, just go with it.  When statistical probabilities exist, use them, but don’t fear when they’re absent.  When thinking about where you’re going to live for the next year, think through the probabilities.  If you’re picking where to go for dinner next week, think through the probabilities.  But if you’re at a bar, and the person of your dreams walks in – forget the probabilities.  In the short run, they probably don’t matter anyway.

Is the optimal life worth living

Like all good blogs, I’ll start off by quoting some historical philosopher. Socrates once said

“The unexamined life is not worth living”

I counter him with, is the over examined life worth living. In a day and age where everything we do is stored on someone’s server for our access, we can monitor just about everything we do. With products such as Nike’s running companion, apps that track your sleep or even apps that count how many push up’s you’ve done this year. The question is why do we download and use all these services. Mint helps us budget and learn our spending habits, dating sites help us find loved ones and car apps allow us to pay our insurance. All of this data makes us examine our lives way more that we did 40 years ago. The question then becomes, with all these apps and services, can we optimize life.  The optimization problem, mathematically is shown below

We are basically trying to minimize (or maximize) our function (say life), held to some constraints (say money, love, time). This may sound too romantic, but if I handed you the ultimate function to have the best life would you take it? How do you know that this optimal function is for you too, what if the function I handed you was optimal for you too? As a statistician, and someone who believes in the numbers, I somewhat struggle with this thought. Even if you handed me an optimal function to solve for, I’m not sure I would take it. First and foremost, that would be a really really complex equation, something that would take computers years to solve. I love reading books every Friday, but if this function said, reading books makes you 4 times less likely to make money, would I stop reading books?

Optimly™

I would say that the data era, as it continues to sweep the world, makes us more aware of our problems. As these errors become way more apparent, we will have the completely opposite problem we had 40 years ago. The question will no longer be what’s wrong with me, but rather which of these 70 problems should I tackle first. Will we have apps that help us manage our apps? Will optimly, the name of said app, do a cost benefit analysis on the fly for you? Will it tell you at what point you should have your last drink if you plan on making that 10:45 flight tomorrow. Will optimly take into consideration the 3 hours you were chatting up the girl for, will it vibrate to say ‘Bro, you had your shot, time to go home’.  Image if we examined all these factors at once, if we worried that the drink I had Tuesday night at 11:01 (which was a good one mind you), how that would effect me 6 months down the road, I’m sure data scientists are developing apps that help determine this. Don’t get me wrong, I love data, I love optimization, but I think sometimes we have to take a step back and ask ourselves; are we over thinking the problem?

Smarter Data

If data is here to help us make better decisions, and ultimately optimize the life, do we comprehend the other percentages? My case in point is, in the NFL, you are way better off going for it on 4 down and 2 then you are either punting the ball or going for a field goal. However, every time a coach does it , he is either lucky or dumb, never smart. The dumb portion, being that 60% of the time he will make the first down and the other 40% he will not make the first down. The sad part is, that’s life. You can use Optimly as exactly as it says, in bed by 10 pm, only 2 drinks [to avoid cancer of course], you can bring a girl flowers but she may still not like you. Life, as I have harped on, is about the percentages. Its what makes it interesting, we a subtle betting against the odds, we doing things that optmily (knows or thinks depending on your interpretation) are dumb, like  never stay out past 2 am. Yet some of our best stories are Optimly’s worst nightmares, the staying up as the sunrises with your best friends, not studying until right before the examine, spending way too much a gift for someone or my personal favorite, having a competition to see who can chug the beer fastest. I much be preaching too much, but life isn’t supposed to be optimized, its supposed to be lived. Use the data to put you in the right places, but don’t be afraid to shy against the ‘optimal path’ every once in a while.