Pit of Success: a 2003 viral ad lib
I think I’ve told this story a bunch of times but I don’t think I’ve ever written it down so here goes.
You might have heard this description:
The Pit of Success: in stark contrast to a summit, a peak, or a journey across a desert to find victory through many trials and surprises, we want our customers to simply fall into winning practices by using our platform and frameworks. To the extent that we make it easy to get into trouble we fail.
Those are words I gave Brad Abrams right after a talk I gave in 2003. He wanted a soundbite for his blog. But there’s some interesting context to the talk itself.
The event was the Microsoft Research Mindswap, October 3rd. An annual event where a bunch of cool people from MSR have a full day offsite with a bunch of senior engineer types. This particular year was about managed languages and I had been asked to give a talk on the CLR and in particular on CLR and .NET performance. I was the CLR perf architect at the time so I guess that made some sense.
I prepared a talk that was kind of a psyche out. The first 90% of the talk was about the various things we had done to improve performance. It was a long list and a lot of the work was Herculean — nay Sisyphusian — in its execution.
The chief thing we were working on at the time was making the code less costly when used in large amounts, which meant reducing binary size and in particular reducing the parts of the binary that could not be shared — the so-call “private pages”. This work was very difficult and involved carefully tracing page faults and then removing or colocating hot code and data while relegating unused items to pages that would never be loaded. There was a lot of splitting and repacking of information and we were saving a few 10s of kilobytes with each new idea.
Over the course of maybe 8 (very productive) weeks there had been about a half dozen such ideas. I planned to go over this all earnestly characterizing these all as huge wins.
And then I planned to drop the hammer.
In that same period, minor changes in the startup path that used a little bit more of System.Xml had completely eclipsed our wins by roughly a factor of 10.
This was the real thesis of the talk. The framework was too hard to use. Well maybe not hard to use — the problem was that it was easy to use. And it was especially easy to use it poorly. It was in fact quite hard to use it correctly and economically.
So on the day of the talk I was the second speech, first after the keynote. I’m sure it was a lovely keynote but I remember nothing of it. I was lost in my own thoughts psyching myself up to give my talk and I was a bit nervous. This was not an ordinary crowd. There were at least four Turing Award winners in the audience and as many MS Technical Fellows and probably as many Distinguished Engineers. So, yeah, holy mackerel. MSR had some serious mojo.
So I got up there and I gave the speech of my life. My energy was going up and up as I gave the talk and by the time I got to the switcheroo I had everyone’s attention. I wish I could remember exactly what I said, but I can’t; it was totally unscripted. But it went something like this: “Success can’t look like this! We’ll never win like that! Success has to be like falling into a BIG PIT. You can’t help but win. You try to do it wrong, but nope, you fell in The Pit of Success. You Win Again!”
I’m sure that’s not exactly what I said but it’s probably pretty close.
After the talk Brad Abrams approached me; Brad and I were close colleagues in those years often working together. He wanted to write about the talk and that’s when he asked me for a soundbite. So I wrote that “…in stark contrast…” quote for him. He had quite a following and the meme kind of got a life of its own.
These days people sometimes quote the “Pit of Success” philosophy back to me — sometimes they know and sometimes they don’t. It’s usually amusing.
After Brad had written his article, I thought to myself “Hey I should have my own blog” and so Brad helped me set one up. So that was the start of Rico Mariani’s Performance Tidbits which ran for over a decade.
The rest of the offsite went really well. I wasn’t the keynote speaker but I think in retrospect my speech ended up setting the tone for the offsite.
And that’s the story of how an unplanned ad lib turned into a common best-practice tip for the industry. Wish I could put that in a bottle.
