_meta

I left it in Los Angeles.

You may have seen this graphic visualizing the relative speeds of accessing L1, L2 cache, RAM, and a disk seeking. Jeff Dean expounds in a presentation including other similar measurements, and hacker news hypothesizes about other analogies. I’m surprised nobody hit on this one, though:

Just slow things down by a factor of 10^9:

If L1 cache is like remembering something off the top of your head, then L2 cache is like remembering it after thinking for a couple of seconds - or maybe glancing at a cheat sheet next to your monitor.

RAM lookup is like walking down the hall, finding someone who knows the answer, asking them, and walking back.

Disk seeking, by comparison, would be like walking from New York to Los Angeles, taking two months to find whatever it is you need, and walking back to New York (assuming you walk 10 hours a day, at 5-6 miles per hour).

That’s a long walk.

(Number from here)

Math:

13700000 sec / (3600 sec/hour) * (24 hour/day) ~ 160 days

(2*2776 miles) / (5.5 miles/hour) / (10 hours/day) / (24 hours/day) ~ 100 days

Which leaves you two months of wandering around LA :)

Bing vs. Google

Bing commercial: http://www.youtube.com/watch?v=egwT1KjG6tM

Google commercial: http://www.youtube.com/watch?v=nnsSUqgkDwU

Speaks tomes, IMO.

RIP, Python

I’ve been trying to keep a cool head about technology as of late, but PEP 3003 — Python Language Moratorium makes me want to scream. I understand the practicality of it - that IronPython and Jython are having trouble implementing the new Python syntax when it’s a constantly shifting target.

But children grow: they grow out of kid clothes, out of kid books, out of kid bicycles. And although that’s all quite impractical, taxidermy is not a valid solution.

And I don’t know what’s worse - that the moratorium was proposed and implemented, or that nobody bothered to speak up against it. Both make me sick.

EDIT (12/7/09): I was wrong about this. Jesse Noller convinced me the Moratorium is what’s necessary for Python, as a language, to grow. In hindsight, I should have made the argument that this is bad for Software Engineering as a whole, but I’ll leave that for another post.

Sleep depravation, adrenaline, and a thought experiment

On my way to stay at a friend’s place last week, I took the wrong train uptown, ended up in the middle of Harlem sometime close to midnight. To make things more interesting, this was the end of a long story (better told another time) because of which I hadn’t slept for about 40 hours. Although I didn’t realize it at the time, between the adrenaline rush of walking through Harlem and the sleep deprivation, but something interesting happend in my mind.

Sometime after I met up with my friend when the conversation turned to linguistics, and we ended on the wikipedia article for phonemes - the smallest segmental unit of sound employed to form meaningful contrasts between utterances. I’d heard the definition at least a couple of times before, but this time, something unexpected jumped out at me:

The definition of phonemes involves a person. A living, a breathing, but most importantly, a listening person. The k in “kit” and “skill” are different sounds, but is only noticeable notices (different phonemes) if your native language has two words differing by that sound. For example, if a new word were introduced into the English language, say, a khat (you wear it on your head, but also serves as a cape), the single phoneme of /k/ would be split into /k/ and /kʰ/.

This is curious, indeed. However, this train of conversation ended in a question even more curious - what are the phonemes of a newborn? This is the heart of the thought experiment:

What are the phonemes of a newborn? How do they develop during language acquisition?

Imagine you’re a newborn, and you hear sounds. But you know no language, so all of the sounds are a blur. As you hear things, certain patterns come up over and over - ka, da, bu, ba, mi, doy, toy. Traditionally, one would start thinking about “learning” those patterns, as if you were writing them down on a blank page. But what if, instead, we think of your phoneme partitioning into more and more parts, until you reach a set of “mature phonemes” - all those of your native tongue.

In what order would that happen? Does it partition into vowels and consonants, then directly into mature linguistic phonemes? Or does it partition into syllables, then into mature phonemes? Can we answer these by looking at speech development, or is there too much noise introduced by the physical aspects of vocalization? Just because some sounds require extensive vocal dexterity doesn’t mean they’re hard to discern by ear (r and z come to mind).

I’m also suggesting that we can generalize phonemes to include learning to differentiate between other sounds. For example, it’s clear that both music intuition and native language develops with less effort during young age than later in life. Can the understanding of patterns underlying the chromatic scale be described by the same model as that of language? More importantly, if we are to test hypotheses about a model of generalized phonemes, can the variations in the time people started playing an instrument make up for the lack of available data’s variation in age at which people learned their first language?

To end on a personal note, my daughter’s first commonly used noun was “aff” (meaning dog), and first verb was “oo-pa”, meaning “fell” in Russian. She was using both long before she could mimic my going “ooo” or “aaa”, even before she started saying “mama” or “dada” referring to my wife and me. This would suggest at least some syllables are at least equally as fundamental as phones (distinct sounds).

The Problem with Unit Testing

Let’s get one thing straight: when done right, unit testing can be a huge benefit in most software projects. The key is doing it right, within the context of an organization creating software. That’s hard. It’s hard because the value of unit tests is affected by many variables, and measuring “number of unit tests” and “percent of code coverage” makes it too easy to ignore them. Foremost on my mind are:

Importance of correctness (C): Not all code is equally vital to get correct. Sometimes you add a disclaimer that something still being perfected, and wait for users to tell you whether they care or not. If users are OK with a quickly hacked estimate, writing corner-case unit tests first and getting the code perfect can be an enormous waste of resources.

Speed of iteration (I): If a system’s requirements are expected to change quickly, implementing them with extensive unit tests every time will take significantly longer than quickly creating working prototypes. Approaching a problem by concretely defining it with unit tests first will slow down the time of evolution, which is vital in the exploratory stages of a product.

Complexity of architecture (A): A simple, intuitive architecture has less need for unit tests. It’s the fundamental advantage of good abstractions - an abstraction can be unit tested once, guaranteeing the correctness of all implementation. In a component or problem domain where abstractions aren’t readily available, and architecture must be complex, unit tests are significantly more valuable.

The most basic formula for unit test value, then, is (A * C) / I. This formula needs to be re-evaluated for every component in a system at all stages of a projects. User-facing API’s will have significantly lower I, and hence benefit greatly from unit tests. Moving from “Alpha” to “Beta” to “Release” increases C, and decreases I.

This exposes the fundamental problem of unit tests, when considered in the context of organizations developing software: unit tests give people a false sense of confidence in their perception of quality. It falls under a generalization of Greenspun’s fundamental problem in software - that there are no published standards for how software development. Over and over, I see “number of unit tests” and “percent of code coverage” used as simple heuristics to measure code and architecture quality. But this couldn’t be further from the truth: first off, these numbers are easy to game, and secondly, they’re meaningless without context. Using them as heuristics gives bad incentives to developers and widens the gap between perception and reality regarding quality, and we’ve got way too much of both as is.

Remember your seconds

One of my best friends in college had a watch with a little explosion animation that went off every second. John said there’s 60 seconds in a minute, 60 minutes in an hour, 24 hours in a day, 365.2425 days in a year, and about 80 years in a lifetime. Every time a second explodes, there’s a little less of your life left.

Seconds are easy to forget, and that’s why the watch is important - to reminds you of the seconds. John would interrupt trailing conversations with “gotta go, the seconds are exploding”. That’s probably very important advice, if one can remember it: don’t waste your seconds, or they’ll add up to a wasted lifetime.

Understanding Education

There has been a lot of talk about “the higher education bubble” in the US - the basic gist being that colleges are charging more money than the value they are providing. The most concrete summation of the points I’ve seen is Robert Cingey’s “Burn Baby Burn”. It does a great job explaining the position, but suffers from some fundamental misconceptions.

Misconception: The value a University provides to customers (parents) is knowledge.

Reality: For most customers, the value of University is that it provides them social status (my child is going to Harvard), and their children (1) a stepping stone between the dependent state of childhood and self-reliant adulthood, and (2) a degree. Yes, a mature, determined, and hard working adult can get the knowledge most get out of their undergraduate years in a fraction of the time, but that’s not what parents care about, and even if they did, most kids going off to college are none of the above.

Misconception: The value a University provides to consumers (students) is knowledge from professors

Reality: I thought a lot about this during my undergrad years (in particular, why I should or shouldn’t drop out), and the answer is crystal clear in hindsight: time. I was never one for problem sets or studying: by senior year, I probably spent less than 10% of my time on anything which would bring me closer to graduating. I wasn’t much of a socialite, either, so the rest I spent on anything my curiosity would lead me to (and, ahem, girls). So I spent the vast majority of my senior year reading - certainly more time than required by any full-time job. That, coupled with the ready availability of people much smarter than me, is much more valuable than anything textbooks can provide.

Most critiques of higher education miss the fact that in most undergraduate programs, the customers and consumers are different people - parents and students, respectively - with different value provided to each.

Misconception: Professors see their teaching as having economic value.

Reality: Professors see their research as having economic value. There was a consistent rumor at the Rutgers CS department that the undergraduate teaching award was a kiss of death to anyone’s chance at getting tenure. If you’re great at teaching undergrads, the reasoning supposedly goes, then you’re not putting enough effort into your research. Now, this might just be my imagination, but it seems the only real motivation for tenured professors to spend time with undergrads outside of office hours is to recruit smart grad students to the school, and preferably to their lab.

Part of Cingey’s plan is that “each expert charges by the minute with those charges adjusting over time until a real market value is reached”. But just last week, I contacted one of my former professors about particular research project I was interested in. He wasn’t familiar with it, but pointed me to a colleague who might be. The colleague, whom I had never met before, was more than happy to meet up to go over the work and discuss any questions I had (and get this: for free). But it seems so outlandish to even bring up paying a professor to talk about research they’re interested in. It’s part of their duty as academics to pass on their knowledge to interested students. They get paid by the number of grants they generate, which supposedly correlates to the amount of novel research they create. Teaching undergrads is simply a (very worthwhile) duty of academia as a whole.

In order for a new system to “pop the education bubble”, one needs to provide parents with an alternative to “we’ll help your kids grow up” and students with an alternative to “we’ll get rid of your parents”. An educational system based on iTunes isn’t really doing either.

1.

2.

3.

Backtracking Decisions about Programming Language

Two ideas:

One: “You’ll never find a programming language that frees you from the burden of clarifying your ideas.”

Two: “I suppose [human] protocol is that the party who doesn’t understand will ask for the speaker to rephrase until s/he does understand” -ASW

Backtrack assumption (one), that programming languages need to be unambiguous, and ask how could we analyze existing programs to disambiguate the meaning of code. How would we model programming languages after instructions we would give another human - first, explain ambiguiously, then clarify the ambiguities? In particular:

1. Could we use machine learning algorithms to analyze existing source code to decide ambiguous instructions?

2. Could we do so without loss of generality (not for specific domains)?

3. If we can’t do (2), how can we specify program domain-specific patterns to look for?

What is Bezos apologizing for?

So Bezos appologizes to Amazon customers, but for what? For providing the publishers with the rights granted to them under copyright law?

Surely, it isn’t Amazon’s responsibility to judge which laws are moral and which aren’t - that’s what we pay all those people in Washington for. And what exactly is wrong with copyright law as it stands? The Constitution allows congress to enact copyright laws in order to

…promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries

Well, it’s certainly not unthinkable to imagine Authors and Inventors, given the right to license their creations in a way that allows them to un-sell them at will, will be more likely to create them to begin with.

The misunderstanding seems to lie in the idea that when I buy a book that’s somehow the same as buying a loaf of bread. I suppose it’s ironic that this frame is promoted by those claiming that copyright violation is “theft”, but that’s not the legal reality at all - a purchase of a copyrighted work is limited by copyright law in a large variety of ways - from limiting your “right” to copy and share it with a friend, to the creation of derivative works, to public performances of the work, etc.

The publishers are not the enemy

Amazon is not the enemy. Neither are publishers. Let me tell you a secret: publishers aren’t smart enough to be evil.

Simple Biology

“Publishers” are like very simple molusks of the genus “corporation”. Their central decision neurons (there’s less than a dozen in each one) have been selected by a thorough process of pedigree, polite eloquence at dinner parties, and an instinctive covering of the anus. A close relative of the “publisher” is the species “record company”, which has been dying a loud and violent death to a change in the “digitization of content”.

Amazon

Along comes Amazon. This is also a “corporation”, but a very different kind. Amazon sells virtual server magic on an unprecendented scale. Jeff Bezos (one of the central neurons) coined the term AAI - Artificial Artificial Intelligence - to describe something they make there. They made an iPhone app that lets you take a picture and have someone halfway around the world find the product.

Amazon says “hey, let me digitize that content for ya”. I don’t know about the looks on their faces, but I can certainly imagine the state of the publishers’ sphinkters at that point. But Amazon makes a strong case: the internet is not a fad, digitization is coming either way, and our partnership will make you a player in publishing of the 21st century.

Bad Dream

Now imagine some big publishers’ executive (in a bad dream, most likely) stumbling on his gandson’s news.YC tab, and thinking, for a moment, what it would be like to not negotiate the rights to pull content off users’ devices at whim. I bet it would gravitate toward his next meeting with the board, the part where someone asks, slowly:

So copyright law gives us rights over our content that you decided to waive why, exactly?

But that would never happen, because if there’s one thing he knows how to to best, it’s to avoid risk.

Simple Economics

So avoid risk is what he does, and e-reader manufacturers are happy to compete for content by maximizing his risk avoidance. They’re looking for market share, after all, and the device with the most content will win. I imagine it must suck working in publisher relations for Sony right now. When someone looks you in the eye and says:

On the Kindle, I can distribute content and pull it back the moment I feel like it, why should we give up that right by publishing with you?

My answer better be convincing, ‘cuz he’ll be explaining it to the board.

Take-away

Take a deep breath. Hold it in, count to three, and breathe out. Relax.

Life isn’t perfect, but you’re not surrounded by enemies, either. Copyright law creates an inoptimal system of incentives for corporations, and inoptimal parts of government are susceptible to influence by special interest.

That’s OK. Both will get better with time. Give a bit each month to the EFF (I’d suggest about as much you give to publishing, phone, and cable companies), do whatever else you can spare, but don’t stress out, and don’t take it personally. We’ve fixed bigger problems, so it’ll be OK. I promise.

Programming revisited

This is so painfully obvious, I wonder if it’s worth repeating:

“Even if you’re designing for professional programmers, in the end your programming language is basically a user-interface design. You will get much better results regardless of what you’re trying to do if you think of it as a user-interface design.” -Alan Kay

But then I see how languages are designed - with little or no consideration given to details of how people will interact with the code (user experience is the job people writing IDE’s and text editors), and I realize how little we understand about what programming really is.

A hundred years from now, this is the conversation I look forward to having with my great-grandson:

“But gramps, why would you need to go to school just to tell computers what to do?”

“Well, back then, we hadn’t figured out ways of modeling thought in a way computers could act on like we do now.”

“What do you mean? Why couldn’t you just tell computers what you wanted?”

“Well, for one, people weren’t nearly as precise thinkers - they’d constantly confuse metaphors into what they were thinking.”

“But people have these problems now, too - thought is inherently imprecise, and we’re constantly learning to express ourselves precisely - isn’t that what clearthought is for?”

“Hm, I guess clearthought is kind of like what programming was then, with what used to be called philosophy mixed in. Either way, with computer speeds back then, the kinds of probability models they use to deal with ambiguity had to be really primitive. Computers would just execute exactly what you told them, even if there was no chance you could possibly want to do it.”

“Really? What would they do?”

“Well, break.”

“How can a computer break? You mean, they’d keep asking for clarification?”

“Well, in the very beginning, all programs were written in text, and you could only ask the computer to do things once, and then you’d get errors when you asked it to do the wrong thing. There was no clarification loop, computers would do precisely exactly what you told them to.”

“No clarification loop? But how would you know the computer was doing what you wanted it to?”

“Well, people would say the same thing three times - once in code, then again declaring types and again in tests.”

“That seems really silly. Why not just tell the computer what to do? I don’t get it.”

“Lucky you.”