What’s up with zip?

November 25, 2013

Did you know that across modern programming languages there are 4 completely different implementations of the zip function?

If you run the following psuedo-code in Ruby, Python, Javascript, and F#, you’ll get a completely different result in each language.

Join me as I take a quick dive into how (and why) the functionality of zip varies across different languages with easy to understand examples.

What is zip?

According to Wikipedia, zip (known as convolution in computer science) is a function that “maps a tuple of sequences into a sequence of tuples.” Rephrased and simplified, zip takes a pair of lists and outputs a list of pairs.

When you have lists of equal lengths, the functionality is straightforward. However, when you have lists of differing lengths, there are a few different ways the function can behave.

I was surprised to find out that modern programming languages do not agree on the implementation of this case.

Minimum length

Languages like Python, Clojure, and Common Lisp use the length of the shortest list to determine the length of the returned list of pairs.

I emailed Raymond Hettinger, one of the core contributors to Python, and he had the following to say about this design decision.

The history of Python’s zip() is documented here:  http://www.python.org/dev/peps/pep-0201/Barry [the author of that PEP] is an Emacs guy, so ELisp is part of his core vocabulary. The roots of all zips trace back to Lisp where zipping stops with the shortest input list:  http://jtra.cz/stuff/lisp/sclr/mapcar.htmlStopping with the shortest is a useful behavior because it allows infinite input streams to be combined with finite streams (a reasonably common functional programming technique).

The potentially negative side effect of this implementation is that you can destroy the data of the longer input sequence.

Max length

While Javascript doesn’t have a built in zip, one of the most popular Javascript libaries, underscore.js, provides a built in zip. In this implementation, zip has the opposite functionality of the Python implementation: it uses the length of the longest list and fills the pairs with undefined when the shorter list runs out.

In a Github issue discussing this topic, the maintainer of the library, Jeremy Ashkenas, decided that they would leave the implementation as-is:

Without any super strong arguments in any particular direction (infinite lists are unconvincing in JavaScript, because _.zip has to be eager), I think we should leave this as-is. Not destroying any of the incoming data is a nice feature, and you can always stop iterating when you see undefined values, or compact out undefined parts of your result.

Lengths must be equal

F# (and map based implementations of zip in Racket) won’t even let you use zip with lists of different lengths.

Personally, I think this implementation runs the least risk for new people in a language: if you try a case, which could return different results across different languages, you’ll be protected from losing data (or generating confusing null values) by the type check.

Length of first list

Ruby may have the best (theoretically) and worst (functionally) implementation of zip: it uses the length of the list that you call zip on (it’s a method on the Array object) as the length of the final list of pairs.

With this implementation, if we use zip correctly, we can get the best of both worlds. That said, it’s a language subtlety that will almost certainly be lost of the vast majority of Ruby users — potentially adding more confusion that it’s worth.

Is there a right answer?

If we look at the Wikipedia definition of zip, we see the following line:

Let \ell denote the length of the longest word, i.e. the maximum of |x|, |y|

This suggests that the length of the longest list should be used; however, Wikipedia turns right back on itself and adds:

A variation of the convolution operation is defined…where \underline{\ell} is the minimum length of the input words

In other words, no one really knows. I certainly don’t, bringing me to my next point…

Do you know more?

If you know more about where zip comes from, what the “correct” definition is, or why there’s such variation, please enlighten us — post a comment, tweet at me, or send me an email and I’ll add any additional information to this piece.

Thanks to Joe Wegner and Avinash D’Souza for reading drafts of this. 

It’ll all be small soon

November 22, 2013

Yesterday was my 21st birthday.

Counting down the final minutes of my 18th birthday, I was lonely and nervous. The clock was ticking on college applications, I was struggling with old friendships, and I was terrified of growing up. So, I did what any melancholy teenager would do and wrote my future self a letter. A year later, I read it and wrote another. And so a tradition began.

Each letter gives me an insight into who I was and what I felt at that moment. My language conveys the mannerisms I had; my tone conveys my emotional state; my writings convey my fears, hopes, expectations.

I always find my fears the most interesting. If you pick any random moment from my short adolescent lifetime, you’ll find a version of Jesse who is seriously worried about a handful things. It could be the health of someone close to me, the relationships I’m trying to maintain or create, the organizations I’m a part of, or something I’m building that I desperately want to succeed. The anxiety is tangible.

But, three years in, I’m yet to have a fear that didn’t seem trivially small exactly a year later.

It’s not because the fears aren’t legitimate. Looking back, most of the fears I’ve had were realized in a serious way: the bad things I’m afraid of happening often happen. It’s not because I learned to be fearless. Every year, something new has me up late at night with worry.

Rather, I suffer from a classic human condition. The inability to emotionally internalize concepts which I rationally know.

I understand that all fears fade away and all bad things get less bad, but I feel them nonetheless. My new concerns naturally expand into the empty space left by those which have shrunk with time. I’m caught in a never ending cycle captured through vivid snapshots taken every 365 days, charting my path through it all.

See you next year, Jesse.

November 21st, 2013

Metamorphoses

November 10, 2013

Reading this quote by Gloria Anzaldúa gave me a better understanding of writing.

A poem works for me not when it says what I want it to say and not when it evokes what I want it to. It works when the subject I started out with metamorphoses alchemically into a different one, one that has been discovered, or uncovered, by the poem. It works when it surprises me, when it says something I have repressed or pretended not to know . The meaning and worth of my writing is measured by how much I put myself on the line and how much nakedness I achieve.”

Please read her essay on Third World Women Writers.

 

The smartest investment Southwest Airlines never made

November 7, 2013

When I was in high school, I used to hate Southwest. I didn’t fly it often, but the few times I did, I always complained to my parents about one thing: the lack of in-flight movies (or the latest and greatest in-flight entertainment system). If I was going to be stuck in a metal box for 6 hours, I at least wanted to watch a somewhat-new, not-so-great movie on a tiny screen a few rows away.

Putting aside my obvious ingratitude for the pure ability to fly across the country, my complaints weren’t that unreasonable: while Southwest kept their cabins screen-free, other airlines were piling on the awesomeness.

It wasn’t unexpected that Southwest made the decision to not add in-flight entertainment. Systems like the back-of-the-seat screens we see on airlines like Delta, can cost anywhere between $2 million and $5 million per plane. For a budget airline with over 500 planes, a $2 billion dollar investment (with a <$10 billion market cap at the time) is hard to justify. That said, other low-cost airlines like JetBlue were doing it, so I don’t think anyone would have been surprised if they had made the jump. But Southwest didn’t.

Today, I saw that Southwest, in coordination with Dish, has started offering free in-flight television to customers with internet connected devices. In 2013, with over 100 million iPads and iPhones (and millions of other devices) in circulation in the United States alone, there’s no longer need for the in-flight media systems planes installed in the early 2000s.

I hadn’t realized it until I read that headline: by not installing in-flight entertainment systems, Southwest probably made one of the best long-term (anti) investments in airline history.

But let’s remember that the first real internet connected, media consuming handheld device was first released in 2007 (the iPhone). The iPad didn’t come around until years later, and even then, many were still skeptical that such devices would gain mass adoption. All the while, airlines were already moving full steam ahead on in-flight entertainment solutions. What’s the conclusion?

Either there are some very very smart people at Southwest who made a remarkably prescient prediction about the future of media consumption or Southwest accidentally made one of the best long-term (anti) investments in airline history.

What do you think? Tell me on Twitter here.

If you got rejected from YC, your life is over

November 5, 2013

If you got rejected from YC…

You will never be able to hear talks from experts on startups and tech.

You will never get to to learn from Paul Graham.

You will definitely never meet him.

You will never be able to meet other founders.

You will never get to go to cool parties.

You will never get accepted once you get rejected, because that never happens.

You will never be able to raise money.

You will never ever ever ever be successful. Never.

Your company will never ever ever ever be successful. Never Ever.

Seriously, your life is over.

Definitely don’t email me or tweet at me and ask how Clef has done since we got rejected last year.

Being alone

October 29, 2013

Preface: As a white, heterosexual, cis, male, I’m granted, from birth, an extraordinary amount of privilege. One part of this privilege is the ability to make aloneness a choice; rarely, if at all, in our society will I be forced into aloneness because I’m different. This is something I keep in mind, but I’m unsure how to address. As you read this essay, please remember the position I come from and critically engage with how that effects the thoughts I have.

When we’re alone, we’ve been conditioned by society to believe that we should stay alone.

On Friday evening at 7:30pm, I was alone taking a nap on the floor of Clef‘s office.

The two people I spend 95% of my waking hours with had both left for different parts of the country, my brother who lives in San Francisco was traveling to NYC, and, just like that, the number of real friends I had in the Bay Area was reduced to 0.

I’d already finished reading a paper and had a few more lined up, was communicating over the internet with people I didn’t know, and planned on maybe watching an episode of Boardwalk Empire before I biked home, did my laundry, and went to bed at 11pm. When we’re alone, we’ve been conditioned by society to believe that we should stay alone.

Then, browsing Twitter, this happened.

hipchat

The prospect of going to a party where I knew no one and wasn’t technically invited scared the shit out of me. As I sat at my computer finishing up a support issue, I ran through the possible worst-case scenarios in my head.

I could get turned away at the door. There might only be a few people there and I’d have to explain who I was and why I was there. I might stand in a corner by myself staring awkwardly at my phone while everyone else around me had a wonderful time. People might think “look at that weird guy all alone in the corner.” People might laugh me out the door.

As I pondered the scenario more and more, the actual worst case scenario became apparent: I’d show up,  have one bad interaction, which would have zero impact on my future life (because it would almost certainly be remembered by no one but myself), and leave.

So, with no friends and no plan, I headed across the bridge from Oakland to San Francisco.

If you put yourself out there, people are kind

As you can probably guess, none of my worst-case scenarios came to fruition. When I walked in, the first person I met was Netta, who upon hearing that I’d come alone, quickly introduced me to the entire Watsi team. As the night continued, every person I met was overwhelmingly warm and welcoming.

Yes, I stood outside the door awkwardly until another group of partygoers came because I didn’t know the combo to get in (but we made friends in the elevator and they didn’t seem to notice my alone-ness). Yes, I had more than a few moments where I stood in the middle of the crowd looking awkward (but I wasn’t the only person doing that). Yes, a few people seemed taken aback for a moment when I told them I’d come alone and “uninvited” (but they quickly transitioned to introducing me to their friends).

But, for every uncomfortable moment I had, there were 10 filled with smiles and fun conversations. And, in all the moments I had by myself, I had the chance to cement some thoughts on being alone that I’d been pondering for the last few months.

Being alone can be liberating

As we create friendships, we inevitably construct identity frameworks that our relationships operate within.

We can define an identity framework as roughly the way we present ourselves in a given situation: how we act, how we talk, what we say and what we don’t say (among other things). These frameworks let us understand who our friends are and how, and why, they act certain ways. They are what we refer to when we say we “know someone” (I might even say that they are what we refer to when we say we know ourselves).

By facilitating connection through uniformity, however, these frameworks also constrict how we act when we’re with people we are intimately familiar with. If a friend understands us through a certain identity framework, differing from that framework while around them can potentially cause tension and confusion.

“If you act X way around me, why are you acting Y way around them.”

This dichotomy is challenging: in order to nurture the friendships we have, we need to maintain the identity frameworks we’ve created; however, in order to grow as people (if we accept that growth is important), we need to create new ones and change old ones. Questioning these frameworks is undeniably important, but it is also very hard.

When alone in a sea of strangers, no such frameworks exist, so change is easier — we can experiment with who we are and how we act. We can construct a new identity framework for every new conversation we have. We can try being someone else.

Every time we try something new, we learn something new about ourselves. When we push the boundary of the frameworks we’ve created, we often find that there’s really no boundary at all.

Pushing back on discomfort is important

Humans feel uncomfortable a lot. The tingle down our spine, the slight nausea, the racing heart.

One solution to this discomfort is to step away from situations that make us feel it. To surround ourselves with things we are familiar with. To interpret discomfort as a binary absolute and determine that we’d rather have no discomfort than some.

Another cure is to try to understand why we feel uncomfortable. To put ourselves in situations that make us feel discomfort and critically examine the underlying emotional feelings and logical thoughts that prompt that sickly feeling in the base of our stomachs. To interpret discomfort as a spectrum, wherein we can identify, and address, factors, which push us towards one side or the other. To be comfortable with feeling discomfort.

Being alone, both in a crowd and by ourselves, can be an extremely valuable exercise in questioning and understanding that discomfort.

We’re not the only ones alone

In every situation we enter, there are always other people who are alone. Some may literally be alone, like me at the Watsi party. Others may be physically surrounded by friends, but emotionally alone in a crowd. Either way, recognizing and understanding that the aloneness we all feel is an undeniable part of the human condition is liberating.

When we accept that our feelings are just permutations of a normal condition, we can begin to address the specific reasons we feel alone. When we accept that everyone feels alone, the weight of being an outsider starts to lift and it can become easier to reach out to others.

Aloneness is not loneliness

Aloneness is a noun that describes the physical state of having no one else (or no friends) present. Loneliness is a noun that describes the aching sadness one can suffer from feeling alone.

The two are often conflated in modern society — but they don’t have to be.

When we’re alone, we don’t have to be lonely.

I’d love to hear your thoughts about being alone in the comments or on Twitter.

Thanks to Andrew Russell, Faye Wang, Alexa Cerf, Brennen Byrne, and Mark Hudnall for reading drafts of this.

Make your life easier with better error messages

October 22, 2013

One of the nicest things about running the code you write in your own ecosystem is the ability to debug on the go. Even if your code is deployed and in production, if you get an error, just check the logs, follow the stack trace, fix the issue, retry and redeploy. There’s no barrier between you and your code, no buffer through which you need to interpret errors.

When you start writing code that other people run, everything changes.

Screen Shot 2013-10-22 at 7.27.15 AM

Something went wrong, please try again

At Clef, we strive to make the user integration process as easy as possible. We have one-click installers, walkthrough tutorials (for our one click installers), and 24/7 hands on installation support. Unfortunately, when we first wrote our WordPress plugin, we neglected to properly consider how to handle error cases.

In our plugin, the primary place where something can go wrong is in the login OAuth handshake between the user’s server and Clef’s servers. In that handshake, there are two API requests to our servers, some JSON parsing, and the finding (or creation) of a user in the WordPress database. Nothing too complicated, but all code fails and ours is no exception.

When we first wrote the code to do this handshake, we handled error responses from our API like this:

This was great because it took 5 seconds to write. This was horrible because we poured tens of hours down the drain debugging user issues in the dark. For instance, this is a real conversation I had with a user:

Screen_Shot_2013-10-22_at_8.03.08_AM

 

Debugging code is hard. Debugging code through chat through screenshots by passing snippets of code for users to paste into their files is damn near impossible.

Make your error messages better

g1382455216726682168

When you’re writing error handling, don’t just think about your users’ experience. It may not be that big a deal if a user sees “Something went wrong, please refresh and try again,” refreshes, and it works.

If it doesn’t work though, and the user tells you that they saw the error “Something went wrong, please refresh and try again,” and that’s the same message you use for every error, you’re gonna have a bad time.

What’s the worst error message you’ve ever written?

Follow me on Twitter here and let’s talk about it!

A frustrating design inconsistency in iTunes

October 18, 2013

If you’ve ever searched for a song in iTunes, you almost certainly know what I’m talking about.

Screen Shot 2013-10-18 at 8.36.21 AM

 

Opening things in OSX

Imagine you’re in OSX trying to open a folder.

If you click once on the folder, you’ll get a blue highlight over the row with the folder name. If you click twice, you’ll be transported from the folder you are currently in to the one you double clicked on. It works on your desktop, in a finder window, in a file picker drop down, and when you search — everywhere.

The design pattern also makes sense: the destructive action (changing state) requires twice the intention as the non-destructive action (setting yourself up to gain more information or do a destructive action like drag).

Screen Shot 2013-10-18 at 8.42.19 AM

Searching in iTunes

Now, let’s do a search in iTunes.

When we search, we are presented with a view that closely maps a search in Finder. Our search results are divided into 3 (or more, if you have non-music) categories, which closely parallel the objects we find in Finder. We have artists and albums, which parallel folders (because they group items together) and songs, which parallel files.

Screen-Shot-2013-10-18-at-8.36s

So, if you were to try to access all of an artist or album’s songs, how would you go about it? A double click?

If you’ve ever used the search feature before in iTunes, you probably already know the answer. If you double click, the album will start playing and leave you right where you are.

If you hover over the album, you’ll see an arrow pop up. Maybe if we click that we’ll go to the actual album.

Screen Shot 2013-10-18 at 9.04.54 AM

 

Nope. If we click that arrow, we get a drop down with no option to go to the album.

So, how do we actually get to the album? Click once…and wait.

The idea is there

The primary difference between this situation and the Finder situation is that instead of having one destructive action and one non-destructive action, we have two destructive actions: (1) playing the songs and (2) switching to the artist/album view. The mapping they’ve created is intuitive once you wrap your head around it; the more destructive action (putting on music) takes twice the intention as the less destructive one.

Unfortunately, user experience design isn’t only about intention.

Consistency is important

A user shouldn’t have to think twice when they want to do a similar action in a new place — the action should be mapped in the same way that it was mapped wherever they did it first. This allow users to quickly pickup new systems by using old controls and actions. When you introduce inconsistencies into a system, you leave your user confused and frustrated (and potentially embarrassed — more than once, I’ve accidentally played music in a quiet setting because of this flaw).

Apple’s notoriously good for creating intuitive user experiences — here though, they’ve failed.

Are you frustrated by other user experience inconsistencies?

Follow me on Twitter here and let me know.

 

Make bad data visualizations

October 15, 2013

Last week, I attended Open Oakland[1] for the first time.

I started talking to a group of members who were working on tool to better expose campaign finance information through data visualizations. The idea had been started at a meeting in the middle of August, so when I jumped in (middle of October) the tool already had a Github repo and was well defined in a Google Doc. I’ve created a few shitty graphs in my life, so after a few minutes of conversation, I was excited to dive into an already well defined project and codebase where I could learn from the structure and gradually start to make contributions.

When I went to the Github repo, I found something very different: an empty folder.

So, I did the only thing I knew how to. I built a shitty graph.

Financial contributions over the last year in Oakland
Screen-Shot-2013-10-08-at-8.42

 

All of the labels were cut off, lots of data was missing, it relied on you running a Python simple server in the root, and I had a solid 5% understanding of how the code behind it worked (d3). In other words, it was bad.

Making good data visualizations is really hard

Making good data visualizations is really hard. You need to collect the relevant data, filter out all the noise, put it into a usable form, and then display it in a pretty graph. You also need to build up the APIs to access the data, build the server to host the visualizations,  and worry about how to protect those visualizations, so people outside of your organization can’t see them. If you’re lucky (or smart), some of these steps are already done for you.

At Clef, we decided to build our analytics from scratch (looking back, we wouldn’t do that again, but it’s working great now) and it ended up taking a solid couple weeks of full time work to get a system we were comfortable with up and running. Really though, those two weeks weren’t the problem: the three months of putting it off because we didn’t know where to start were.

Visualizing data is also really important

Until we had our data up on a dashboard 24/7 in our office, we were never really able to understand how we were actually doing (and why that was the case). Once we had the dashboard up, we were able to take the insights we saw everyday and directly apply them to the work we were doing — letting us focus on improving the metrics that mattered and not on the ones that didn’t.

At Open Oakland, even those my bar graph was remarkably shitty, it gave us (and by us, I mean the people who knew more than me) insight into the state of the 2014 mayor’s race that no one else had — only because data visualizations like this don’t (yet) exist.

When you can’t have good, settle for bad

At some point, you’ll have good data visualizations. When you have a team of engineers working full time on building beautiful dashboards and graphing key metrics, you’ll have such good data visualizations and they’ll help your business and improve your day.

Don’t wait until that point to see what’s going on — build a shitty visualization and you’ll gain more insight than you can imagine.

Want to discuss this more? Join me on Twitter.

[1] If you live in Oakland, are a technologist, and care about the City, but don’t know how to best apply your skills, you should check Open Oakland out.

IE seems right, Chrome seems wrong

October 10, 2013

At Clef, we work with a lot of iframes. If you use our easy-install button, whenever you click it, you’re really clicking an iframe, which communicates with the main window (the page you’re currently on), which opens another iframe (the overlay with the Clef Wave) that allows you to sync and log in. Behind the scenes, all of this is accomplished with postMessage — a javascript browser API that allows developers to “safely enable cross-origin communication.” In other words, it allows the iframes that hold our button and Wave to communicate safely with the page they are hosted on.

We broke Clef

A few weeks ago, after deploying some changes to our javascript, an Internet Explorer user notified us that we’d completely broken Clef — when they clicked the button, nothing happened. We weren’t seeing any issues in Chrome, Safari, Firefox, so we quickly assumed that we’d hit an IE “idiosyncrasy” that we would have to work around. Digging a little deeper, we found the source of the problem.

When we’re using postMessage, we need to make sure you’re not interpreting other people’s messages (Facebook posts a ton). For this reason, we verify that the message is coming from our domain (‘https://clef.io’), and if it isn’t, return. Examining the code execution in IE, we noticed that when our message was posted, it failed this test and returned — thus, the window did nothing. What was going on?

host vs. hostname

Host contains the port, while hostname does not…or at least that’s what is supposed to happen. In our normalize function, we use an HTML link attribute to parse out the host and protocol of the origin where the event came from. Examining our code, we quickly realized that we had a bug — if host contains the port (the correct behavior), the domain check would fail (‘https://clef.io:443’ != ‘https://clef.io’).

Strangely enough, everything was working fine in Chrome, but seemed totally broken in IE. Let’s take a look at how the two browsers handle the host attribute.

Internet Explorer 10

Screen-Shot-2013-10-10-at-8.57.44-AM

Chrome 30

Screen Shot 2013-10-10 at 8.56.55 AM

Uh oh. Can you spot the difference? For both HTTP and HTTPS connections, Chrome scrubs the port number from host if it’s the default (80 for HTTP, 443 for HTTPS) — even if you explicitly specify it. If you use a non-default port number, it always includes it.

Screen_Shot_2013-10-10_at_8.56.55_AM

That doesn’t seem correct. When no port number is specified, it’s debatable whether host should include the port (IE obviously extrapolates it from the default port for the protocol). If a port number is specified, it should absolutely be included in the host.

My usual resource (MDN) for documentation on web things was surprisingly unhelpful with this issue. Their page on the host attribute has an example, which has a typo (assumes port 80 for HTTPS) and doesn’t work correctly in Chrome or Firefox (it follows the IE behavior).

From the looks of it, Chrome was wrong and IE was right.  That being said, I’m not 100% sure what’s going on here — is this a bug in Chrome and Firefox or am I misunderstanding something?

Can you help me figure out what’s going on?

I’m not an expert on browsers, so I’d love to hear from someone who is. Why is there this discrepancy? Is one implementation more correct than the other?

Let me know what you think on Twitter or Hacker News.