Worlds Colliding. Part II

Back in March, I wrote a post entitled Worlds Colliding explaining the failure of Google Buzz as a failure to understand the fundamental structure of complex networks.

Buzz received a large amount of criticism for automatically adding the most contacted people from your inbox to your Buzz follower list. My post explained that because individuals in social network are a member of many social contexts (family, work, friends, etc), nodes from all of these to a single list would cause these contexts to collide (e.g. adding both your wife and your (no longer) secret mistress to your list of followers).

The last couple of days, the following talk (from July 1st) by Paul Adams who is a User Experience Researcher at Google has been very visible on the interwebs.

From the looks of it, the good people at the Googleplex have either been reading my blog and the accompanying scientific paper and are scrambling to keep up (I consider this scenario highly unlikely) or, the User Experience Group at Google was never in touch with the group behind Buzz.

Let me repeat that last part for dramatic effect: the User Experience Group at Google was never in touch with the group behind Buzz. The knowledge about pervasive overlap and overlapping communities was present within Google, but never diffused to their initial social networking attempt. So the failure of Buzz was in some sense due to separate worlds within Google not communicating properly. That strikes me as textbook case of tragic irony.

Update, July 15th

I’ve included YY‘s recent slides from the New Frontiers in Complex Networks conference as a quick intro to our thinking regarding pervasive overlap.

The proper reference is Link communities reveal multiscale complexity in networks. Nature (2010), doi:10.1038/nature09182.

Mood, twitter, and the new shape of America

Twitter is a gigantic repository for our collective state of mind.

Every second, thousands of tweets reveal what everybody and their mother had for lunch, what Justin Bieber is up to, or what magnificent link you should be checking out right now. Individually, each tweet is mostly interesting to friends/fans of the tweeter, but taken together they add up to something more.

In analogy to individual neurons firing together to add up to the human consciousness, the billions of tweets have meaningful macro-states that contain information about the whole system rather than the individual tweeters. But we need to do a little data mining to extract meaningful information about these states, to expose our collective states of mind.

As a proof-of-concept we’ve1 been studying the mood2 of all of the public tweets. While there are many services that will allow you to study the mood of your own tweets (and also an neat little DIY project to show you the global average of twitter), much less effort has gone into studying how the mood breaks down according to geography. Below, I show a brand new video displaying the pulsating 24-hour twitter mood cycle of the United States (I’ll explain just what you’re looking at, in the following).

In the video, green corresponds to a happy mood and red corresponds to a grumpier state of mind. The area of each state is scaled according to the number of tweets originating in that state. Note how the East Coast is consistently 3 hours ahead of the West Coast, so when we’re sleeping in Boston, the Californians are tweeting away. It’s also interesting that better weather seems to make you happier (or rather, that better weather is correlated with happier tweets): Florida and California seems to be consistently in a better mood than the remaining US. Also note how New Mexico and Delaware behave very differently from their neighbors. Full results, individual maps, and a high-res poster can be found on the dedicated Twitter Mood website.

How to construct the mood map

Since many twitter users list their location, we’ve assigned every tweet in our (massive) database to a US county and extracted their mood. This allows us to average over tweets and plot the mood of the US as a function of geography (and time). However, since the US is unevenly populated, the resulting maps are boring since only a few counties (the centers of cities) contain most of the tweets (not too many tweets in Ellsworth, Nebraska yet).

Luckily, brilliant people have come up with a cool way of solving this problem using a technique called density equalizing maps3. (or cartograms) The idea here is simple: warp the map in such a way that certain features of shape are conserved, but in such a way that the (population) density becomes the same everywhere. The resulting maps look like something from an alternate universe and allow us to show the US mood much more clearly.

Notes

  1. The twittermood project members are Alan Mislove, YY Ahn, JP Onnela, Niels Rosenquist, and undersigned.
  2. For a deeper explanation of how we evaluate the mood of tweets, see the Twitter Mood website.
  3. An easily accessible explanation of the density equalizing maps, is posted on the Twitter Mood website.

Erdös Number

The scientific version of the Bacon number is the Erdös number. Via a post on Finn Nielsen’s blog, I learned that i have a reasonably low Erdös number – three. (I also learned that Finn is one of the few people with a finite Erdös-Bacon number). The reason for both Finn’s and my own low Erdös number, is that my PhD advisor Lars Kai Hansen has co-authored a (highly cited) paper with Peter Salamon who has a bacon number of one. The links are:

  • P. Salamon and P. Erdös. The Solution to a Problem of Grünbaum, Canadian Mathematical Bulletin, 31: 129-138 (1988).
  • L.K. Hansen and P. Salamon. Neural Network Ensembles, I.E.E.E. Transactions on Pattern Analysis and Machine Intelligence, 12: 993-1001 (1990).
  • S. Lehmann, M. Schwartz, L.K.Hansen. Biclique communities. Physical Review E 78:016108 (2008).

With respect to the Erdös-Bacon number, I could make the case that I should have a number of four. The reason is that I actually appear in the documentary (it’s just an uncredited half-second shot of me sitting at my computer) Connected – The power of six degrees, which features my ex-boss and renowned scientist Albert-Laszlo Barabási. Here’s the trailer:

But since I don’t appear on IMDb, I guess it doesn’t really count…