Tag Archives: python

Heidegger, in retrospect

My last project at the ETC was Heidegger, at our Silicon Valley campus. The client was Electronic Arts and we were very lucky to be able to work with several people from the Dead Space team at Visceral. Our project was about finding a way to deduce player types though gameplay analytics and from there predict how likely people will be to like another game.

I worked on the interface with the very talented Anabelle Lee. She took care of aesthetics and I was in charge of functionality. I took that opportunity to improve my skills with the HTML 5 Canvas and jQuery. But, most of all, I got to experiment with Websockets and the fascinating Tornado web server, written in Python.

I also wrote and edited our promo video. I did not film it, though. That part went to other members of our team.

I would like to take the opportunity to thank Ben Medler, our fantastic client contact for just being so awesome.

A web server optimized for websockets

As part of my OCCO project, I made a web server that makes the use of websockets easy. I din’t make it from scratch, of course, I made it with Tornado. Tornado is a very lightweight, flexible and extensible web server written in Python. It makes the use of websockets trivial.

My whole code is on GitHub:

https://github.com/eje211/heidegger

This site is a mockup of a modification of the interface of Electronic Arts’ Origin game distribution system. My team added a panel that informs the player of their Bartle player type. My websocket feature allows me to change the player type in real time during presentations. This was really made to have a good reason to use websockets more than anything else. The Javascript part of websockets never changes. It’s the part that’s on the server that’s often problematic. In this case, I have a page with the player type chart and one or more pages with a remote controller to change it. The code to handle this is absolutely trivial:

I keep a list for each of the two types of websocket handlers and keep them updated as the sockets are connected and disconnect. And, as message come, I just pass them along to the relevant pages. That’s really all there is to it.

Tornado also requires me to deal with the delivery of files manually. That was new to me, coming from Apache, Lift and Zope. It was interesting and made me aware of many details.

Why I love Scala – Part 1: Boxes

I Love ScalaThe Scala programming language logo

Here’s a small example of why I love Scala. First of all, if you don’t know what Scala is, it’s a relatively recent language that compiles to Java binaries that can be run on the Java Virtual Machine. It keeps the best from Java: the JVM is now very fast, very efficient and runs on pretty much every device you could want. Plus, all Java classes are also Scala classes. So as long as something has been ported to Java, you can use it in Scala without waiting for someone to port it. The language itself hardly has anything to do with Java. Unlike Java, Scala is extremely concise and makes very high and frequent use of implicit typing and implicit declarations. In Scala, when

you don’t need to tell Scala that a in an Int. Scala will figure it out for you. It is strictly statically typed, but you don’t need to state the obvious about types. Just program what you need.

The Problem

In this post, I’m going to talk about a programming issue that Scala handles, I think, better than any programming language I’ve ever use: the special unassigned case.

For example, let’s say that you have a string, for example, "abc". And you want to find the first occurrence of a specific character in that string. In Java, for example, you’d use indexOf:

this would evaluate to 2. The index of "a", would evaluate to 0. And that’s bad. That’s very bad. Because

is false, even though a definitely has an index. In most languages, 0 is false, even tough it doesn’t always mean false. And how about the index of "w"?

Java and C++ use the special value -1 as a way to indicate that the index was not found. C doesn’t have an equivalent for indexOf, as it doesn’t even technically have strings but char arrays. But this clever little bit of code: <http://cboard.cprogramming.com/c-programming/100548-strpos-function-c.html> also uses -1 as the special “not found” value.

There are many problems with using -1 as meaning “not found”. First of all, -1 simply does not actually mean “not found.” Second, unlike 0, -1 evaluates to true. All in all, this is usable, but not ideal.

Let’s look at the haphazard mess that is PHP. PHP gives us strpos(). That function will return 2 for strpos(“abc”, “c”) and 0 for strpos("abc", "a"). And in PHP, 0 will also evaluate to false. But for strpos("abc", "w"), it will return a boolean false, which is equal to 0, but not identical. That means that even though all the programming teachers in the world have been, rightly, encouraging their students not write code like

PHP actually requires us to write

And then re-evaluate the whole thing or save the result in the first place. That’s PHP-style messiness.

Python gives us a much cleaner, safer, more elegant and semantically accurate way of dealing with the whole matter. Each string as a built-in index function:

If an unknown index is found, Python will raise an exception, which is perfectly fine in Python. Pythonic code means that it’s better to ask for forgiveness than permission, unlike the C, C++ and Java principle of “look before you leap.” So in Python, you’d do:

Ta da! If you really want to get a -1 when the value is not found, you can always use the find() method instead of index() that will do just that: return -1 if the substring is not found. But if you do that, it probably means that you just don’t understand Python. Ugh.

A Solution

The question now is: how does Scala do it? Well, here my argument sort of falls apart at first. Scala tries not to add too much to what Java already does well, this means it doesn’t implement strings at all and just uses Java’s Strings. By default, Scala uses Java’s bad way of handling that example. Let’s fix this. Let’s create a new class, a SuperString, that will actually deal with this situation in a more idiomatic way. (We’ll see the details on how to do that later.) Our new strings will have a method called find that will do things the Scala way!

What does that return? Well, since SuperString does things the Scala way, it won’t in fact return an Int. Scala has Ints, of course, but Ints aren’t really what we need here, because, precisely, there is a special case where we don’t get a result at all.

Scala provides us with a very powerful tool for that situation: the Option. David Pollack, creator of Scala’s killer app, the Lift Web Framework, thought that Options were great but that it was possible to do even better, so he did. He re-did his own Option class: the Box.

A Box can contain one of three values.

If something went seriously wrong, the Box will contain the special Failure object. This is not for special cases like when a substring is not found, or Python’s ValueError or KeyError exceptions. This is for when something really bad happens.

If the Box has no value in it, it contains the special Empty object.

If the Box does have something in it, it contains the special Full object. Boxes and Full are both generic types. Unlike C++, C# and Java, where type arguments are surrounded by angle brackets, in Scala, generic arguments are surrounded by square brackets. So in our example, the find() method will return a Box[Int], that can contain a Failure, an Empty or a Full[Int].

So now that we have our Box[Int], what can we do with it? As it turns out, quite a lot. First of all, we can check if it’s empty. If result is the Box[int]

will be true if the Box contains an Empty or a Failure, false otherwise.

will be true if the Box contains a Full, false otherwise.

But you usually won’t need to use those. You can, in theory, open a Box with open_!

But again, what’s the point of having a Box if you’re just going to open it? The very fact that there is an exclamation point in the name is meant to warn you that you’d better not be using it unless you have a good reason. If the Box contains anything other than Full, it’s going to throw an exception anyway when it’s opened. And if you check to see if it’s defined first, it’s just as clumsy as the C++, C# and Java ways. No. One of the things you’re likely want to do with the Box, is, say, for example, map it. As a functional language, Scala is very big on mapping.

What that does is that, if the Box is Full, it takes what’s inside of it, puts it in a variable, here r, lets you use it in a function, lambda like here, or otherwise, and then re-package it in a Box. If the original Box contained an Empty or a Failure, the new one will too. It’s that simple.

But wait! Scala has other cool tricks going on here. result is a Box[Int], right? So r has to be an Int. It can’t be anything else! No point in telling Scala about it. Scala can figure it out. It’s actually considered good practice to let Scala deal with it.

But wait! We’re still not done! We have one argument and we’re using it once. In that case, we don’t even have to name it. If all your arguments are only used once and in the order they were passed, Scala can invoke them automatically with the default variable _. So really, all we need here is:

At this point, this no longer returns a Box[Int] but a Box[String]. Any type of object can be Boxed, including user-defined objects, of course.

Now, if we need to do something with our String, we will at some point need to open the Box. But what if our String was not found and our Box is Empty? The Box lets us deal with that with the special operator openOr, which takes as an argument a default value in case the Box is Empty:

But wait! It’s not all! Since Scala is essentially a functional language, it doesn’t need dots and parentheses for method invocation that only take one argument. So, really, all we need here is:

A few parentheses are remaining. Let’s get rid of them by using format instead of those messy plus signs:

And finally:

This returns either something like The character was at position 0. or The character was not found. Perfect. Clear. Simple.

But we can still do more! By which I mean less, in the long term.

I Really Shouldn’t Have to Say This

This only worked because we used the find method on our SuperString.

Just to be explicit, the class SuperString is nothing more than:

Oops. This class does not actually use Boxes. Let’s add them.

(Yes, in Scala, classes take arguments. That means that a class declaration is also part of the constructor. Scala likes concision.)

Of course, a String can easily be converted to a SuperString with a function like:

So, what this would give us is:

  • A class called SuperString that has a find method,
  • A String class with which we could probably make good use of our find method, but that we can’t extend because it’s final,
  • A method that can totally convert an object of one class (String) into an object of the other (SuperString).

Couldn’t Scala figure out on its own how to combine these three things? Well, yes and no. Yes, it could but that would be messy. What if we don’t want it to? Just because we can convert one type into another doesn’t mean we always want to. So what if we indicated that we do want to? Scala does let us tell it that with the almost magical implicit keyword. Let’s make that function from before implicit:

So now, code like:

will return a Full(0) and

will return an Empty. When scala fails to find a find method in the String, it tries to look for another type of object that has a find method and sees if there is an implicit way it can do the conversion. If it can, it does. And so, it looks like we’ve added a method to a final class, when in fact we’ve done nothing of the sort. We could also remove the dot and parentheses for the method call. Super cool.

The Final Line

So to get our index and format it and provide a default message if it fails, all we have to do now is:

This will evaluate to:

Whereas this:

evaluates to:

This is fully statically typed, extremely concise, clear and powerful. We don’t need a special case to check if the text is not found, it’s all handled by the Box. We don’t need to specify a default output for the special case, the Box handle that as well.

Once you can wrap your head around the basic concepts, it’s all smooth sailing.

Final Note

This code uses libraries from the Lift web framework. To get Scala working with them, the easiest thing to do is to run it from the Lift console. Download Lift. Make sure you have Java installed. You don’t actually need Scala. Lift will get its own version of Scala for you. From a Lift project’s main directory, run:

Then, from the Scala console, run:

You’re set.

Reminiscing about Buzz and Wave at the birth of Google+

It’s not a secret that I love Google. I love their original web search service, I love Gmail, I love Google Docs, I love Google Maps, I love that they use Linux, I love that they use Python, I love their products.

Still, like more or less everyone, I hated Buzz and I got very disillusioned by Wave very quickly; though I probably held on to some hope about Wave a bit after most people had given up on it.

It would be very easy at this stage to make a list of why I think Google Wave went meh and Google Buzz went bust. Today, as Google is trying to keep their new service, Google+, from going down in flames the way Buzz and Wave did, that list is what I’m shamelessly about to do. That said, it’s not really a list: all the items on it can be rephrased as, “The problem was that it was more about Google than about its users.”

Buzz

First, Buzz. It never bothered me that Buzz disclosed users’ locations or that it peeked into their contact lists. I never let Google know about my location at the time and I never actually went so far as to give Buzz a serious try. And here is why: what bugged me was that Buzz wanted me to change all of my friends or maybe even get new one. Let me explain. I have one general-purpose microblogging system I use: Twitter, like pretty much everyone. There are two main places where my Tweets show up: Twitter and Facebook. Actually, when people see my Tweets, most of the time, they see them on Facebook.

If Buzz had seamlessly let me pick up Buzzes from Twitter or let me send Buzzes to Twitter and Facebook, I would probably have given it serious consideration. But that’s not how it went. Buzz was, at its launch, a completely independent service that required everyone who used it to get everyone they knew to switch to it. According to Google, if you wanted to stay friends with me, you’d have to give up Facebook and Twitter for Buzz. But why would I inconvenience everyone I know plus myself in that way when not switching to Buzz and keeping Facebook and Twitter was so much easier?

But Buzz didn’t stop there. It nagged. Oh, how it nagged! My main email interface is web-based Gmail. Google put a link to Buzz in the main Gmail menu, as if it were part of Gmail. But following the link opened a new window making it a double annoyance: adding a link where it did not belong and opening a window unnecessarily. Then, they did the same with iOS interface of Gmail. The there is very little available space at the top of the screen of the iPhone and Google put not one but two links to Buzz there: a text link and a horrible, nagging little Buzz logo. I thought of leaving Gmail. Then, when users were trying to read their mail, a nag screen prevented them from using Gmail in order to urge them to try Buzz. I almost left Gmail at that point.

Wave

Wave was a sad, similar story. It reminded me of how brilliant Gmail was. Gmail added so much to the existing concept of email and did it all by staying discreet and in the background. Wave could, technically, have done the same thing. You could have sent a wave to someone on Google and if they only had Gmail, they would get it as a an email and if they also have Wave, they would get it as a wave. Sure, there might have been a few issues, mostly that Wave-specific features would not have gone through by email: the then famous Wave apps would not have worked and the conversation could not have continued as a Wave, but it would have been smooth and easy to use and, most of all, non-intrusive. It would not have mattered whether people had Wave or not to just use it and it would have encouraged them to switch to Wave if they had received a wave as an email. The focus would have been the users, the people, and not Google.

Instead, just like with Buzz, in order for Wave to work, everyone had to stop using email and start using Wave. This, of course, did not happen. If I know someone, let’s call him E Maginary, and I’m not sure if E has Wave, it’s much easier to send him an email. If E does have Wave and I send him a wave, that means that for me he has to check his waves and for everyone else he has to check his email. Even if more than one person sends him waves, it’s still an annoying extra step until more people use waves than email. And that annoyance is exactly what makes such a tipping point impossible to reach. So E is going to find it annoying if I send him a wave. And honestly I sympathize with him. Had it been possible to receive a wave as an email, this would not have been the case.

Plus

Now there is a major difference between Wave and Buzz. Buzz had pretty much no reason to exist rather than profit Google. It tried to become popular almost exclusively by nagging Google’s users. Wave, on the other hand, was a fantastic product with potentially unlimited potential that got destroyed by putting Google first and Google’s users last. I won’t hide the fact that I’m still angry about that.

Some great ideas that were made unusable by a few big flaws can be fixed and changed into amazing stuff, even after a long time has passed. Apple users know that. The G4 Cube led to the Mac Mini; the Newton led to the iPhone; and what is probably still Steve Jobs greatest technical accomplishment, NeXTSTEP, led in great part to the creation of Mac OS X. For some of these, particularly the Newton and the iPhone, I still think that despite their difference in success, the first one is more worthy of praise than its successor. But that’s just me.

Maybe Google+ will have something genuinely useful to offer, maybe not. My main hope for it is not really that it’s good (I’m not part of Google, so I don’t really care) but that if it is good, its greatness is be unencumbered and unrestrained by anything around it.