Category Archives: programming

Example of a MongoCaseClassListField in Lift

Here’s a couple of hours days I’d like to get back: figuring out how to use the Lift Web Framework‘s oh-so-elusive and yet alluring MongoCaseClassLiftField  class. The idea seems just too wonderful to be true: you can store pretty much any object into a list, save it as a document in Mongo and get it back, pristine.

Of course, by “any object”, I mean any set of data you want to define. The objects have to be made in the Scala way: case classes that all inherit from a common regular class or trait.

In this example, I’m going to store different kinds of animals inside of a list.

You must override the  formats  variable in the way shown above, and you must set it INSIDE of the  MongoCaseClassListField  object, otherwise Very Bad Things will happen.

Now, you can do this:

Notice how, even though Dog and Cat have the same fields, Lift stores each object back to its proper type. This is because Lift has placed type hints inside of Mongo in extra “jsonClass” fields (you can override the name of the field if you need to):

So now, you can easily take advantage of Mongo’s document system and Scala’s type classes for pretty much free.

Why no dialog trees in Go, Jill!

My current side project is a text adventure tentatively called Go, Jill!. I want it to be a mostly-free roaming world with a main linear story where the player character, the titular Jill, will be able to talk about a lot of things to a lot of people. But one thing I don’t want is dialog trees.

Interactive story and branching

Dialog trees are very similar to the concept of branching stories. Usually, when people hear the term “interactive story” they think of branching story. It seems to make sense, right? How else is a story going to be interactive? In fact, there are many arguments that can be made both in favor of and against branching in interactive stories. Most of them were masterfully presented in the book Interactive Storytelling for Video Games.

In general, branching gives a better sense of agency, and that’s a good thing. But, as counterbalance to that, branching carries many downsides, some of which are:

  • It’s hard to write one good story, let alone several good ones.
  • Similarly, in a scripted story (as opposed to an improvised one), a good plot is usually made by planting narrative seeds that will bear fruit later. This is further complicated if the story branches.
  • A story should generally be surprising. If it branches, people will either be progressing at random or not be surprised.
  • Generally, the end of a story either presents a conclusion or a clearly purposeful sense of non-conclusiveness (like The Sopranos). If the story branches, one may always have the impression that the conclusion they saw is only one of many and that often feels wrong.

An ending being only one of many if fine if the game one plays is really set to make people construct their own narrative. Several role-playing games are made precisely to allow that. However, while they are clearly set in a narrative-like world, they don’t have an actual narrative in the generally accepted sense of the term. The ending the players arrive at simply did not exist before they played. It is not one of many pre-existing ones.

Dialog trees

Dialog trees are mostly a form of branching. Even if great care is placed in making sure that they never have negative consequences (like Ron Gilbert did) every time a choice is made it usually prevents another one. In a game of strategy that’s good, but in a narrative context, that’s bad. Furthermore, most dialog trees don’t take the care Ron Gilbert did and actively use dialog trees to alter the course of the story and branch it out.

And here, we have many problems. The first one is that characters usually know more about their own world than the player does. That’s something that very visible in David Cage’s work. A character can open a drawer and find a photo than makes him sad. The player had no way of knowing that the photo would be there, but the character did. At the end, such choices are just random to the player.

Dialog trees are just like that only more so. When we talk to someone in life, we usually have a good idea of what will offend them and what will make them happy. In games, it’s mostly luck. It’s actually worse than luck because the narration of the games forces it to be surprising, on an innocent-sounding phrase is unpredictably likely to turn out to be offensive to a character for the sake of drama. If it weren’t, the story would be boring.

Also, some of the choices offered in dialog trees might be a lot less subtle than what the player has in mind. Let’s say a character asks the player “Should I go talk to my friend about her husband being unfaithful?” and the possible choices are “Yes,” “No,” and “I don’t know, maybe.” And answer like “Yes” can lead to

  • “You totally should. You’d be a horrible friend if you didn’t.”
  • Or “Yeah. I think so. If you don’t and she found out you knew, she’s going to blame you for it.”
  • Or “If you want. I mean, it’s your decision.”
  • Or “Sure. I’ll go with you. I can’t wait to see the bitch go down!”
  • Or “Yes. Let’s go to the café that just opened downtown.”

Even if the whole first line is given, the conversation might end up having the player character say things that the player never expected.

This is randomness, and randomness and storytelling don’t go well together. This is one of the areas where it is particularly tough for story and gameplay to merge. As Jesse Schell is very found of teaching, randomness is generally good for games. He also likes to say that story is often good for games as well. For my part, I’ll add that randomness is usually terrible for stories.

My alternative

Let’s be sensible: for point-and-click games, dialog trees are still the best solution. But Go, Jill! is going to be a text adventure. For that medium, dialog trees can easily be a bad decision. It’s very important to use one’s medium as much as possible. And I have a blog post about just that topic. For a text adventure, I always found dialog trees to be clunky. A sort of haphazard interface inside of the main interface. Text adventures do not have a perfect interface, obviously, but throwing a dialog tree in the middle of it takes away their main attribute: free exploration or at least the illusion of it. If I’m playing a text adventure, I like to feel like I can type anything. I know that only a few words will actually work, but if I play along and the game is well made, that feeling can still hold.

Some people often like to say that text adventures are misleading because they claim the user can type anything in the input field but only very few of these commands will work. First of all, a list of suggested commands were usually distributed with games at the time. Second, all games restrict actions. Try and have Mario walk in front of a pipe in the 2D games! Get Nathan Drake to talk the people who are shooting at him into changing sides! Take a saw and try dividing up those Tetris tiles! Go ahead, do it! Games are restrictive by nature. Text adventures, like all games, try to use their limitations to be better from them, as opposed to artificially build clutter and pretend that those limitations do not exist.

What I intend to do with Go, Jill! is to encourage the players to have Jill ask about and tell about as many things as possible to the non-player characters, with a specific focus on relations between people. It’ll be a lot of content, but I’ll try to make it work. Again, this will mostly be about exploration. The main consequence to most of that content will be the way the player explores and understands the story. With some exceptions, the actual gameplay will be unaffected. So, for example, Jill will be able to ask character A what he thinks of his romantic partner’s relationship with his mother. This will not be a suggested topic, just not I’ll throw in there. If the player tries it, there will be a response.

The idea is that from one same mass of text, each player will be able to explore the parts of the story that they want at will. Progress will depend on them following the main story path, but it will truly be an interactive experience in the way most of the backstory and underlying relationships will only emerge if the player seeks them out.

There is no negative consequence to picking the wrong option because there is no option to pick. There is no frustration for getting to the wrong ending because there is only one ending. But if a player is not interested in an aspect of the backstory or a character’s opinion, they simply won’t have to pursue it. If, on the other hand they want to learn all they can about what most of the characters think about things, that will be available to them with no in-game downside.

This will definitely not be a game one plays to win but one that one plays to explore.

Heidegger, in retrospect

My last project at the ETC was Heidegger, at our Silicon Valley campus. The client was Electronic Arts and we were very lucky to be able to work with several people from the Dead Space team at Visceral. Our project was about finding a way to deduce player types though gameplay analytics and from there predict how likely people will be to like another game.

I worked on the interface with the very talented Anabelle Lee. She took care of aesthetics and I was in charge of functionality. I took that opportunity to improve my skills with the HTML 5 Canvas and jQuery. But, most of all, I got to experiment with Websockets and the fascinating Tornado web server, written in Python.

I also wrote and edited our promo video. I did not film it, though. That part went to other members of our team.

I would like to take the opportunity to thank Ben Medler, our fantastic client contact for just being so awesome.

Adding a toolbar to Frotz: harder than I expected

I’ve just spent a few hours trying to add a simple MFC toolbar to Frotz. I took a class on Visual C++ about ten years ago, and I think I’ll need to roll back and look into MFC again.

For the sake of clarity, MFC is the Microsoft Foundation Classes: it’s a set of object-oriented tools that create a high-level layer to use Windows functionalities like menus, button or, for example, toolbars. But using MFC is not obvious. At least it’s not to me. And Frotz for Windows is a fairly complex application. As much as Frotz itself is easy, even though I secretly expected it to be hard, Windows Frotz that I thought would be relatively straightforward is much more complex than I expected it to be.

What I must do now, is create a new MFC application with only the functionality I want to add to Frotz and then and only then add that functionality to Frotz. It feels a bit frustrating, but if I want to make this project useful, I think it will be necessary to step back, gather myself and then come back to the present point.

Main process loop found and other news

I’ve just been reading more of the Frotz code and I actually found the main process loop. It was much simpler than I anticipated. Actually, it delegates all the operations to an array of pointers to a function. In that way, it’s remarkably similar to an interpreted language, I think. Here’s the actual main loop:

One thing I’d like to figure out now is what the  finished  variable does. I’ve figured out it’s used to simulate interrupts, or at least I think it is.

This code links to the actual opcodes. They are stored in things like this:

This is an array of pointers to functions assigned in-line, which I find extremely cool. All the pointers are void to void because they take their arguments from and return their values into the Z-machine’s registry.

There will be more on this later in this blog, but even though these are op-codes, the ones I’m showing in the current blog post are actually directly related to interactive fiction: they move and look into game objects. This too is extremely cool.

So the main loop goes down the op codes in the game file, and executes the matching function based on which number in the array shown above or one like it a pointer to the function that executes the functionality of the op-code actually is. Again, there will be more about these functions later.

Finally, about two weeks ago, when I could not get Frotz for Windows to compile at all, I bought a boxed version of Microsoft Visual C++ on line to be physically delivered to the ETC-SV. It has still not arrived. I’m glad that I ended up not relying on it after all. I found a copy of Visual C++ 2003 online, as explained earlier in this blog. Then, of course, David Kinder told me he himself he used Visual Studio 2008…

UPDATE: The variable finished  was not, as I first suspected, about interrupts. It’s the stack of functions going on inside the Z-machine! So when it reaches zero, it means… I still don’t know but I’m going to figure it out!

UPDATE 2: The macro CODE_BYTE  expands to:

This is obviously how the Z-machine steps through its memory. Though I’m still not completely sure of what all of the pointers mean. There’s *pcp , *zmp , which I think stands for “Z memory” and a few others. I’ll have to figure that out as well.


Frotz re-compiled (yay…)

I got stuck for an hour on one final problem I could not solve, even with Shaveen and Mohit’s help: an include file refused to be included. I could import it by explicitly giving its path, but the compiler refused to find it via the include path. In the end, I had to resort to explicitly adding the path. It’s very strange. I am baffled.

But I can now compile Frotz on my ETC computer as well as at home.

During the process, Mohit pointed out to me how to change Windows environment variables. It may be that we changed the right one but that Windows needs to be restarted for the changes to be effective. Still, addding the directory to the Visual C++ search path really should have been enough. I am slightly angry about the absurdity of it all.


I get more help from David Kinder

I got a new response from David Kind, the developer of Windows Frotz on the IF forum. I learned something that I consider important, mostly because I would never have figured it out without this project: the

  •  standard, built-in Windows unzip function can corrupt non-ASCII characters. (That was really a revelation.)

He also told me about a fix he added to the code to adapt to a change in the PNG API. The fix changes two lines of code, and one of them, I had actually figured out on my own last weekend. This may all be very minor, but really, the more problems I run into, the more I get to learn the messy parts of Windows development, the part you can’t learn in books.

It really time now for me to get working on actually looking into the virtual machine itself. I should get somewhere with that this weekend.

Compilation on ETC machine almost smooth

I really am glad in a way that I’ve had to many problems with Visual Studio. By actually trying to solve them, I’ve become a lot more comfortable with it.

In order to make good use of my newfound skills and in order to work on my project at EA, I decided yesterday to compile Windows Frotz on my ETC machine.

I’d like to reiterate at this stage that the regular Frotz is Unix-only, which means that it won’t natively work on Windows, unlike Linux and OS X. It will probably work on Windows with Cygwin but I don’t want to get into that at this stage.

I installed the version of Visual Studio that David Kinder, the developer of Windows Frotz recommended, Visual Studio 2008, after downloading it through Carnegie Mellon’s Dreamspark program. The project opened and, by then, I knew how to deal with the missing libraries. It really all went very smoothly until Visual Studio complained about missing a DirectX header, specifically dsound.h.

Based on what I found on the web, I had to install the DirectX Software Development Kit, or DXSDK. That proved to be very problematic. I need two header files that probably total 10 kilobytes in size, but Microsoft will only provide those as part of the whole DXSDK package, which is a 550 megabyte installation file that installs to 1.2 gigabyte package. And the installation process takes a while. And the installation invariably failed until I restarted my computer today.

I’m not near my ETC computer now, but I suppose that part of the compilation will resolve once I add the DirectX directory to my include path in Visual Studio.

What puzzles me greatly here, however, is that I don’t remember ever doing that on my Mac. And it’s not as if the DirectX SDK could have remained from a previous project: it was a completely fresh install of Windows. It’s not really a huge issue and I now have better understanding of how the DirectX SDK works. But it still bothers me that I can’t explain it.

SUCCESS! Frotz compiled!

I figured out that there was a file in the ScaleGfx package that I had never used: ScaleGfx.def . I figured out where it went (after several failed attempts) and I got a ScaleGfx.lib. Then Frotz compiled. I tried to open it, but then it asked for a ScaleGfx.dll. I spent quite a bit of time trying to compile that until I realized that the DLL had to be in the final distributed version of Frotz. There is really no need for me to compile it and doing so would just be inefficient. So I did just that: I got a final copy of Frotz, copied the DLL to the one I had compiled and IT WORKED! I have a working version of Frotz for Windows that I can most likely change! That part took a lot longer than I expected, but I really, strongly feel that I’ve learned a lot in the process. And it’s the sort of thing that you can’t learn in classes and books and tutorials: those are usually structured in such a way that you avoid precisely the problems that I had to confront here. I’m going to call it quits for today, but at last I have a base code I can change. Next, I’ll probably go back to the original Frotz and try to get some specific information out of it.

UPDATE: Yesterday, David Kent, the developer of Windows Frotz had in fact replied to my forum post. It’s great to know that he can be reached! I’ll try not to bother him too much, though.

A static library turns out to be a static issue

One library is really getting in way. It took me a while to figure out that it’s by the author of Frotz for Windows, David Kinder. The project and it’s set up now wants to import it as a static library. It didn’t actually say that, of course. It just said it expected a file with a “.lib” extension. But, based on what I’ve looked up, that means it’s expecting a static library, which is an unlinked compiled file.

It took me a while (again) but I managed to figure out how to compile that. The problem is that nothing in that file is public. That means that the compiler can include the file but, from the compier’s point of view, the file in empty. I’ve tried to make elements in it public, but the compiler does not seem to see that. All the web pages I can find on the issue are for Visual Studio 2008 or later.

I’m going to email Emily Short and ask her to give me David Kinder’s email. It think, at this point, it is necessary that I do that.