RSS Feed IDS on YouTube IDS on Facebook IDS on YouTube

Several months ago, Steve and I attended the launch party of the Experience Music Project’s Indie Game Revolution expo, a year long project to display how independent game development has shaped and formed a medium that so many of us grew up with.  Among many titles which we hadn’t yet heard of were some now household names of the indie world, like Nidhogg.  Nintendo co-sponsoring an event showcasing independent developers and having their own booth displaying no less than a dozen titles was an idea almost as alien to the industry a year ago as virtual reality was three years ago.  Yet, here we are and the community is better for it.

One particular title caught my eye as we explored the different works on display.  I love the idea of interactive media that bends the barrier of what we consider a “game” to be, and I think Panoramical in particular is a great example of what can be done when we ask, “What if, instead of making a ‘game’ with a win condition, we just try to enable a user to build an experience in our world?”  Instead of giving the player a controller or touchscreen, Panoramical enables players to experience the game with a soundboard.  Sliders, volume knobs, etc. were all functional in their own ways.  What a great idea!  A diverse group of crowd seemed to be enjoying it, and, just as importantly, they seemed to be enjoying the creations of each other.


Panoramical by Fernando Ramallo & David Kanaga. Go check it out.

Steve and I have had numerous discussions of what makes a game qualify as a by-the-book game.  The only conclusion we’ve come to is that a real game must have a success and fail condition.  By that definition, Panoramical and other works like it don’t fall under the “game” category, but that doesn’t diminish their worth nor gratitude to have them included in our medium.  Panoramical falls in this grey area between a film and a video game, like Dear Esther and others.  They tell a story or allow us to explore a world, and to take on the burden of allowing users to visually explore a soundscape is no small task.

Exploring what it is to be a “game” vs. another medium is a topic for another post.  This is about how we interact with games.  Controls have always been interesting to me.  I loved using a joystick to play Descent growing up.  A first person flight shooter in a zero-G environment was perfect for that method.  However, remove a dimension, and it becomes unwieldy.  In Doom or Counter-Strike, a joystick is about as useful as a Guitar Hero/Rock Band drum set.  Yet, they both have their target applications and are very effective at it.  Otherwise, people wouldn’t pay hundreds for a joystick or thousands to build a multi-monitor flight simulation station.

Controls are evolving right before our very eyes right now.  Valve has their new Steam controller with some very interesting ideas.  Nintendo pushed the envelope with the Wii stick and nunchuck to limited success.  Microsoft removed the physical piece alltogether with the Kinect and now seems to be taking it a step further with Hololens.  However, we need to be asking ourselves why.  To what end do these new interactions allow us to convey feelings to our users?

Kinect Harry Potter

Kinect Harry Potter. A useful control metaphor? I think not.

Controls are merely a method of indirection.  They enable a player to interact with the game.  Good controls should be transparent and make the game feel like an extension of the player’s will.  This is very difficult to achieve in a medium where dragons, spaceships, 4D geometry, and altering the space-time continuum are par for the course.  A good controller for a particular game or genre does not and most likely should not be exactly what you would use to perform the tasks in game.  For example, electronic boxing gloves for Street Fighter are a terrible idea.  No one wants to actually get on a skateboard in front of their TV to play Tony Hawk.  It’s not practical.

This leads us to understand why controllers are the way they are.  It’s because games enable us to feel like we’re doing the impossible.  The Marine in Doom can run at over 90mph.  Ryu in Street Fighter can jump over three times his own height.  When the assumptions of reality are broken, so must our assumptions of how to interact in reality.  So, we have a keyboard and mouse to navigate treacherous corridors at 90mph with lightning reflexes.  We have a joystick and 6 buttons to allow us to input complex commands in tens to hundreds of milliseconds instead of seconds and with extreme precision.

Controllers and our controls have evolved to meet the needs of the game itself, and this is a lesson we must not forget moving forward.  The metaphor of the game must not be broken to meet a contrived control metaphor.  This is where Kinect ultimately failed.  You can’t simulate a sword fight if users aren’t holding swords or feel the impact.  It feels less realistic than if they were doing it via controller.  This is such an important point that I’m going to repeat it in another way.  Controls that attempt to parallel reality and can’t convey the subtlety are immeasurably worse than those that ignore reality and focus on making the game an extension of the player’s mind.  Button layout doesn’t matter if players aren’t thinking “what button do I press?” and are thinking “how can I get over there?” or “how can I solve this problem?”


A counterpoint might be Guitar Hero.  A custom controller for a game had been attempted numerous times, but this one deserves special attention.  Next time you play (or go to your local thrift store and buy it for $10, because, trust me, it’s there), hold the controller up to the screen.  The buttons exactly match what’s on the screen and you strum every time they cross the bottom threshold.  It’s so simple that a 9 year old can figure it out, and thousands did.  Did you ever notice that the controller made a satisfying click when you strum it?  It’s because it was made with Cherry MX switches, the very same that have been in old IBM keyboards since the 80s.  Everything was calculated to make you feel like a rock star and, in that, Guitar Hero succeeds exactly where Kinect fails:  the controller is a perfect extension of the metaphor of the game itself.  In music, if you miss a note, just keep going.  Screw up too much and you’re booed off stage.  It’s an elegant solution because the controller was designed with the game.  Kinect was designed, then games were made to match the metaphor, and the disconnect shows.

As we evolve forward with new methods of indirection, we need to keep in mind that our games must be a projection of the player’s desires.  If we fail at this, then the game itself is a failure.  There is nothing worse than a player being removed from the experience because controller doesn’t allow them to quickly perform what they want within the rules of your constructed world.  Professional Starcraft players can control entire armies of dozens of units at over 400 actions per minute using a keyboard and mouse.  Don’t try to give those players a touch surface because it’s how actual military commanders coordinate squad movements in battle.  It will fail and the game, mechanics and story be damned, will be a waste of good ideas.  Afford the player the tools for success in an unrealistic world and allow them to exploit it and they will feel successful.  Now, I’m going to go play Super Meat Boy, because the controls are perfect.

More Reading:  Extra Credits:  Kinect Disconnect

Every developer seems to have that bug. You know, THAT bug. The elusive one that only seems to creep up during playtesting or while testing another feature? This isn’t specific to game development either. Web service developers or even operating system kernel developers have all seen, at least once in their careers, a bug that only seems to pop up when they aren’t looking for it. And it’s not just bad; it’s breaking. Debugging something you can’t figure out how to reproduce is an exercise in balancing sanity with sobriety.

With Minecart Madness, we had the Super Jump bug. We coined that name because the player would jump incredibly high for seemingly no reason. And it didn’t always happen. And we couldn’t figure out why. But it came up when we were debugging issues with track placement. It came up with we were debugging issues with obstacles on the track. It came up when we were figuring out how to handle tunneling. It kept happening. Why?

Filing a Good Bug

Super Jump Bug filing into IDS bug database

Super Jump Bug filing into our bug database

By profession, I’m a tester. My job is to break things and tell you why it’s broken. A really good bug filing should contain the following information:

  • First version where the bug was reproducible
  • Last version where the bug was not reproducible
  • Clear, concise reproduction steps with the minimum steps needed to cause bug
  • Any special machine settings (resolution dependence, high/low memory situation, specific phone, etc.)
  • Callstack if it’s a crash, but can also be useful if it’s not to show exactly where in code the bug is happening
  • Severity, i.e. how breaking of a bug it is
  • Extra manifestations, i.e. does this happen in all situations? For example, if your cross-platform Unity game is crashing on the iPhone, does it also crash in the same situation on Android?

With that in mind, let’s look back at this bug filing. Of the seven tenets listed above, this contains exactly zero. Each of these useful chunks of information rely on a single other event to occur: consistent reproduction. We couldn’t figure out how to make it happen. It just seemed to happen sometimes. Hence, the repro steps above.

What is “Breaking?”

As with all bugs, the Super Jump Bug had to go through a triage process where we decide if the bug is breaking enough to warrant fixing. These discussions usually involve expected amount of time to fix, expected degradation of the user experience, frequency of occurrences, and risk of fix. For the Super Jump Bug, we didn’t know the fix and we’d already attempted to reproduce it and fix it unsuccessfully. Through playtests, users would hit this bug as well.  When a user would hit the bug, he/she would notice immediately and ask what happened. We told those users that we were aware of it and not to worry. We were worried. The feeling of having to downplay a bug you know is breaking and can’t solve is crushing. With that in mind, we did not “Won’t Fix” this bug, if only to satisfy ourselves of knowing we solved it.

Procrastination as a Tool

Like most developers, we decided to just let this bug sit. And sit. And sit. Sometimes, as we’re all aware, one of two things can happen to bugs we let sit. First, the bug “magically” fixes itself. You change some specific values for core mechanics like jumping or movement, increase or decrease your net resolution, and suddenly, the bug isn’t there. It most likely is, but it is being masked or mitigated by your change. A bug that users can’t see, for an indie dev, is not a bug. We don’t have time to make things nice. We have to ship. The second thing that can happen while procrastinating is a revelation. A sudden moment of clarity. For a small moment in time, you become Neo. You can see the code and you realize the bug without ever thinking about it. That is how we solved the Super Jump Bug.

What is Jumping?

All mechanics in games need to feel like an extension to the players mind. Never has this been more true than in touch based mechanics. We touch the character and we want him to jump as we imagine in our minds, run as fast or slow as we picture, or target exactly where we point. Curiously, our minds don’t exactly imagine reality based physics. Not until high school physics classes do we realize that our assumptions about how things move are often wrong. But, this is a game, so we are expected to break with reality. Indeed, jumping should feel as the user would expect, not as it would be on Earth. Therefore, our jump algorithm doesn’t function as a NBA player leaping on the court does.

We break jumping into two parts. First, the initial hop. This happens with just a single touch and is the minimum jump height. This allows for targeted little jumps to cover the smallest of crevices. After that, for a specific amount of frames, we provide an additional upward velocity until either the user releases his/her finger from the screen or our acceptable amount of frames are exhausted. This provides the aforementioned feeling of the character on screen being an extension of the player’s finger, and therefore, mind. This creates a jumping experience that is fun and satisfying. More importantly, though, it makes the player feel liable for all successes and failures regarding hit and missed jumps.

What was the bug?

The code for the jump looked like this in revision 160:

velocity.Y -= _amountTouches == 1 ? HopHeight : fUpwardJumpAcceleration;

Revision 161, after the bug was fixed, contained the following instead:

velocity.Y = _amountTouches == 1 ? -HopHeight : _velocity.Y - fUpwardJumpAcceleration;

Of course, you see the changes, but do you see the bug? Hint: the bug is not on this line at all. To understand the bug, we need to know that, after landing, we reset _amountTouches to 0 in the Minecart’s update method. By doing this in the subsequent update, the Minecart is effectively running a frame behind on determining if it is in the air or on the ground. It goes to follow that a second HopHeight jump can occur.

Reproducing a Bug After Fixing It

Often, we determine a fix for a bug before the bug can be accurately reproduced. Only after conjuring a fix can we deduce a sequence of valid repro steps. For this bug, based on the fix in the earlier revision, the repro steps were as follows:

  1. Jump like any user would, but do not remove finger
  2. Land, keeping finger on screen
  3. Jump again by holding finger (we wait for touch events, never the absence)
  4. Repeat 3-4 times.

Step 3 is by design. We will never not let a user jump while on the ground. After 3-4 iterations of these steps, a large double jump would occur. Every. Time.

Learning and Moving Forward

The Super Jump Bug was a single bug with a single line fix affecting only certain users who happened to touch the screen for long enough in edge cases. However, it highlights that debugging may not be a scientific process. We can see that, because we are changing variables a full update behind, using the built-in debugger wouldn’t have gotten us closer. We tried. But, you can’t hold the screen AND hit break points in the emulator. In fact, reliance on the given tools was a hindrance to our progress for bug #107, the Super Jump Bug.

Brian Kernighan, one of the inventors of C, once wrote, “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” Intimate familiarity with each and every line of your code is paramount to shipping a quality product. Copy/paste is not a design pattern and will lead to bugs. We can confidently say that if we had borrowed another jump algorithm, this would have never gotten fixed. Code simply. Document thoroughly. Be patient. The product and your users will thank you.

A few months ago, Justin and I began investigating ways to make our next game (in Unity3D) look great. We knew we wanted to stick with 2D art, but didn’t want to tacitly accept the “conventional” expectations associated with 2D art in games.


We discovered SpriteLamp, which essentially allows you to generate dynamic lighting on pixel art. It accomplishes this by producing normal maps, depth maps, and anisotropy maps for use in shaders. All you provide are 2-5 “lighting profiles” of what an object would look like lit from a specific direction (top, bottom, left, right, or front). This animation sort of sells itself:

A zombie lit with the help of SpriteLamp

Courtesy of the SpriteLamp Kickstarter

We encourage anyone interested to check out the Kickstarter or SnakeHillGames for more information. SpriteLamp was successfully funded, and we’ve received beta access. The tool, even in its beta state, is very usable, and has UI that is easy enough to understand for now:

The UI for SpriteLamp

The UI for SpriteLamp

Although we’re not artists, even we could see how exciting this would be to get working in Unity. SpriteLamp’s developer, Finn Morgan, said that a shader for Unity will be provided later, but we decided that we couldn’t wait, so we wrote it ourselves.

Continue reading … →

Imagine for a moment that you’ve just spent a year creating a game, fixed hundreds of bugs, and you finally release it. You’re worried about a lot of things: Will anyone play the game? Will they like it? Will there be any problems? As it turns out, the answer to that is usually yes to all three to some extent. But there will always be problems after launch.

For Minecart Madness, our launch issue was pretty worrisome: on first generation Windows Phone 7 devices, the game ran slowly. Very slowly. So slow that it was unplayable. So what happened?

When Windows Phone first launched in 2010 with such phones as the Samsung Focus, they were more than capable of running games at 30fps, but compared to the next generation (like the Nokia Lumia 800), the original models were slow. Approaching the launch of Minecart Madness, we did most of our testing on our day-to-day phones, which were not first generation Windows Phones.

Looking for Solutions

We immediately began searching for ways to speed up the game on those slower phones. The first thing to go involved the minecart’s headlamp – we use a second rendering pass to draw it with additive blending. Even after disabling that, at certain times we still saw slowdowns, so we chose to reduce the rendering size from the then-native 800×480 pixels.


In particular, whenever lots of track supports were on screen, we still saw slowdowns even with the headlamp blending disabled.

An across the board cut on pixels sounds drastic (and it is), but other possible solutions involved more re-architecting than we wanted to do. After all, every day that the problem persisted were potential lost players and negative reviews. With a bit of empirical testing, we determined that 60% size would allow the game to run fast enough and still look acceptable. Luckily, with XNA we were able to scale all rendering with a single parameter change.


At 60% size, there’s a noticeable quality drop (though less so while playing), but everything is still readable.

So now we knew how to fix the problem, but we needed to figure out when to fix it. See, we wanted to fix the problem for older devices, while still providing the pretty visuals for newer phones.


The most obvious solution was a blacklist of phones, and explicitly checking for the first generation devices. The downsides to this approach should be obvious: it’s easy to be too aggressive or not aggressive enough with the blacklist, and it comes with a maintenance cost of keeping it up to date. We turned away from this plan almost immediately.

Framerate Checking

Our next attempt focused around the fact that the first few seconds of every Minecart Madness run are the same – a flat track with 3 coins and a shield power-up. If the game doesn’t run at 30fps for the first second or two, then there’s no problem. It’s only when the player is trying to jump, dash, or navigating the randomly-generated track that it becomes painful to play.

By measuring the time between beginning to draw and ending (for XNA, SpriteBatch.Begin and SpriteBatch.End), we planned to check if a frame was taking “too long”. At 30fps, each frame should be 33.33ms, so we gave ourselves some breathing room and marked a frame as “slow” if it took longer than 37ms. Then, if we have more than 10 slow frames in a given time interval, we switch to the “low quality” mode outlined earlier in this article.

We thought that with this approach, we would be ready to publish the much-needed performance update. However, even when running on a first-generation Windows Phone, we were not falling back to low quality. Well, why not?


In most modern rendering APIs, a process called batching greatly improves performance. Rather than sending each drawing command to the GPU individually, the API will collect many calls together, until either rendering is ended, or the queue of command data surpasses an internal limit.

You might think that for XNA, SpriteBatch.End would be the call where rendering would actually occur. The documentation seems to agree with that reasoning. However, as it turned out, we were missing a lot of the processing time. To fully capture the time for a frame, we begin measuring at the beginning of Draw, and added an override for Game.EndDraw. By finishing our measurement here, we ensure that all rendering time is included. After making this change, the first-generation phones drop to low quality very quickly, providing a playable experience.

Handling Hiccups

We noticed that even on the newest Windows Phones, sometimes we would still fallback to low quality. Sometimes this was due to background processes on the phone, and sometimes it was due to the amount of rendering we were doing, such as the screenshot above when there are lots of track supports to draw.

To account for this, we reset back to high quality each time the player begins a new round. Then, if a fast phone just happens to have an unfortunate set of frames, next game it will switch back to looking crisp. Slower phones also reset to high quality, but as expected, they jump to low quality quickly for each round.

At this point, we finally had a runtime solution for detecting performance problems, and correcting on the fly to provide a great gaming experience for all of our players. We were able to push the update out only a week after the game launched, an impressive turnaround time in our opinion for such a complicated issue. Since we published this update, we haven’t had to touch this code and the complaints about the game being unplayable have completely dropped off.


As with any solution to a problem, there are wins and losses.  A solution is only worth shipping if the benefits outweigh the drawbacks, and there are always drawbacks.  The might be development time, test time, potential destabilization, or performance costs.  Our solution, while elegant in terms of total code change, is not perfect.  In Handling Hiccups, we talked about how we reset to high quality each round.

On the slower phones, the change to a low quality mode will trigger after 10 frames (1/3 second).  This initial slowness does not go undetected by all users.  When play-testing the fix on slow phones, players noticed that the game, right upon triggering a new run, will stutter.  Users almost unanimously agreed that this was a drastic improvement and a tolerable stutter on their phones.  Users became used to the stutter for at the beginning and adjusted accordingly.

Very few noticed the graphical quality reduction, interestingly.  We did.  A fast motion game, such as Minecart Madness, we have learned, has a benefit of masking low graphical quality.  We had discussions about lowering the quality of only specific parts of the game, such as the track, and maintaining high resolution for other graphics.  We concluded that change was not necessary.


We made a mistake.  This is a perfect example of development not being over after shipping.  Ultimately, it is not possible to test on every config.  Identifying your potential weaknesses and testing them early and often is crucial.  For Minecart Madness, gameplay enjoyment is heavily dependent on frame-by-frame performance.  We should have been looking for low performance devices.  Is your game a turn-based strategy game such as Civilization?  Testing on devices with different resolutions and DPI such that users can read all text and easily click on all buttons should be regular on the development cycle.

Moving forward, we know we will make mistakes.  We will likely miss bugs and have to send out day zero patches.  However, we hope that what we have learned from this simple oversight and our future projects will never ship preventing users from playing or enjoying our titles.  For our developer friends reading this, we understand that shipping a title and putting yourself out there requires confidence and resolution.  However, never be so confident in your work that you overlook simple bugs.  Testing may not be the most enjoyable activity when you could be playing your own game or adding new features, but it is crucial to do, end-to-end.

One year ago, we launched Minecart Madness for Windows Phone. While we’ve been busy working on a new project since then, we thought now would be a good time to look back and analyze how Minecart Madness has done.

Minecart Madness


Minecart Madness launched on October 24th, 2012, and the free version launched a few days later on November 1st. We delayed the free version because we wanted to link the player to the paid version (only once per session), and the store link is not resolved until an app is published. It turns out that linking the player to the paid version was not very useful. After all, why pay $1 for a game that you’re already playing for free? Mobile users are increasingly accepting of ads in free games, and our ads were already not very invasive.

We had issues at launch where the game ran terribly on some devices (we’ll cover the technical side of this in a separate post). While we managed to get the problem fixed with a quick turnaround, some damage was already done – we had a few early negative reviews because of this problem.

Ad Networks

Our stance on in-game ads is pretty simple – display and rotate them as often as possible, but never when it would interrupt or frustrate the player. In Minecart Madness, obstacles can appear from the left, top, and right sides of the screen, and the player needs to see the bottom of the screen to tell where they can land. As a result, our only options were to decrease the vertical height of the game, and add an ad in that space, or not display them while playing. The choice was obvious – we only display ads in menus, or between the player dying and starting again. There’s no doubt that this decreased our ad revenue, but so would angering your players, and we care more about the quality of the game than earning a few extra dollars.

Initially, we only used Microsoft’s PubCenter for ads. Despite a pretty good number of ad impressions, we didn’t make much from ad revenue. We discovered a great library called Ad Rotator that let us easily add additional ad networks. We published an update that added InnerActive, AdDuplex, and MobFox.

After testing the waters with these new ad networks, InnerActive was the clear winner, with a much higher revenue rate, and over 70% of our total ad revenue. We can only speculate how our revenue would change if we focused on InnerActive from the start, but for any future games we create with ads, they’ll definitely be our primary ad network.



Despite less-than-stellar ad revenue, we made this game because it was a game we wanted to play, and we hoped that others would want to play it too. During development, we decided that we would consider 1,000 downloads to be a minimum bar for success.  We were blown away with our download numbers. As of October 24, 2013, we have 1,691,754 downloads! Initially, our download trend matched what we expected – a large initial surge, followed by decreasing downloads per day. However, on two occasions we saw our downloads surge again. To our delight, our downloads have not dropped below 2,000 per day.


The top ten countries that downloaded our game are, in descending order:

  1. Russia
  2. China
  3. Mexico
  4. Brazil
  5. Italy
  1. India
  2. United States
  3. Thailand
  4. France
  5. United Kingdom

Those ten countries combined account for 1.22 million downloads. In total, our game has been downloaded in 173 countries, many of which with only a few hundred downloads. However, every download counts, so it was great that we published our game in nearly all available countries.

Brazil is particularly interesting to see in the #4 spot. When we launched Minecart Madness, Brazil required a unique rating outside of the free ESRB and PEGI we obtained, so we did not publish in Brazil. At some point, Brazil began accepting ESRB or PEGI ratings, and after being informed by a Brazilian fan of Windows Phone, we finally published our game in Brazil in late February 2013. This explains one of the later surges in the download chart above. When downloads are compared by amount of time since publishing in a country, Brazil is nearly on par with Russia, so we heavily encourage other developers to release in Brazil.



As already mentioned, we suffered from some initial negative reviews from the game being unplayable on some phones. However, after fixing that issue, we’ve recovered and are sitting at 4 star ratings in 7 of our top 10 countries. We’ve gotten a lot of feedback that Minecart Madness is a lot of fun, but there’s admittedly not much depth to it.

Before we released, we had a lot of grand designs for features to help with this, but we made (in our opinion) the right call to draw a line somewhere and just publish the game. The incentive for continued play could be higher with features such as leaderboards and daily challenges. We hoped to include some of these features in future updates, but we ultimately determined to shift focus to other projects. We still get numerous reviews regarding how addictive the game is, which we attribute to the procedurally generated levels, a differentiator we are particularly proud of. We will cover our strategy regarding responding to reviews in the Windows Phone ecosystem in a future post.


First of all, no matter how large or small the ecosystem for a platform may be, there will always be players. If you build something that you enjoy yourself, you will not be alone. Testing your game should feel like fun, an exploration of your own creation. Shipping is important and you will have to cut in order to pull it off. The journey of making a game does not end with shipping and, in many cases, it is only beginning. Seeing how your art effects players of different ages and backgrounds is exciting, but also humbling. We have learned from this experience and we continue to try to gain an understanding of what we can do better in the future.

We are increasingly appreciative of our players. There are nearly 1.7 million people who went into the store, looked for a game among thousands of deserving, enjoyable titles, and chose to install and play ours. That number increases by over 2,000 every day. While we are making our next game, people are still installing our current one, and that is something we will never forget. Thank you!