The Life and Times of Brock

So the iPad came on sale the other day, and has sold huge numbers and generated massive amounts of hype. The computer is a tablet, or a slate, which is a form factor that is designed to sit squarely in-between smartphones and laptops.

The world was split the day the announcement came out. Half of them (mostly first-gen iPhone users I’d be willing to bet) praised Steve Jobs on his incredible vision of the future, while many others gave it one massive ‘meh’. The concept of this computer is designed so that anyone is able to use it without much hassle, even people who haven’t used computers before.

But for power users, it’s not exactly their type of beast. I couldn’t see myself scurrying off to spend a minimum of US$500 on this type of device, in that it lacks what I want of my computers: multitasking (background processing) and access to some sort of file structure.

Firstly, due to the small screen size and limited processing capacity of the iPod Touch and iPhone, it makes sense for a user to only want to be running a single application at a time. Using too many applications drop battery life significantly, makes programs run much slower, and there is also the idea of switching tasks on such a small screen which is difficult to fathom (from the Windows Mobile- and Android-based devices I’ve seen and used, it’s failed quite considerably, though apparently the Palm Pre is quite elegant in multitasking). However, in a screen that is 10 inches, people would assume there would be room to have some type of dock on the bottom that would allow for task switching. But it wasn’t so.

Secondly, I want to be able to access the file structure of programs. Apple has moved away from the old ‘Explorer’ interface where everything is controlled by apps, which exist in a form of virtual vacuum. Sometimes I want to share files between computers just by dragging and dropping, and not having to go through some intermediate middleware that nobody really needs that much (I’m looking at you, iTunes!). I would like to be able to synchronize Live Mesh with my tablet, and be able to access and edit those files from anywhere, but it’s not so. It must be done through some sort of intermediate app.

In other news, Microsoft released details of their concept of tablet computing, which is basically the antithesis to Apple’s iPad, named the Microsoft Courier (which is a far better name anyway). Gizmodo went on to blog about the Courier and basically what it does, where it’s headed, and how it’s used.

Rather than being a single screen tablet, the Courier establishes itself in an incredibly nice metaphor of a newspaper, booklet, or journal, which automatically enables multi-tasking—the ability to use several apps at a single time. While one screen is using the web browser, another one can be taking notes, chatting to friends, looking through photos, playing music… the list is endless. I assume that, by the video posted below, there will be some pretty handy gestures that we’ll end up using to change applications.

The main movement that these big companies are trying to do, is that they’re trying to extend this metaphor of invisible computing: not necessarily that the computers themselves can’t be seen, but that they are not noticeable distinctly as computers. The way the iPod market is, I would be willing to bet money that only a minor percentage would consider any iPod a ‘computer’, because of the basic things: no keyboard, no mouse, it’s either worn, or carried in a pocket, and isn’t used in the same way that a ‘regular’ computer is used. The punch-line is: these companies want us using computers without even realizing we’re using computers. Hence why there’s such a strong attraction to the Microsoft Courier: we are all familiar with books, pens and paper, so why not just digitize what we are familiar with, in a familiar way?

If we are talking about invisible computing, why not just give a quick mention to the Microsoft Surface?

Here, the basic idea of having a coffee tablet as a computer is incredibly appealing (as much as naysayers would disagree, I’m sure). It uses a strong metaphor of an actual desktop, meaning that objects in the computer have physics and collision detection attached to them, in a way to trick us into using these devices, not as a computer, but as we would a real-world object. It will even interact with your objects just by placing them on the table.

This way, it can be as intuitive and natural as possible. If you’ve got an image on one device and want it copied to the other, place both devices on there, grab the image and throw it to the other phone. The computer will do the rest of the work, which is usually pretty messy work, all the while the user is believing that they actually ‘gave’ a phone an image, which it accepted and took for its own. This is Surface’s idea of using Windows’ File Explorer interface, but with a much stronger visual metaphor running behind it.

* * *

Okay so the main problem with the mobile space is that it’s currently dominated by touchscreens, and as such it makes it difficult to create first-person immersion in a game.

The technology is near-perfect for games that require an overhead view (such as an RPG a la Final Fantasy), or a third-person view (a la Diablo), where the touch commands can be quite easily mapped and pinpointed. Also, because of gesture-based movements (using the accelerometer) it’s quite easy, and possible, to have racing games.

Casual, puzzle and racing games basically dominate the market. However, now with Final Fantasy I and II on the market, hopefully the RPG market will get heavier (and then eventually saturate, but that’s a story more inclined with economics instead of game ideas).

As I see it, the main problem that people have with touch technology is that they’re still stuck in the days of having either a mouse pointer, a keyboard, or a controller. As such, nearly all games I’ve played that require some type of first- or third-person controls, it will emulate the PS3 and Xbox controllers, essentially having two touch-based ‘joysticks’ on the screen.

The first fps I played on a mobile device was actually Doom for Android, which used a virtual D-Pad (horrible) to move, and a roller ball for shooting (press it to shoot) and moving (roll forward/backwards to move, left/right to turn). All in all, it gave me a bad feeling in my throat every time I played it and tried to like it. Basically from that moment on, I told myself there had to be a better way to manipulate a character in first-person.

Needless to say, there are problems with the current idea of using touchpad-based joysticks, for a few reasons.

1. The touch-sensitivity is very broad, especially based on one’s fingers. Movements are very ambiguous, and hard to make concise and accurate.
2. A double-edged sword: either the ‘joystick’ pads are too far in the middle of the screen, and cause the fingers to take up too much visual content, or they are too far to the bottom corners that it makes it nearly impossible to move it backwards or to the side (on both sides where the edge is).
3. The pads are usually too big, and take up too much space, or they are made too small (reference point number 1) and hard to use (the optimal thing to do here would be to make them semi-transparent, anyway).

Basically, what I’m saying is that we are using a new input technology, and both the software and our way of thinking also needs an update.

Noting here, Final Fantasy I is using an arrow-based D-Pad approach to movement in the game. It’s not actually that bad because it’s a boolean approach (ie. there are only four points that are either on or off, and not a more fuzzy approach of a joystick-like movement scheme), and that it’s transparent. It keeps the fingers away from the content of the page, while still being easy enough for a user to automatically learn how it works.

The battle screen here is thankfully touch-based, with big, bold icons to show what act a person could do. A lot of thought has gone into this to be optimal for touch devices.

If we have a look here, it’s an emulator for the iPhone and iPod Touch, using exactly the same controls. I can understand why this is like this (after all, it’s en emulator, not a completely rethought game), but the reason I put this here was to show that this is the mind-set of most developers at the current moment (that is, grab a pre-existing technology, and hack it into a touch-based game. Profits ftw!).

In keeping with my idea of updating the user interface to match with the touch-based device, here is what I would have personally done if I were making an RPG on a mobile touch device (and who knows? I still actually might):

Of course it would look slightly better than that (hopefully with transparent images around the side that maybe blur the background like Aero Glass, but not exactly destroy the game’s vision), but I’m sure you can catch where I’m coming from here; both hands are used to do an even amount of tasks, and it’s visually taking up less of the main content (the central area) than the version of Final Fantasy above (not that above is necessarily bad, I just believe that this is more optimized for touch). And of course I would try to offer the option of changing back to the other option if the user wanted, but I just prefer this solution.

If the player wants to access the menu, they won’t mind so much if their fingers cover the central area (they are, after all, going to a splash screen). If the player wants to talk, or interact with anything they are facing, pressing the middle part of the screen (with either finger, and the character would start interaction (or fire the event, whichever works).

Also, with exactly the same buttons I’ve got here, I can cause the characters to run (by double-tapping a button and holding it, or pressing both up/up or down/down buttons for multitouch-based devices), or turn but not move (by tapping one of the ‘buttons’ for less than a few milliseconds).

What I’m saying is, this idea fills the criteria for an RPG, while optimizing it for touch by not trying to make it exactly as its last platform was (as in, a D-Pad on the left side, action buttons on the right etc).

Then I started thinking, “why can’t we apply those principles of simplicity to first-person games?” and this is what I came up with:

Again, simple movement touch-pads that surrounds the main area, stopping the user from obstructing the vision with his own fingers. In order to sprint (if this is a multitouch device), both forward buttons can be used at the same time (and same with running backwards, I suppose). If the device is not capable of multitouch, a double-tap can be substituted on one of the directional buttons. A player can still move diagonally forward (there would be a hot-spot between the side-keys and the forward/back keys which enables diagonal movement), and change weapons quite easily (when the weapon-change button is pressed, the menu pops out—it is not there for regular gameplay). The reload section would have a quick visual interface which would show how many bullets were left in the current gun, and the menu would subsequently pause the game.

I imagine that the facing of the game would either be controlled by the accelerometer (by moving the device itself) or by swiping the finger across the middle, as is seen in most applications.

From here on, the part about shooting the weapon (which even begs the question about why we need so much violence in a mobile, handheld game?): we can choose to press wherever on the screen, and the weapon shoots in that direction, or we could choose to have the weapon honed in the middle of the screen, unmoving. Either way, they both have their own reasons, and I would have to try it out to make a proper decision as to which way I prefer.

In most games, there is the ability to aim ‘down the scope’ of some weapons. This would be handled in the same section of “tap to shoot”, but would be done by holding the finger down for a set number of time. This would cause the weapon to be placed close to the eye, and subsequently more accurate (as opposed to ‘firing from the hip’ as our good friends in Call of Duty 4 inform us). To break from aiming down the scope, one can tap and hold the middle, or just choose to sprint somewhere (as mentioned above).

So basically all I need to do now is create a prototype of this. If someone is reading this, and such an interface already exists on a mobile device, I would like to know to try it out and see how it works (if it does, come to a realization that I’m in over my head here and quit, and if it fails, pretty much radically change my ideas and release a Family Guy-based jigsaw puzzle—Either way it’s back to the drawing board).

Either way, tell me your thoughts.

1. The touch technology has no been very good until capacitive touch – that is, they’ve needed to use some type of stylus or it just was not accurate enough for a finger to use
2. Touch screens have been integrated into monitors, which are standing vertically, meaning that, in order to interact with them, the fingers and hands must cover the content of the monitor to actually use them. Also,
3. Nobody wants to lift their arms up all day to work on a single display. This phenomena is what is called the ‘gorilla arm’, where it would cause strain in the arms and shoulders of the users if it went for too long.

I imagine my concept would be somewhat similar to that, without spreading it out as much. Make it less circular, more to the ergonomics of how we use our hand. Here is Windows 7’s default touch keyboard looks:

There are a couple of problems with the default. Firstly, it covers up too much screen content. Secondly, it’s too rigid and square.

My touch keyboard would be curved, more in-line with the DialKeys (but maybe a midpoint between the two), with everything being Glass (transparent), so that the top screen can still see content behind it, yet see what the fingers are touching.

Now, the positioning of the keyboard would be crucial. Should it go low, just above the taskbar (as are the DialKeys), or should the taskbar be moved elsewhere (to the top or sides?), and then how do we access it when we need it (I’m going to assume we throw the keyboard away when we don’t need it, and activate it when we do).

I still need to do some concepts, and will update this post with drawings and so on. I imagine there is a tab on the side of the page (as is with current Windows 7 touch interface) that is ‘dragged out’ and becomes the keyboard.

Then, finally my last concept is to have vibration creators just in the corner of the screen which are activated every time a key is pressed, to give the user tactile feedback every time they tap a key, as per most Android devices.

This post is far from finished. Hopefully if I’m able to purchase a touch-screen monitor, I’ll be able to figure out how to hack it up and turn it into a mouse/keyboard replacement.