Shift + Space Documentation

Design Challenge

Take a digital appliance, analyze its usability and suggest improvements.
I will analyze two symbiotic devices: the Keyboard and Mouse.

Objective

To reduce the repititious task of switching between keyboard and mouse. Quite freqeuntly we must switch from the keyboard to mouse/ pointing device in order to access certain buttons that cannot easily be accessed via the keyboard alone. Take for example an email client, after completing an email, I must take my hand off the keyboard, move it to the mouse, navigate to my desired button, click, then move hand back to keyboard. I find this process repititious, inefficient, and annoying.

Observations

Counting the number of steps/ operations it takes to compose and send an email with a keyboard and mouse.

Emotional analysis:

The total number of operations between the two options is minimal but the emotional difference is very apparent. Every time I switch from keyboard to mouse I waste time, energy performing that motion and become mentally annoyed. Over time, this repitition becomes a nuisance. Keyboard shortcuts is one answer, but they are not always available and are often unknown by most users.

Research - IBM TrackPoint

The "TrackPoint" is a centrally located knub, that allows users to control the cursor without moving their hands from the keyboard. Users say it is faster and more convenient than a mouse or trackpad. However, the TrackPoint has some major issues:

1. Is not easy to use. Has a steep learning curve.
2. Is centrally located, which forces the user to return to a fixed location on the keyboard.
3. Requires fine, precise motor coordiantion which can be taxing over long periods of use.

Research - Trackpad

The trackpad is a great alternative to the mouse, especially when space is a limitation. Compared the Trackpoint, the trackpad is easy to use, however it still requires fine motor control, which still makes it impossible to use for disabled users. Some trackpad problems:

1. Is centrally located, which forces the user to return to a fixed location below the keyboard.
2. Requires fine, precise motor coordiantion which can be taxing over long periods of use.

Idea #1 Make each keyboard key behave like a TrackPoint

Instead of one centrally located TrackPoint cursor, I propose making every keys on the keyboard into a cursor controlling the device. This allows users to maintain their current hand position, which eliminates unnecessary switching between keyboard and mouse / pointing device. However, this still requires fine, percision control. Additionally, this method is not cost effective. Trackpoint repeated 70+ times = very expensive keyboard!

Idea #2 The Magic Keyboard: make entire keyboard surface into a trackpad

If the entire keybaord surface is a Trackpad, users can benefit from having a larger trackpad area, allowing greater maneuverability- free flowing, natural hand motions as compared to small, precise nudges.This also eliminates the unnecessary switching between keybouard and mouse.

A mouse click can be performed by simultaneously pushing any two adjacent keys- such as T+Y or U+I or U+H and etc.

Right click can be performed with three adjacent keys- such as H+J+K or S+D+F or B+H+U and etc.

Highlighting can be acheived with a combination of a regular click and shift- similar to highlighting in Jef Raskin's ARCHY program.

Below is notation for creating an email to kidmang@yahoo.com and sending it via NYU webmail.

Notation Key: [Lup] lift hand up
[Ldwn] put hand down
[MV] move mouse
[T] trackpad hand movement
[P] press button (keyboard or mouse)
[R] release button
[H] hold

Proposed Option (maximized keyboard usage, NO MOUSE)
[T] track to "compose" button [P] [P] press and release two keys simultaneously [R] [R]
[T] track to "To:" field [P] [P] press and release two keys simultaneously [R] [R]
// Type "kidmang@yahoo.com" (37 operations)
[P] k [R] [P] i [R] [P] d [R] [P] m [R] [P] a [R] [P] n [R] [P] g [R] [P] shift [H] [P] 2 [R] [R] [P] y [R] [P] a [R] [P] h [R] [P] o [R] [P] o [R] [P] . [R] [P] c [R] [P] o [R] [P] m [R]
[T] track to "Body" field [P] [P] press and release two keys simultaneously [R] [R]
// Type "usability test" (28 operations)
[P] u [R] [P] s [R] [P] a [R] [P] b [R] [P] i [R] [P] l [R] [P] i [R] [P] t [R] [P] y [R] [P] space [R] [P] t [R] [P] e [R] [P] s [R] [P] t [R]
[T] track to "Send" button [P] [P] press and release two keys simultaneously [R] [R]
Total = 85 operations

Despite being the same number of operations, this method allows the user to remain on the keyboard, which is the main advantage. Even with increased keystrokes, I would prefer this over switching between keyboard and mouse.

Here is a video demonstrating what this keyboard interaction would look like:

Second revision

Create a software prototype for operational testing
Given only 2 weeks to prototype this project, I don't have time build a functional tracking keyboard. So will focus on developing software to control cursor movement via the keyboard.
I used Amit's mouse control code and made some changes (sped up cursor speed, switched keys for right handed use, added buttons to click on, implemented TYPING and CURSOR MODES) to see what it feels like to use your keyboard for cursor movements. Although this was not my original intention, I feel this software test will give a sense if the concept is worth further exploration.



This processing applet is designed for right handed people (sorry lefties), as the mouse control keys are coded for the J,K,L,I,M,U,Y,O,P,N,",","." keys. There are two modes of operation: TYPING MODE and CURSOR MODE. (click here to download the zipped processing file)

How to use this software:
Press CAPS LOCK to enter TYPING MODE. You will see the output of your typing in the command console.
Unpress CAPS LOCK to enter CURSOR MODE. Here you will use the keyboard to mouse over the designated buttons and click on them to change the applet's background color.

This test evaluates if working from the keyboard alone (or for the most part) is effective/ efficient. This is NOT designed to replace the mouse or trackpad, as they have their own specialized benefits, but rather, is an optional substitute when you don't want to switch between keyboard and mouse.

Here is a video of me running the software test.

User Reactions: The users I tested the software prototype with enjoyed using the keyboard exclusively, however they wanted more control over speed and accuracy.
They found it annoying to wait for the cursor to move long distances.

It was easy for them to highlight the small buttons. The only challenge was the super small 10 x 10 pixel button, which required more percision to get at.
Some users also wanted to highlight and be able to draw boxes and shapes like they do on a mouse or trackpad.

Here are some user testing videos with their comments and suggestions.

Improvements:

Snapping- when you navigate close enough to a button, the cursor automatically snaps to the button, eliminating the need for fine adjustments.
The option of acelerating and decelerating.

The main use for this application is to get at buttons which aren't accessible via the keyboard, such as the send button in a web mail program. A more direct solution is to implement a TABBING function that jumps to and between inaccessible buttons- similar to Mac OSX's apple TAB and PC's Alt TAB, which allows you to switch between running applications.