In this article, we explain our development process, some of the research issues, and the emergent design concepts. Samsung gave us significant control over strategy and design. We could consider the entire user experience, from performance to pleasure, and the entire USA mobile market was our user base to study. Our team faced formidable challenges. We needed to synthesize emerging technologies, but to make them useful, not just technology-driven solutions.

Our project resulted in the conceptual design for a wireless, mobile information device that we called “Mob-i”. The “I” stands for both “information” and “identity.” We delivered to Samsung a catalogue of design concepts, some of which appear in the accompanying Figures.

Initial Research Phase

To design usable “baby-face” devices (with small display areas) is difficult. For consumer devices, there is also the fundamental challenge to design useful products. To achieve both goals, we found it necessary to follow a user-centered design process. We began by selecting three areas of research, each of which was analyzed by one of AM+A’s Designer/Analysts: cultural and market trends (especially to learn about advanced product developments in the faster growing markets of Europe and Asia), emerging mobile device technologies, and advanced user interface techniques. Quickly acquiring specific expertise, we could then provide informed reality checks for our designs based on our knowledge of consumer desires, technical feasibility, and user-interface usability. An additional dimension of our research came from direct observation of users.

Initially, even the device’s main purposes were undefined by the client, which forced us to provide a solid foundation for design decisions. We needed first to identify the users’ unmet needs and desires. From these we could derive required functions, from which we could derive appropriate forms:

Observing Users To Discover Their Needs

To develop innovative products, we needed to identify unmet needs and desires of users. We believed that focus groups would not be reliable, because users often find it difficult to envision products or behaviors they have not yet encountered. We believed that observation in the field would be necessary to fully understand people’s mobile experience.

We targeted a limited number, but a wide variation, of users for a contextual inquiry in order to observe as many situations and behaviors as possible. Our research showed we should analyze subject types with varying profiles, e.g., parent vs. teenager, technophile vs. technophobe. Our selected user set consisted of a limited number, but a diverse set of people, e.g., a minister who used many high-tech gadgets, a male college student, a female high school student, a male professional commuter, and a female, single-parent software entrepreneur.

We shadowed each user was during one day of typical activities. We focused on observing their use of information. We watched not only how the subjects used electronic information sources such as mobile phones and personal digital assistants, but also traditional sources of information such as newspapers, magazines, and sticky notes. We maintained a running log in text and images and recorded unmet needs, inefficiencies, and information artifacts. Each user also continued afterwards a similar observation, recording activities on a paper self-log or on a tape recorder that we provided. Self-logging on a tape recorder was particularly helpful in many situations in which paper logging was inconvenient, e.g., while walking. Later, we brought back all users for a group session of interpretation.

The user-observation activities yielded many significant insights that later provided inspiration for our brainstorming workshops. Some examples are of the information sources and the brainstorming activities are described below:

Figure 1 shows the personal organizer of a teenaged girl, which has a large amount of personalization. The owner is using the organizer to display intimate photos and memories in addition to raw information. The device goes beyond pure functionality to provide warm, reassuring imagery. Based on these observations, we developed concepts for an affective user interface that would make Mob-i seem more personal. (See Figure 4: Affective Personality.)

Figure 2 shows a typical problem of a user needing to use several devices simultaneously. The subject is receiving information by phone and manually transferring the information to a paper appointment book. On phone-PDA combinations for which the device must be moved away from the ear for hand-and-eye activity, this kind of use is very difficult. Such observations helped us to design appropriate hands-free and single-device interaction. (See Figure 5: Detachable Ear-piece.)

Analytical Framework for Usage Space

Based on our research, we developed a framework of usage spaces shown in Figure 3 below that organized many inter-related uses, helped to organize the complexity of people’s behavior, and helped to form solutions that are systematic, not isolated. Each space is explained briefly below.

The Information space includes static reference information, such as weather reports, dictionaries, and entertainment listings. Organizational information includes all personal information management functions. Business information supports specific professionals, such as a drug interaction reference for doctors.

The Self-Enhancement space describes ways in which the device extends one’s normal capabilities. Active Assistant functions include extending one’s memory or reinforcing one’s intentions. For instance, “FriendMinder”, a project from Stanford Persuasive Technology Lab, encourages regular contact with loved ones [http://hci.stanford.edu/captology]. Health and Safety describes the ability for mobile devices to enhance safety through contact with emergency response teams or family.

The Relationships space describes where wireless, mobile devices will become especially successful by extending existing social relations beyond how phones connect people. Products like Cybiko allow users to be aware of other Cybiko devices in the area and enable people to connect to temporary communities of other Cybiko users. For instance, they are capable of enhancing intimacy between two people who wish to be in more constant contact through “presence.” (See Figure 6: Relationship Room)

The Entertainment space includes enjoyment of portable media such as music, astrology readings, jokes, and games involving multiple users and location-based elements. These contents become particularly desirable in captive situations or for short bursts of spontaneous stress-relief.

The M-commerce space refers to a variety of uses including electronic money and e-coupons that will enable viable commercial services. We referred to this space as “they” because, unlike the other spaces, m-commerce involves services being provided from the business world to the individual.

Making Wireless, Mobile Devices the First Truly Personal Computers

The Identity space at the center of the five usage spaces contains information about the user. Users carry handheld devices in their pockets or bags, making them almost “wearable” computers. Because handheld devices are, or will be, “always-with-you” and “always-on”, these devices have a great potential to become knowledgeable about their owners over time. The “I” in Figure 4 represents the device’s use as the seat of memory of the user.

Wireless, mobile information devices must therefore provide persistent, personalized data, much like “cookies” do for web surfing. However, they should be able to do so over more of the users’ lives, because they will accompany users wherever they go. Memory of basic user information and more complex behavior patterns will allow such devices to reduce the amount of input needed, e.g., reducing the need to enter personal contact information more than once for many circumstances, remembering a user’s vehicle preferences when renting a car, or offering the weather in the city you to which you are scheduled to travel. (See Figure 7: My yellow pages)

Promise of Mobile Computing

As designers, we sought to exploit the unique form factors of wireless, mobile hand-held devices. We first contrasted the inherent strengths and weaknesses of hand-helds in comparison to desktop and laptop computers, then designed concepts to use the strengths and mitigate the weaknesses. Strengths include the fact that hand-helds (1) are always with you; (2) they are portable for social settings; (3) they are voice-user-interface ready; (4) they come with built-in telephony and connectivity; and (5) they are affordable. Weaknesses are that they have difficult input and limited output capabilities.

For example, compared to laptop computers, which are also shrinking in size, we noted that handheld computers are more popular in social settings due to their lower cost and greater durability. In general, we concentrated our designs on people-oriented tasks rather than data-oriented ones or entertainment-oriented ones, like checking stocks, surfing the web, or playing games.

We also tried to emphasize Mob-i’s objectives by segmenting the basic ways that people use information:

Creation: using authoring tools, such as a spreadsheet tool to create content from a blank slate

Gathering: includes picture-taking, filming, recording audio, gathering sensor input, and filling in input forms (such as express-mail shipping pick-up agents use in the field)

Processing: organizing, manipulating, and interpreting information through applications and file managers

Retrieval: accessing information from the web, or from personal storage

Communication: transferring information either to or from the device

Wireless, mobile devices should excel at communication tasks, for which only the destination needs to be specified. They also may be made to perform adequately at retrieval tasks like email reading, web surfing, and gathering tasks such as digital picture taking. For example, we emphasized information gathering features because they present fewer input and output demands.

Two Competing Design Philosophies

When designing products that merge many functions, two distinct philosophies compete. The information-appliance philosophy emphasizes the value of appliances that are consciously designed with one primary application in mind, like the toaster vs. coffee maker in the kitchen. Conversely, the Swiss Army knife philosophy, like its namesake, emphasizes designing a single product that is useful in many different situations.

We believe this debate unnecessarily polarizes the issue; successful wireless, mobile devices lie somewhere in between these two extremes, providing access to high-value information and enabling mobility-based tasks. The challenge will be to identify what tasks the user will consider essential vs. those the user will access later using desktop computer.

To discover the right mix, we let our user subjects prioritize features themselves. We ran an exercise in which they listed each desirable feature on a card and taped them together to form long chains on a wall. Everyone could visualize that more functionality leads to more complex devices.

Another way that we thought about the value cost ratio of various features was through taking an inventory of the mobile belongings that the subjects carried in their pockets, wallets, backpacks, and briefcases, which we photographed.

For example, one subject used a two-tiered bag system. A larger bag was used during the day along with more information tools (including textbooks, newspapers, various electronic devices, etc.). This subject would switch to a smaller bag with essentials only for night-time mobility (cell phone and appointment book). In some situations, even this smaller bag might be left behind, with only a wallet being needed. Even within bags, users demonstrated specific strategies for which devices (e.g., pens) would be most accessible in external pockets. The wireless, mobile device can be understood as a container of information and tools. Metaphorically, designers should decide whether they are intending to create a wallet, purse, backpack, or briefcase for their users.

Input and Output Limitations

Hand-held devices are often limited by a small “baby-face” display and tedious text input. Throughout the project, this input and output issue came up as a constant challenge. The tremendous constraints forced us to consider creative solutions, which we describe below.

Contextual awareness means that mobile devices can react in a more sophisticated way to the users’ input, automatically assisting users as much as possible, thus saving time and reducing the need for explicit input. For example, the phone can automatically default to dialing a friend’s work number during normal working hours in the friend’s own time zone. On evenings and weekends, the device can default to dialing the friend’s home number. If the device can go further and indicate the “presence” of the person to be called (e.g. available, in-a-meeting, driving), then possibly a call won’t even be necessary. (See Figure 8: Activities During Calls)

Location Awareness is enabled by the use of GPS and Bluetooth technologies. User-interface designers focusing on interaction soon will be able to utilize knowledge of a user’s location as a way to aid usability. During our user observations, subjects encountered many problems that could have been aided by a location-based reminder. This location-based reminder can facilitate efficient task completion by reminding a user, at the appropriate time and place, when/where he or she can do a certain task. (See Figure 9: Location-based Reminders)

Time-shifting means that Mob-i supports new modes of mobile work by affording a “start now, finish later” approach. For instance, users may simply mark each message as “Reply at Home”, “Reply at Office”, or “Print out at Office,” etc. When they reach that location, they will be prompted to complete these partially finished tasks.

Brainstorming, Prototyping, and Final Deliverables

Following the research and strategy phase, we ran a series of structured brainstorming sessions. By analyzing these sessions and previous efforts, we provided Samsung with an illustrated catalogue of about 100 conceptual product directions. We conveyed each concept through a descriptive scenario and illustrations. Samsung decided which ones to emphasize and to develop further.

Finally, we created an interactive prototype in Macromedia Flash to demonstrate those scenarios selected by Samsung. This prototype focused on concepts that involved more complex interaction or animation. For example, the prototype shows what it is like to receive an e-coupon upon entering a store, or screen calls based on the user’s location. We demonstrated solutions to challenging but important situations, like creating appointments, using voice commands, and taking notes during a phone call.

Conclusion

Because this project was open-ended and future-oriented, AM+A was able to spend a greater-than-normal amount of project time in research, product strategy, and user observation. The use of frameworks to organize and prioritize insights proved very valuable. The results of the project were varied, realistic, compelling, detailed depictions of coherent concepts for one or more products. Samsung engineers would determine, in the final analysis, which concepts to incorporate into specific manufactured products. Through extensive communication, including meetings with the client in California and Korea, Samsung’s user interface design group was able to observe and learn from a user-centered design team essential components of our development process. Thus, Samsung felt that short-term and long-term goals were achieved.

Additional Figures: Use Scenario Examples

Figure 4: Affective Personality
Mob-i has an engaging and expressive personality, shown through an expressive animated character. This character, which resides on the title bar of every screen and lets the user know what is happening on the device in a constant, noticeable, but non-obtrusive manner. Indicators include signal strength, memory, battery life, reminders etc. Moreover, Mob-i’s personality is intended to enable users to feel more attachment to their personal pocket assistants. Users can download new personalities, which load new images, phrases and speech voices onto the device.

Figure 5: Detachable Ear-piece
Mob-i’s main body communicates to a wireless headphone-microphone via Bluetooth. This soft ear-bud, which pops out of the back of the device, can be worn comfortably in the ear like a hearing aid. Separating the ear attachment allows the user to perform visual tasks, such as note taking, during calls, without using a speaker phone which compromises privacy, and without the inconvenience of trailing wires. Wearing the ear-bud regularly creates a more discrete experience; users are notified of calls and reminders privately and silently by voice output from the device. Text reminders are read through text-to-speech and voice recordings are played back.

Figure 6: Relationship Room
Each person in your address book has an easy-to-enter Relationship Room. This room is a one-to-one chat, scribble, and photo area. You can exchange personal messages with people in real-time, or leave them love notes for later. The rooms are kept over time and eventually form a mutual scrapbook of memories and/or a data/transaction archive.

Figure 7: My yellow pages
Jack is getting repairs done by a carpenter. He will need to call back in a few days to see if the work is done and then will need to see the map again to remember how to get there. Mob-i maintains a history of regular businesses that Jack uses. Jack saves businesses to his “yellow pages" list (e.g., the veterinarian for the pet), separate from his personal address book.

Figure 8: Activities during calls
The What-can-I do button allows the user to perform common tasks during a phone call. For instance, if the user is talking to Jack, the menu is adapted to include shortcuts for making an appointment with Jack and looking up Jack’s address.

Figure 9: Location based reminders
Because Mob-I knows where Jack is, Mob-i can help Jack to remember to do something when he is at a good place to do it. This capability allows Jack to separate his to-do’s for work, home, and out-running-errands. Previously, Jack left himself messages on his home answering machine, but now he leaves messages for himself on Mob-I, and Mob-i reminds him of those messages. Examples include the following: Remind me to take out the garbage when I get home on Thursdays, Check oil next time I get in the car, Pick up jumper cables at the mall, and Get my CDs back when I’m at Jill’s house.

Editor’s Note: An expanded version of this article was originally published in American Center for Design, 5 April 2002. Copyright ©2002 by Aaron Marcus and Associates, Inc.

---

Aaron Marcus is an internationally recognized authority on designing user interfaces, multimedia, documents, and online services. He has authored several books and is the President of Aaron Marcus and Associates (AM+A). He was recently named a "Master of the 20th Century" by the International Council of Graphic Design Associations, the professional world body for graphic design and visual communication. Marcus can be reached at Aaron@AMandA.com.

Marcus acknowledges the assistance of Eugene Chen, who directed the practice of user interface design and analysis at AM+A and who co-authored previous versions of this article. Prior to AM+A, he helped launch software products. Chen has also worked as a professional composer and sound designer, creating audio branding, original music and sound installations for such clients as Eidos and Levi Strauss. Chen can be reached at Eugene.chen@AMandA.com.