Unsolved Problems in Unified Messaging

From MobileDesign

The promise of Unified Messaging (UM) is to deliver all messages - voice, email, SMS, instant messages, etc. - into one mailbox. Users can access voice mail using their computer, and email using the phone. This goal introduces several technical challenges, many of which have been met. However, it also introduces two user experience problems, which have not yet been solved.

[edit] Modality Matching

The first of these problems is that of modality matching. When I listen to something on the phone, I hear it and (unless I am using a speaker phone) nobody else does. Telephones have evolved to support this one-listener use, and they are quite good at transmitting information verbally. Thus, messages are left with the reasonable assumption that only the intended recipient will hear the voicemail, especially when the welcome message indicates only one user (as opposed to "The Ballard Family" or "The Staff Phone").

Computers are similarly adept at transmitting information visually. If I get any auditory information on the computer, anybody can hear it, unlike with the telephone. The person in the next cubicle can hear my computer. (This is one reason why dictation has not taken off for most office workers.)

With Unified Messaging, this expectation of privacy and modality is ignored. When a text message is played using text-to-speech, the resulting message sounds a little strange, but most early adopters will accept it. However, technology does not yet give us a reliable way to convert a voicemail - on an unknown subject with an unknown length from an unknown person over a phone line of unknown quality - into text. This is something the technologists are working on, but they have not yet succeeded. Thus, receiving a voicemail over email means that there is a sound file that the user must play, thereby exposing the unknown contents of the file to office mates or cubicle neighbors. While headphones help, they are not always appropriate or comfortable to wear for hours a day.

[edit] Message Control

The second major user experience problem is that of message control. When any incoming message can interrupt the user at any time, we must assume that the user does not want to experience email spam at the dinner table or anything but high priority messages while at the boardroom table. Therefore, the system must not pass through every message at any moment.

There are several factors that influence whether the user wants to be interrupted with a message notification, or even the message directly. Some factors include:

• Content of message or message subject - whether regarding today's meeting, next week's meeting, or a critical project.
• Sender's prioritization of message.
• Number of recipients of the message
• The recipient type of the user for the message - whether to, cc, or bcc.
• The identification of other recipients of the message.
• Which address or phone number the message was sent to.
• User-defined filters.
• Sender identification - a message from your boss is more important than a message from a mailing list or a message offering you a product. Properties of the sender could include:
  • Whether the sender is in the user's address book.
  • Whether the sender is in the user's work group. This can be determined by checking in a company server or by the phone number exchange or by the email address.
  • Whether the sender is in the user's company.
  • The user's categorization of the sender. Examples include business, personal, and family.
• User state - in a meeting, driving home, at dinner with family, etc. This may be detected by:
  • Time of day - during work hours, after hours, during lunch
  • User location - using GPS or locally-available information, determine whether the user is in the car, in a conference room, etc.
  • Access device - which device the user is using to obtain the messages (phone, home computer, voice, work computer, etc.

Note that these factors represent part of the user's context - the part most easily discernible. Determining user context (while maintaining privacy) is one of the larger unsolved problems associated with [ubiquitous computing].</a>

Taking these in order, determining the content of the message is quite difficult and not reliable using current tools. Too many users either don't use, or abuse, prioritization. I have only worked with one coworker who consistently applied an appropriate priority to every message; I have had a handful of coworkers who applied a "high priority" status to urgent messages sometimes. Everybody else picked one level of urgency and stuck with it. So the message content at best requires a very large R&D effort to implement and the sender prioritization is impractical given human behavior, except for mailing lists.

The number of recipients and recipient type factors are variably useful, depending on the behavior exhibited by the user's cohort. At work a BCC probably means a political issue. At home a BCC means a friend of mine is sending a bulk email and trying to respect everybody's privacy - or it could mean spam. If I am the only recipient that is useful information, but insufficient to predict urgency. The other recipients of the message are useful, and the issues associated with the sender identification apply.

The address that the message was sent to is a critical aspect of determining message priority, but only when the user has multiple accounts. This, of course, is one way users can use filters to build their own prioritization system without any additional tools. I have created a few filters that relegate my mailing lists and certain email accounts to not notify me when they are received.

Using the sender identification is a tantalizing idea. After all, the sender is included in every message and the action is a simple database lookup.

The problem with any of this is that of configuration. Imagine taking the most technically achievable and useful of the above variables: time of day, sender identification, and access device. To make this work, users have to

• define relevant times of day (at work, at lunch, commuting, family time, sleep, weekend, and whatever is relevant to their life)
• define categories of senders
  • getting address book up to date
  • develop a useful categorization scheme for contacts
• define what to do with each type of contact during each time of day, or at least based on which access device
• determine what to do with people not in contacts during each time period
• if in a corporate environment, what to do with non-contacts from within the company
• correctly categorize all new contacts
• update schemes when life or work schedules change

All this takes quite a bit of organization, time, and skull sweat - for me, it may very well take more time than preparing my taxes. Users aren't going to do this.

[edit] Solving the Problems

The problem of modality matching will be solved when the technologists develop a reasonable method of transcribing voicemail into text. This will take a few years. One current solution would have the voicemail shipped to another country where English speakers earning low salary transcribe the voicemail; businesses like [CopyTalk] are already doing some version of this. I don't know how scalable this might be. In the meantime, I won't use a messaging system that sends voicemail to my computer. Really - it annoys me as a user that much.

Solving the configuration problem is a lot trickier. Possibly the best bet is to use technology to determine how contacts are related to various other files on the computer, and then categorize contacts based on similarity. While this will work for business contacts, it may not work for personal contacts with whom collaborative computing is more rare.

A further strategy is to collect and analyze records on the computer establishing when the user replies to messages from which people (and which are consistently deleted). This should allow the unified messaging software to determine typical times of day and what to do with different categories of contacts during those times of day.

The next problem to solve is how to learn from mistakes. If the system bothers the user while commuting about email spam, a simple user command can tell the system it was wrong. However, if the message from the wife saying she is in labor gets missed, the user has no immediate opportunity to correct the system. While solving the problem later should reduce the likelihood of future problems (however unlikely that correction is), that still means the user missed a critical message.

Unfortunately, if the software caused the user to miss a critical message, the software company is probably open for a few lawsuits. But if the user caused the message to be missed (due to configuring it himself), most users will never use the software at all.

Does anybody have any better ideas?