Exploring Smartphone-based Web User Interfaces For

Exploring Smartphone-basedWeb User Interfaces for AppliancesKatie Derthick*#James Scott* Nicolas Villar*#University of Washington*Microsoft ResearchUSAUnited Kingdomderthick@uw.edu{jws, nvillar}@microsoft.comABSTRACTChristian Winkler*††University of UlmGermanychristian.winkler@uni-ulm.deor segment-based LCDs for output. This limits the UIs ofsuch appliances, often obscuring or even leaving out all butbasic functionality.We describe the SAWUI architecture by whichsmartphones can easily show user interfaces for nearbyappliances, with no modification or pre-installation ofsoftware on the phone, no reliance on cloud services ornetworking infrastructure, and modest additional hardwarein the appliance. In contrast to appliances’ physical userinterfaces, which are often as simple as buttons, icons andLEDs, SAWUIs leverage smartphones’ powerful UIhardware to provide personalized, self-explanatory,adaptive, and localized UIs.As an illustrative example, the home cappuccino machineillustrated in Figure 1 (c) is a capable machine retailing for500 USD, but has just 7 buttons on its UI. One button, for“latte,” serves up a particular amount of coffee and steamedmilk when pressed. However, by manipulating the samebutton in an obscure procedure that is non-obvious withoutreading the manual, these default amounts can be changed.In fact, 20 distinct functions can be accessed using those 7buttons (e.g., set water hardness, cleaning functions, etc.),so all but 7 functions involve using the buttons in a way thatgoes beyond just “press,” and are therefore somewhatobscure. In addition, there are other settings that themachine internally exercises control over which are notexposed on the UI at all, such as the temperatures andpressures used in the coffee-making process. Nor can themachine detect which user is in front of it and offer acustomized experience, e.g., providing their preferred latteproportions or even using their preferred language. In fact,this machine has no writing on it, using icons to representdrink types, which is common for many appliances to avoidhaving to localize them for each country they are sold in.To explore the opportunities created by SAWUIs, weconducted a study asking designers to redesign twoappliances to include SAWUIs. Task characteristicsincluding frequency, proximity, and complexity were usedin deciding whether to place functionality on the physicalUI, the SAWUI, or both. Furthermore, results illustratehow, in addition to support for accomplishing tasks,SAWUIs have the potential to enrich human experiencesaround appliances by increasing user autonomy andsupporting better integration of appliances into users’ socialand personal lives.Author KeywordsSmartphones, appliances, user interfaces.Yet, many users of such coffee machines have smartphonesACM Classification KeywordsH.5.m. Information interfaces and presentation (e.g., HCI):Miscellaneous.Appliance UISmartphone UICost of UIhardware tens hundredsHuman FactorsProcessor8 or 16-bit, kHz orlow MHz32-bit, GHz, withGPUINTRODUCTIONUser inputOn appliances such as coffee machines, washing machines,etc., user interfaces (UIs) are constrained by variousimportant factors [2], particularly the need to minimize costof the physical UI hardware. It is typical to find applianceUIs that have physical buttons or knobs for input, and LEDsPhysical controls(e.g., buttons)Multitouch, sensors(accelerometer,camera, etc.)User outputLEDs, perhaps alow-res screenHigh-res colorscreen, audio,vibration, etc.HelpSeparate usermanualSelf-explanatory UIspossible withimages/video/audioLocalizationOften icons-only tominimize costUser’s preferredlanguageAdaptivenessStaticHighly customized,personalized,context-sensitiveGeneral TermsPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advan

the appliance’s web UI to be displayed automatically, allowing the appliance to take advantage of the powerful smartphone hardware. We refer to such Smartphone-based Appliance Web User Interfaces as SAWUIs. We envision that SAWUIs go beyond simply supporting task-o