“Your customer has five senses and a small universe of devices. Why aren’t you designing for all of them? Go beyond screens, keyboards, and touchscreens by letting your customer’s humanity drive the experience—not a specific device or input type. Learn the techniques you’ll need to build fluid, adaptive experiences for multiple inputs, multiple outputs, and multiple devices.”
Cheryl Platz, back cover for her book Design Beyond Devices: Creating Multimodal, Cross Device Experiences
Design Beyond Devices: Creating Multimodal, Cross-Device Experiences is a groundbreaking UX design and research book that’s applicable beyond the field to anyone working in cross-channel products: product management, content strategy, development, and enthusiasts. I describe the book to non-designers as “the design manual for anyone who hopes to design experiences like the bridge of Star Trek’s Starship Enterprise.” The book was published in December 2020 by veteran design industry publisher Rosenfeld Media in print and digital. Read on for a case study of my experience as a a first-time author in the middle of a global pandemic.
Rather than simply define multimodal experiences as has been done in the past, my book draws directly from my myriad experiences designing for large-scale consumer and enterprise products to present practical case studies, conceptual frameworks, and techniques you can apply to your work starting today. Two full sample chapters are available: Chapter 7 (The Spectrum of Multimodality) and Chapter 9 (Lost in Transition).
An author’s experience
My year writing my book was an transformative exercise that will have long-lasting impacts on my ability to communicate with others. I’ve always been drawn to the written form, but working with my talented editor Marta and my advance reviewers gave me valuable insights about what differentiates my writing, how to further streamline and improve it, and how to communicate more inclusively when conveying complex topics.
Writing the first draft ran from September 2019 – April 1 2020. I would write and submit 1-2 chapters to my editor at a time for feedback, and we’d check in to talk about content strategy as well as any tactical guidance I needed to keep in mind. To the team’s great credit they gave me creative freedom throughout the process to interpret their guidance and shape the book in the direction I felt it needed to go. I welcomed the input and the opportunity to collaborate throughout the process – writing is a lonely art!
My research process ran concurrently with writing and included 10 detailed 1:1 background interviews with key industry insiders, selected carefully to ensure reasonable diversity of perspective while still maintaining breadth and depth of expertise needed. I also conducted extensive literary/digital research on all of the topics I covered and was personally responsible for all of my citations along the way, particularly in the forward-looking sections on topics like extended reality and artificial intelligence.
I was also responsible for all of the diagrams presented in the book – flow diagrams, etc. – and for sourcing all images in the book. While it’s not a visual design book, these choices end up representing a form of art direction and the establishment of a visual storytelling language. (It’s also quite time-consuming work!)
Revising during a turbulent pandemic
A particular challenge I faced was positioning the book in the turbulent year of 2020. My first draft was turned into the publisher on April 1, 2020 – just a few weeks after the entire world changed, and not long enough for any of the text to reflect the new world order.
The next step was to send the manuscript out for technical review to selected experts in the industry to ensure I wasn’t introducing inconsistencies and to challenge new ideas for comprehensibility and viability. Surprisingly, much of the initial feedback was strongly positive around the applicability and particularly the accessibility and readability of the text. The biggest theme was issues with the chapter on artificial intelligence – clearly my writeup was missing key opportunities and my understanding was lagging the latest advances. I took the feedback from my reviewers as an invitation to overhaul the entire chapter and to re-conduct my AI research, leading to a much stronger end result. If I had it to do over again, I think I could have sought out better diversity in my literary sources in addition to my interview sources.
But that AI chapter wasn’t the only overhaul. I felt strongly that I needed to set a new tone in the book given the unfolding world events like the murders of George Floyd and Breonna Taylor and the systemic issues represented by those events. I rewrote Chapter 1 completely, seeking to frame the work in the importance of issues like inclusion. I sought out an alternate perspective from a Black economist who consistently pushed tech to pursue more inclusive principles, and she graciously agreed to share her work. I also updated other portions of my book that dealt with tech ethics to reflect these concepts, and found opportunities to weave in the potential impact of the pandemic itself without over-indexing and sacrificing timelessness.
During this revision phase, as an author I was also responsible for identifying foreword authors and advance reviewers, working on second round of back and forth with my primary editor, finalizing all citations to the publisher’s chosen standards, delivering all final images for publication, locking down the subtitle and marketing copy, and partnering with the publisher’s marketing team on planning for the eventual release.
Reception
Some of my advance reviewers had this to say about the book:
The future is multimodal because we are multimodal. Here is an essential guide to unleashing the potential within that symmetry.
Bill Buxton, Partner Researcher at Microsoft Research and author of Sketching User Experiences
This book harnesses our very selfish desire to make cool things into the ability to benefit clients, stakeholders, customers, and society itself. Chock-full of insightful conceptual models and practical applications, designers will be both inspired and prepared
Sam Ladner, PhD, author of Practical Ethnography: A Guide to Doing Ethnography in the Private Sector and Mixed Methods: A Short Guide to Applied Mixed Methods
Using simple frameworks and questions, Platz creates a practical playbook that challenges teams to understand the problem, reflect on the design constraints, focus on people, and avoid the traps of exclusion in a multimodal world.
Jose Coronado, Vice President of Global Design Operations, JPMorgan
My book’s release date was always difficult – originally Election Day 2020, it was disappointingly delayed a month at the last minute due to COVID impacts at the printer that diffused some of the impact of my pre-arranged book “tour” dates. I’ve certainly learned that people have little cognitive room available for reading in the middle of government upheaval and a raging pandemic. Amidst the noise, the feedback has been strongly positive, with a 4.9 average on Amazon as of this writing across 24 reviewers. I also had the unique experience of seeing my own work and bias spiral diagram cited in an internal Gates Foundation design sprint report out from one of our Global Growth teams just a few weeks after publication!
Published Life: Applying the content
As many design authors will tell you, writing is only half the battle. My book “tour” continues and will likely run on an extended schedule due to the impacts of COVID. I also continue to book podcasts at a fair clip, and a wide variety are already available on topics from the book.
But as Dr. Ladner pointed out in her enthusiastic pre-read notes, the book presented multiple opportunities for expansion via talks, workshops, and take-home materials. I’ve been making good on that promise with a variety of content, all offered through my design company Ideaplatz:
- Capturing Customer Context – Chapter 2
- 30 and 45 minute talks adapted to both design and PM audiences (multiple conferences)
- 2 hour workshop (Interaction 21)
- Downloadable interview guide and printable worksheets
- Shared understanding workshop MURAL template
- Opti-Pessimism: AI, Design, and our Uncertain Future – Chapter 11
- Keynote, World Usability Day Puget Sound
- Multimodal Voice Design 90-minute masterclass (UxLX 2021)
- Multimodal Design Fundamentals 3-workshop series (corporate offering)
These projects have greatly strengthened my skills as a remote course designer and facilitator, yielding feedback from students like this from an international participant at UxLX Masters:
The exercises were the best of all other masterclasses I visited because it integrated MURAL. I really liked your lively style of talking. It was very refreshing compared to the other speakers. And the good examples from your experience about how hard certain things were, like thinking about the transitions.
UxLX Masters participant from “Multimodal Voice Design”, based on content from Design Beyond Devices
While writing may seem a singular pursuit, the experience of being an author requires a far more well-rounded effort that’s strengthened my writing, my collaboration skills, my presentation skills, and my ability to design coursework and take-home materials.
To purchase Design Beyond Devices: Creating Multimodal, Cross-Device Experiences, visit the Rosenfeld Media website or your local Amazon storefront (US link). My publisher Rosenfeld Media offers a free e-book with every print copy! Even if you don’t buy on Amazon, submitting an Amazon review after reading makes a huge difference.