Research in the Real World: Talking with Frank Lockwood of 360i

I talked with Frank Lockwood – Group Director, Experience Design at 360i – about his entry into UX, his methods of research, workflows and strategies for achieving true user-centric experiences.

How did you get into the UX field?

My degree is in Visual Communications (Art Direction). While I was in college I taught myself front-end development and started my career as a double hatted designer and front-end developer. Over time I learned more skills in the UX area of work starting with business analysis, user research, user testing and content strategy.

I probably wasn’t aware that I was heading into the path to be more of a UX professional until I realized the rapid expansion of the field and the need for people with my skills. Once I realized I was actually doing a lot of UX activities I made the shift to move my career as a UX specialist.

I’m currently in a user research class at Pratt. Which types of research do you typically conduct, and how do you make sure you’re getting valuable information from users?

It really depends on the client we have and what’s their level of interest in user research/investment in it. We do a lot of different studies, but for all accounts we work on we do a bare minimum of personas and journey development informed by secondary research, media segmentation, web analytics and social listening.

For more robust projects we leverage a lot of unmoderated testing through services like What Users Do/ Our screeners are informed by upfront research or understanding the communications strategy of who we want to engage and will be driving via the media execution. We develop our testing plan by establishing a business goal like product conversions, newsletter signups or communication goals and ask a user to attempt to achieve the goal. Along the way we will ask them questions about the experience to understand what’s best from a heuristics stand point.

We also leverage rapid prototyping for wireframes/designs through usabilityhub and will do hallway testing internally to get quick feedback.

We’re currently undergoing a user research project where we have phone interviews with actual client users to gain their feedback on the brand, their needs and daily habits. We procured these users through a newsletter and website screener that offered up an incentive if they were selected for the study.

When you think about your last project, what was overall design of the user research? Do you follow Exploratory, Generative or Evaluative processes?   

I have a million projects going on so we have several different approaches. The one mentioned above is more of an exploratory process. Collecting feedback on what users may have expected or get out of the competition that we’re not offering.

With another recent project it was more of an evaluative process where we did preference tests with different design layouts and iterated them over each test.

We flex to what the budget and timeline allows for.

How do you effectively analyze and synthesize the data you get from user research?

Depending on the method, we typically track feedback in spreadsheets and leverage tabs to slowly pair down what we hear consistently and prioritize items that strike highest across all users and test sessions/interviews.

Other methods are to hold business priority workshops with our clients where we expose to them the results and work with them to create a backlog of fixes based on the user and business need.

What is your favorite part of the work you do?

My favorite part is when I get to work on a product that provides true utility for a business and a user. I do enjoy design but if it’s the only focus of building a web experience it’s more subjective than being a true user-centric experience. Design is one small facet of delivering something that’s truly useful, but for many people is the easiest thing to understand about UX so it often takes the front seat of the term “UX”.

The way you get to a real user-centric product is by having an active Discovery and Definition phase where you clearly set up a strategic framework for the course of the project. That’s probably the area I enjoy the most, however I love to stay involved and see my babies come to life.

Dovetail – A Diary Study Tool

The logistics of setting up an effective platform for a diary study can be difficult. You need to provide an easily accessible means for participants to submit their entries, stick to the schedule outlined in your planning stage, and organize your results so you can quickly analyze the data. Dovetail, a free webapp released late last year, is a great option to consider next time you’re running a diary study.

The nice thing about Dovetail is that it’s minimal design. You can tell right off the bat that it was designed with UX people in mind: simple onboarding, straightforward usability and solid performance all delivered in a casual, hip experience on par with the industry standard.

After you sign up for an account, you land on your dashboard where all your projects are kept. A sample study is provided (with pre-loaded questions and timeframes, etc.) as a means of showing you around the interface. I went ahead and created a new study. It’s really quite as simple as:

  1. Naming your project.
  2. Setting a time frame.
  3. Choosing between email or sms as the delivery method
  4. Writing the questions.
  5. Adding participants.
  6. That’s it!

Dovetail automagically sends out prompts to either the email addresses or phone numbers you provide at the times you specify.


The messages are have very clear directions–the participant simply replies to the email they received with the answer to the prompt, and the response is sent back over to the Dovetail interface to access and analyze.


Honestly the analysis functionality is almost too minimal. You can “star” your favorite responses, and add different colored tags to the responses. That’s really it. I can see the tagging system becoming helpful down the line – but it might be an extra burden to have to formulate your own folksonomic controlled vocabulary for your study. Do you tag by key words? topic? emotions? all? That is the one feature that I could see becoming easier with prolonged use of Dovetail.

The easy export to a csv file is an added plus to make it easy for cross platform functionality and visualizations for your analysis and reporting.

I’ve not had luck testing the sms – I specified a time to receive the message, and that time has come and gone. I assume it has the exact same wording as the email, and functions similarly.

Once I trust that the delivery features are trustworthy and punctual, and I have a better idea of how I would want to tag and organize responses, I’m pretty confident Dovetail will be an extremely useful tool to use for diary studies. It makes reaching the participants in-situ a breeze, and offers an intuitive, enjoyable means of administering the diary study for my participants, and myself.

Reticles, Controllers and Buttons: Common Interactions in VR


Ivan Sutherland’s Sword of Damocles VR apparatus – 1968

Virtual reality has stubbornly withstood decades of technological limitation and continual market failure to finally reach the threshold of public adoption. It has the honor of predating the first GUI, has successfully captured the public’s imagination and has undergone several iterations of consumer products. With the technology inside our smart phones giving us the opportunity to dive in to VR and experience its potential as a medium, it is becoming more and more important to design interfaces that afford users control. Just as GUI’s afforded novices with an accessible means of interactivity, so too must there be a framework for users to interact naturally with a system in 3D. Specifically, lets take a look at how controllers, both physical and virtual, are designed to make interfaces more usable.

General Design Guidelines

The inherent disorientation of entering a virtual world makes it all the more necessary for users to feel comfortable and in control. There are now common guidelines recommended by many including Google and Oculus which encourage developers to be very deliberate about scale, spatial audio, freedom of movement and visual cues to make users comfortable. But how do you interact?

The Reticle



With the incorporation of a small circle in the center of a user’s field of vision, one can simply use the power of the gaze to interact with surroundings. Although there is a somewhat diminished sense of immersion, the ability to simply look at an object and interact with it is natural and satisfying. While some interactions require the user to aim the reticle at an object and click a physical button on the head mounted display (HMD) or controller, other interactions involve “look and lock” or “fuse” buttons. These eliminate the need for a physical button, but also take more time for the user to engage the button.



The usage of the reticle is also the most minimal incarnation of a heads up display (HUD), wherein layers of information and interactions are overlaid on a plane between the user’s vision and the environment, much like the the interface for a first person shooter (FPS) video game.

Physical Controllers

While interactions with the reticle are simple and natural, they simply lack the deep functionality of a physical controller. Some are relatively simple, like the recently released Daydream View which comes with a minimalist two button “magic wand” controller. With this controller, users can point to objects on the screen for basic point and click types of interactivity.

Vive controllers

Vive controllers

More complex controllers more powerful with multi-button, tracking controllers for both hands. With two controllers, using your hand in tandem opens up the possibilities for gestural functionality, which more closely mirrors the actual manifestation of your hands in VR. A very compelling use of these controllers can be seen with Google’s Tilt Brush application. Click here for a demo of a user showing how the controllers function with the creative app.

VR Buttons

While there are physical buttons on controllers, buttons existing inside of the VR environment might advance far enough to eliminate the need for physical controllers whatsoever. A big challenge for VR buttons is proper feedback and a appropriate feel. One designer devised a button that emulates sensation “akin to to submerging the surface in a liquid to change its state.”

Credit Mike Alger

Credit Mike Alger

This is one single button and interaction idea that seems to be pleasurable and practical, and which takes advantage of the VR possibility to push through a plane in 3D. Although it offers no feedback physically, and is possibly a bit too skeuomorphic, there is a certain elegance to its design – it asks to be pushed.

Now imagine an array of buttons similar to these that appear when we put on our VR or AR HMDs as parts of our daily lives. There may be a time when our productivity will be closely linked to interfaces displayed our virtual world or augmented world. Instead of a desk with a keyboard, monitor and picture of a deserted island, you have a deeply customizable 3D environment that offers much more interactivity than with our current 2D processes. Designing these interactions and experiences will surely propose difficult challenges to the UX and UI designers of the future.

Our new workspace?

Our new workspace?


Design Critique: NYPL’s SimplyE App

SimplyE Home Screen      IMG_3565      My Library

The New York Public Library (along with most libraries) has wrestled with the challenge of providing an intuitive process for lending e-books. To bridge this wide gulf of execution, they recently released SimplyE, a mobile app that aims to drastically minimize the steps needed to fulfill the goal of accessing e-books from the near 300,000 in library’s catalog.

While SimplyE undoubtedly improves upon its predecessors, there are a few usability issues worth considering in context of Don Norman’s principles in his Design of Everyday Things.

Lets take a look.

Problem 1 – Ready to read? Great! Enjoy the End User License Agreement.

IMG_3559       IMG_3560      IMG_3570

First impressions are important. Upon opening SimplyE for the first time, a license agreement is presented in its entirety for the user to review, immediately demanding a significant cognitive load. Ethical arguments withstanding, signing off on agreements to gain access is perfunctory nowadays, yet SimplyE violates a common convention of easing its users into through the process. If one chooses to reject the agreement, a message informs the user that they must accept to proceed, essentially forcing users to anxiously reinterpret their decision.


Documentation of the app’s rules is essential for transparency, and encourages overall understandability of the its design. Instead of introducing the app to users with a decision requiring too much knowledge in the head, an incremental on-boarding giving users a glimpse of the app along with the opportunity to review the terms, and at the same time an opportunity to register.

IMG_3571       IMG_3574       IMG_3572

An easier (if less blatantly legally transparent) introduction on the Evernote app

Problem 2 – Slip Slidin’ Away

Once you arrive at the main screen of the SimplyE, available e-books are grouped into categories like ‘Best Sellers’ and ‘Nonfiction,’ some with nested sub categories. With identical sequences between different kind categories of e-books as one digs deeper, users are bound to commit description-similarity slips when, let alone be relatively confused as to which hierarchy they are at. Notice how difficult it may be to distinguish between these different levels of the catalog:

IMG_3578      IMG_3579      IMG_3580

The same problem persists for the formats of the ‘My Books’ and ‘Reservations’ tabs. If one removes the title of page at the top, its difficult to discern difference. Even more description-similarity issues are visible for buttons on both pages. The buttons are extremely similar, but do very different things (one downloads a book, one reads it, the other removes it).

IMG_3581     IMG_3582 (1)


To help mediate the slips that result from seeing identical layouts for different pages, the simple cure is to simply distinguish one category from the next. The Spotify app has unique icons which describe different genres, and unique imagery that the user sees when tapping on a genre. As a result it is simple to discern between ‘Chill’ and ‘Pop’:

IMG_3586    IMG_3587    IMG_3589

A similar strategy applies to the layout of the ‘My Books’ and ‘Reservations’ tab. For the buttons, a change of color or shape to distinguish each unique function of the button would help to ensure that mental slips are minimized.