Process + Brand = Experience

Our changing perception of ownership - 6 trends sparking the shift

We may own a lot of stuff, but I think we are starting to have a different view of what should be owned/purchased/bought versus accessed/licensed/rented. The end desire is the same - ability to use an item to solve a problem. But how you access that solution - direct purchase or access rights - may be different. And does that matter? Or do customers care more about finding an economic and sustainable way to solve their problems?

 

How we view ownership in society is changing. In many respects, ownership is the goal of the buyer’s journey, the sales process, and the main interaction people have with a company. Many look to “own” a solution to a problem. We make ownership seem simple - you go to a store (brick and mortar or virtual), buy something, and bring it home. But that’s not really what happens. Subconscious decisions occur during various steps of the process to determine if a product really solves your problem, and if the product has value or worth to you at its current price point. There can be many stakeholders involved in a purchase, depending on the item, how much it costs, who will use it, the type of benefits, and how it will be maintained.

We sometimes kid ourselves into thinking that money doesn’t matter when we are solving a problem. “Money is no object.” But it is. Money is often a deciding factor for a purchase or license. We may decide that the high cost of the solution makes the problem not worth fixing and reduce the priority of the problem in our lives. We may decide that it’s ok to live with the problem as-is, or do some more research and find a free or low-cost substitute workaround. In doing this, there is a chance that we may simply not be seeing how deeply the product will change our lives for the better. We walk away from the solution because it's too difficult to see a new way. We need help visualizing the change.

An evening gown is a great example of a complex purchase that may be influenced by money. A gown may not change your life, but it is a great example of a buy or rent scenario and the factors involved in the decision making around that.

If you attend formal functions often, it makes sense to own a gown. But there are a number of reasons why gown ownership isn't practical:

• Most of us attend a formal event a handful of times in a lifetime. We don't need one in our closet.
• A gown is usually based on high fashion trends. If you only wear one every few years, then it would go out of style before you could wear it again.
• You don't want to be seen wearing a gown twice by the same group.
• Gown maintenance and storage can be costly (depending on the gown, it could take up a lot of room in a closet and require a special dry cleaner to clean it)

For many, it makes more sense to rent a gown for an event and return it the day after. Rent the Runway was founded on the premise of reducing replacement and maintenance costs. They also rent designer clothes that you may only want to wear a handful of times before they are out of fashion. They solve the problem of wearing the right outfit for the right occasion and provide a different way to finance access to the solution. The alternative solution to renting a gown for a formal event is putting the gown on a credit card (some would return the gown the next day, as unethical as that sounds...it is a common option). Another solution is simply not going to the event. But if Rent the Runway does it right (and they have), they found a way to paint a picture of a future where you get the best of all worlds - even cost-wise.

As mentioned above, the cost of ownership is tied not only to the purchase cost, but the cost of maintenance, repairs, and replacement. There are also insurances, warranties and guarantees to consider. There are pros and cons to alternatives like leasing or renting once these other costs are considered.

As an aside….I’d like to clarify the definition of a product and/or service that a company offers. Sometimes we view products from a company's perspective - as items to be sold or items that we need to get someone to buy. From my perspective, a product or service should be seen from a customer perspective as a solution to a problem. The problem could be, for example, hunger (products in a grocery store), boredom (amusement parks, zoos, museums), or desire for knowledge (books, schools).

To clarify, if someone doesn’t buy your solution, it’s because it’s not a priority for that person. He simply found another way to solve his problem without buying from (or spending money with) you. 

Historical view of ownership
Ownership used to be related to investments and assets. People would purchase land as an investment. That land became a place to grow food and generate revenue from the excess harvested. Or they could sell the land because it increased in value as it sat there. Same with a house. Same with a horse. You bought a horse and had transportation, manpower, and wealth that could be sold to another bidder. Sure, you had to feed it and care for it, but those costs were far less than the value it gave providing transportation and other services (even the ability to reproduce and make more to sell). 

In some ways, clothes were an investment (they could be repaired and worn for years). Food was an investment. A printing press was an investment. Owning a book was an investment. Jewelry was an investment.

Then came along automation and mass production. Costume jewelry. Plastic plates. Things that were not made to last. Things that required maintenance, serious repairs, and possible replacement because they were temporary, created for fun, or just a handful of uses.

Over time, even cars were not seen as an investment. They required a lot of maintenance and repairs. And replacement. Things were no longer seen as investments or assets. They were things that cost money and declined in value the longer you owned them. 

According to this Financial Times article, businesses were the first in revising their perspective of ownership. Many stopped owning their offices and business spaces (e.g., supermarkets, airlines). They rented because it was perceived as more economical, flexible (they could change locations and not need to sell), and sustainable.

I see six trends influencing our views of ownership as well as our view of products and solutions and wealth/assets.

#1: Ownership isn't really connected to wealth and investment anymore.

As stated earlier, people in the past would own something because it was an asset with long term value. Buying a good was equated with making an investment, increasing in value over time, adding to your wealth.

With the rise of mass production and consumerism, we have witnessed that not all items we buy will increase our assets and wealth. In fact, some things we buy will decrease our wealth because of maintenance and repair costs. Again, cars are a great example. Many say that a car loses its value as soon as someone drives it off the dealer's lot. Then there are the maintenance and repair costs. After 10 or more years, there are replacement costs. It provides transportation, but are the costs worth it?

 

#2: Our definition of wealth and investment has changed.

What does it mean to be a wealthy person? Our definition of personal wealth is shifting from being based on money to personal experiences and balanced living. Trips are now being seen as a personal investment to broaden our views of the world and how we see ourselves in it. Education has always been perceived as an investment because you can expand how you see the world and yourself.

Sure, what is in your bank account matters when defining wealth, but that is not the only factor. 

The movie Citizen Kane illustrates this best. Although the movie was made in 1941, its message is still relevant, if not more relevant, today. Charles Kane owned a mansion, the largest personal zoo, incredible artwork, but on his deathbed, he wished to find his slide, Rosebud. He had this slide as a kid before he entered the world of wealth. To him, Rosebud represented his most prized possession - his childhood with his family. It makes you wonder if he saw himself as poor, although he had many things.

This begs the question: how do you define wealth? In some ways, your perception of wealth could be a reflection of your value system. 

 

#3: We are increasingly wanting access to things that simply can't be owned.

You can't "own" a song. You can write or perform a song. You can record a song for later listening. You can access that song to listen to it. But you can't "own" it. You can own the CD that the recording is on, but that song is not technically yours. Same with a book. Someone else wrote that book. You are reading it. You may own the paper it is on, but you don't own the tale or the thoughts or ideas outlined in the book. Those are from someone else. The person who originated the thoughts and ideas can provide access to you to read it, or keep it to himself. 

In a way, the digital world and the discussion around media rights is raising a solid point about creative work and ownership. It can't be owned. It's about access.

We are almost extending the model of the library. The library provides access to books that we return. With the rise of the Internet, we now have access to knowledge. But we don't own it. It's free, we can read it at any time, but we will never own everything on the Internet. However, we can always access it.

 

#4: The cost of ownership (maintenance and repairs) may be too high.

The term, "total cost of ownership," (TCO) is now commonplace. Businesses consider this when they purchase a solution. It's no longer a consideration of what's the cheapest solution - it's how much the solution will cost a company or person in the time that the person "owns" the product. It's a long-term cost view of ownership.

Rather than the car example I keep using, let's look at a house. If you don't go to the right appraiser or inspector, you could purchase a house with a faulty foundation or frame. There may be a leaky roof that even the previous owners weren't aware existed. Or the shingles could have been poorly installed. Or the gutters. There is always maintenance for a house, which is why many don't own and rent. Issues can range from structural damage after extreme weather to simple wear and tear to general improvements. There is even gardening and yard work that needs to occur.

Another example for TCO is clothes that require dry cleaning only. Dry cleaning can easily cost $10-15 per shirt depending on the dry cleaner. So a $100 silk blouse may easily cost $250 after 10 wearings.

This is why ownership doesn't always make sense. Wearing a dry clean only shirt could easily double, if not triple, the cost of the shirt. We are told a home is an investment, but in some ways, given that it is based on property values and market rates and how much work you put into it, is it? 

 

#5: Technology is changing so fast that ownership is no longer sustainable. 

If we look at how technology is changing so fast, we can quickly see how items are built to last a few years, if not only a few seasons. We get new technology to replace old, dated technology. We dread buying it because we know we’ll need to replace it. Sometimes, you can't even repair it if it is broken. Further, if you can repair it, the cost is prohibitive and it is cheaper to purchase a new solution. 

The iPod is now essentially dead (and iTunes is probably shortly behind) after only 16 years with the rise of the iPhone and other media devices. Another example is ethernet cables. We no longer connect to the internet with cables (for about 5-7 years now). Dial-up access is a thing of the past. The world is wireless. I predict that keys to homes will be a past novelty in 5 years - already homes are using combination locks rather than keys. And with biometric technology on the horizon, isn't that a better way to lock your door?

Cell phones can now be purchased using a monthly charge and replaced after 2-3 years - you turn in your phone and get a new one. I personally have an issue with this regarding wiping the data off of a device when you return it. However, options are now appearing to accommodate and better support sustainability and waste from the technology changes causing tech products to be obsolete in a handful of years.

 

#6: Time is valuable - owning may take longer to do than other means of accessing an item.

We believe a myth when we go to buy expensive goods: you go to a store or showroom, find what you want, give cash, take it home.

The reality of buying expensive goods: you research your options, you go to the store or showroom several times, you talk to sales people, research more, talk to customers, you research finance options, you decide on what you want to buy, you sign a bunch of paperwork, add in maintenance agreements and additional warranties, and then you bring it home.

Buying could include securing a loan, insurances, warranties, and more. It's not always that simple.

Again, the example of a car. Renting a car for a few days takes minutes. Buying a car is a multi-month process.

An apartment can be rented in days. A home takes at least a week to purchase.

Peer sharing is a new purchase and ownership/access method that is revolutionizing how we use solutions. Airbnb for hotels and Turo for cars are a few examples that only scratch the surface of shared ownership. There are new living environments like cohousing that have smaller living spaces to own or rent with larger common spaces so you can spend more time with your neighbors building community and socializing with them - not simply stay in your room by yourself. Some space is shared; some is for yourself.

To sum up these new models, there are 3 clear options:

• Buying - the item is your possession outright - and your responsibility.
• Renting - you can use the solution, but someone else worries about the maintenance. To them based on their values and how they operate, ownership has value.
• Peer sharing has you invested in the maintenance and share the cost of ownership during your time using the item. 

And I'm sure there will be new models in the future to accommodate new perspectives of ownership and cost.

 

Conclusion

How we view ownership is changing how we access, purchase, and use items. It is also linked to how we view worth and its value. Worth and value have intertwined definitions that are unclear yet related. It also merits its own blog post. 

How you view the value of something is linked to how you access it. Do you own it? Can you have it in your possession physically - can you touch it? Or is it an idea? Is this item going to bring you future wealth in some way?

Does ownership matter?
That’s the larger question. As a society, we grew to believe ownership was the answer to access to solutions to problems. But many solutions today in the sharing economy are reflecting trends that point to a different perspective. Maybe it is time to consider that ownership isn't the only last step for a buyer's journey or a way to measure worth. We need to expand our perspectives and consider all aspects of ownership, from maintenance and repair to replacement to stakeholder involvement to access to if something can even be "owned."

I'm curious what you think the future will bring us regarding new models to access solutions to our problems. Please add to the comments below!

In a world of options - 2 is never enough. The flaw of dichotomies and the need for greater choice.

Let’s face it - binary decisions are easy. Yes or no. Left or right. Vanilla or chocolate.

 

Life gets complicated when you add the maybe, the forward, or the strawberry.

 

But are all decisions truly binary? No. I think we, as a society, like to believe that to keep things simple, or at least seem simple.

 

Is this the right approach for UX or marketing? Or is this desire to create situations where there are binary decisions an accurate reflection of life and the world around us? 

 

In UX we constantly try to create experiences where users choose what's best from a select few options. We construct paths and experiences with the business to make the choices simpler, often creating questions where there is a yes/no answer or multiple choices. But sometimes I wonder if, when we do this, we are creating a too simplistic view of the world and forget to consider expanded perspectives? 

 

There are some features that always appear in designs. One is delete or remove. I notoriously forget to include delete options in my designs. I often refer to that flaw as my personal signature design approach that reflects my optimism. I always wonder why would someone want to remove something when we always march forward. I sometimes forget that we all make mistakes and need an eraser. 

 

Similar to delete, all designs tend to include some type of exit or overall cancel button. People sometimes want to leave a process and not finish it. I have had some curious discussions with business types about the need to include an exit button. They don't want users to leave, but I have to remind them that at times, users want out. Maybe have an autosave feature associated with the app so no data is lost upon exit (and a version control feature to access earlier versions). But what if the user really does want to leave? You can't stop them and need to consider these types of perspectives.

 

What's ironic about me working in UX, creating experiences with a few multiple choice answers, is that I was horrible at taking multiple choice tests. I saw all answers as being possibilities and had a hard time choosing a single best answer. I'd always look at a question from different perspectives and see all of the ways each answer could work. I know I'm not alone in this style of thinking. (And this is why I really don't like multiple choice tests.)

  

I sometimes wonder if by constructing these simplified environments, I'm helping to create oversimplified experiences with answers or choices and options that fit ok. Am I creating a world with answers and solutions that provide the "right" fit?

 

Maybe complexity isn't so bad?

In marketing we do something similar. At first, we try to help prospects figure out what they should buy and encourage them to narrow down their options to one or two. There should be a period where marketers ENCOURAGE customers to think broadly and consider a number of options, even some unorthodox considerations. Some marketers wouldn’t agree with this approach. However, going wide helps prospects define their problem as well as find a solution and explore all options. Rather than rush a prospect find a solution, that just happens to be your solution, help him find the right solution for him. By doing that, you build trust and breaks down false dilemmas that just don't serve any purpose.

 

 

What's the origins of binary obsessions?

I think our obsession with binary choices lies in our culture. Let's face it - we are trained at an early age to consider choosing between 2 extreme, opposing choices:

• Religion: good and bad or God and Satan
• Sports: two teams and we make them extremes (AFL/NFL)
• Politics: two parties - and we highlight their differences rather than similarities.

Our analytical mind responds well to the 2 options and somehow, we have a tendency to make these types of choices an extension of our identity. Because we are choosing not just the best option, but the best option that reflects who we believe we are, our choice is more of a reflection of ourselves and our beliefs. Our choice becomes very personal, and therefore very emotionally connected to our identity. Our actions around that belief and choice become emotionally driven - something to defend at all costs.

 

Is that really a way to make good, logical decisions? Isn’t this now more about emotions driving decision making? Or is it emotions and facts?

 

Being part of the "right" side

For my undergraduate thesis, I studied Edward Albee's plays. I noticed a trend in each one:

• In the first act, he defined a society with rules and added an individual who didn't really fit into the society or follow the rules
• In the second act, the society kills the individual (in the plays, literally. Or those who don't fit in are dismissed and told to leave forever.)

In Who's Afraid of Virginia Wolf? the individuals who didn't fit into society adapted; they got to live. I wonder if this was Albee's way of commenting on society's treatment of people who are different, who don't choose the majority choice. He chose to illustrate this using an us/them situation (which it often is anyway).

 

And an us/them situation where one is right and the other is wrong to the majority group. No one likes to be wrong - on the wrong side of history, wrong about their perspective, wrong about their opinion. Enter perceived dichotomies or false dichotomies - the perception that there are only 2 options. One is right; the other wrong. However, there aren't only 2 options. Ever. There are 2 choices that are clear. Or more. But if you take a beat and consider all aspects of a situation, you'll see how many answers exist to the problem. There are dozens of choices. 

They are merely perceived dichotomies – divisions we project onto the world because of our perceptions and biases. For example, in the U.S. we divide people politically as Liberals and Conservatives, but the terms are so poorly defined it’s easy to find examples of people who have some liberal views and some conservative views.  

 

There are other important alternatives for defining a person’s politics (what do you want to liberate? what do you want to conserve? how do you think it should be done?), but the convenience of the false dichotomy of liberal vs. conservative hides them from consideration. The convenience of binary logic blinds us from how poor a foundation for thought it can be.

-- The false dichotomy of false dichotomies, Scott Berkun

 

In this world, there are few real dichotomies. It seems that we use the false dichotomy to make life easier, but in a way, we are making life more complicated because we aren't solving a problem. I wonder if we are being presented with potentially 2 wrong options to avoid getting a real solution. Is it more for social reasons? 

 

 

If we identify with a side, then we are identifying with a group and this becomes part of our identity. It is at this point when things get dangerous. If we rely on our group and community to validate our beliefs and our identity (which is emotionally tied and driven), we are relying on their knowledge for certain issues.  

The key point here is not that people are irrational; it’s that this irrationality comes from a very rational place. People fail to distinguish what they know from what others know because it is often impossible to draw sharp boundaries between what knowledge resides in our heads and what resides elsewhere.

This is especially true of divisive political issues. Your mind cannot master and retain sufficiently detailed knowledge about many of them. You must rely on your community. But if you are not aware that you are piggybacking on the knowledge of others, it can lead to hubris.

-- Philip Fernback and Steven Sloman, Why We Believe Obvious Untruths, New York Times

Are dichotomies (false or perceived) the best approach to take? The circle continues...

 

To bring this discussion back to UX. During usability testing, a number of participants would look to me for confirmation when they selected a button. They wanted to use the tool in the right way, as if it were a right or wrong way. It was like the test was a quiz. This goes back to either/or thinking and being right or wrong and that being tied to identity.

 

Is the Yin/Yang representation more accurate?

At first glance in eastern culture, there seems to be a dichotomy with yin and yang. In reality, the yin and yang are complementary but opposing forces creating balance. It’s a different take on dichotomies. It shows that the two are necessary for the world; you can’t choose one without having the other. You don't understand good without experiencing evil. You need vanilla to understand chocolate. 

What's affirming about this perspective shows that there isn’t really a wrong or right. Everything just is. This reflects the beliefs in ancient polytheistic religions. There was a group of gods who acted like humans. They did good and bad things. They represented different aspects of earth and life. No one was perfect. They all just "were." There was no one better or worse. They simply existed.

 

This comes back full circle to how people don't like to be wrong. In a false dichotomy culture, they are (like the usability test). In a yin/yang culture, they aren't. If we look at the world through yin/yang, every decision is an ok decision. There are better and worse decisions. It's more complicated than the dichotomy, but it's a more complete and accurate representation about the world. 

 

 

Options appear in conversations.

 

We brainstorm through conversations. We learn about new thoughts and ideas through conversations. Ideas are shared through conversations. More than two options appear through conversations.

 

There are many types of conversations. I did some research on this a while back and based on reading a number of articles, came up with 6 major types of conversations:

• Transactional - request an activity to be completed
• Informational - share and provide information/knowledge freely or by request
• Decision Making - come to consensus for a resolution
• Influential - provide information, knowledge, insights and perspectives to help someone make a decision
• Relationship Building - create a connection through the experience of learning about the other person (shared feelings and thoughts)
• Collaboration/Brainstorm - collaborate to create a solution or a new idea

Transactional conversations are more in line with either/or or choose one of these options. The other conversations have nothing to do with choice. They reflect discussions that drive consensus or understanding or build relationships. I've found from my experience that most conversations aren't transactional; they fall into the other categories. We don't really have a binary world in conversations. 

 

I think this is why the next step in interactions is conversations and using voice commands. We'd not only have greater intelligence built into our devices and computing, but we'd be able to accomplish more with our computers at our sides. It would better reflect how we work - less transactional, less choosing the right or best option and more exploring ideas and building connections. Humans are creators. We enjoy exploring options and ideas. We need to do more of that - not narrow our thoughts to what's "best."

 

 

When we construct experiences, it’s ok to be complicated. It’s our job to help guide people to make the best decision to achieve their goals. Sometime this means we need to encourage people to think more broadly about their problems, their issues, their challenges. We may need to have a relationship building conversation with them, talk about emotions and feelings, the goals of the conversation and what we want to achieve. Not every decision results in a single answer or a metric. There may be more aspects to consider about a problem or a process. And we should start raising those options and making them available in the UX (but do it in a clear and simple way).

 

Let's move away from dichotomies, or rather false dichotomies, and move towards conversations and looking at all options and possibilities. In this world, nothing is good or bad. It just is. If we started seeing all options, maybe we'd solve more problems and be open to more solutions rather than choosing a team and picking a side and trying to be right. Even when we use our apps. We don't need to look to our app creators to validate if we are right. We are right simply by our existence.

Where's your head at? Why change can be easy or hard.

Where's your head at?  Basement Jaxx

 

Most people don't like change - especially changes to an app, Web site or product. Changes can't be controlled. They are made and you need to accept it, whether you want to or not.

Look at what happens when Facebook makes a change. Usually we'll see a number of posts from friends complaining about the change along with claims that they will stop using it because of these new features. 

And then that same person makes another 5 posts about politics, friends, family, cats, or other people's silly children.  

Recently, Apple announced the iPhone 7 and how they are removing the phone jack to support a wired headset device. Their perspective is that people need to move to Bluetooth/wireless devices already. I think that's a fair assessment in general given that Bluetooth has been around for a long long time and overall works pretty well. However, there's a catch - Bluetooth headsets work fairly well some of the time. They don't work 100% of the time like wireless keyboards. I know of 2 headsets that work pretty well; but most just don't. I have been on too many calls with people using Bluetooth headsets where they sound like they are 3 continents away. 

Technically, are we really ready?  

No. And Apple should have researched that more before making it's big decision (I would be surprised if they did research that before making the decision because historically, Apple rarely does user research. I have heard Apple employees say that innovation doesn't come from users and research. Horrified me a bit. I guess they hang out with Frog Design kids too often. But onwards!).

The issue I have with this phone jack change is the maturity of the technology. Does it mean I won't buy an Apple phone? Probably not. Does it mean I'll need to get a better headset? Sigh. Yes.

I'll accept the change unwillingly because I still want to use Apple products. But in a weird way, it's fun to complain about it, criticize them, and say how they should know better.

 

Does all change come with complaining? No.

I have witnessed UX changes that resulted in no user complains; if anything, people have used it more! In fact, I have worked on projects where we made a signficant UX change that results in increased revenue Day One of launch. Or people adopted it quickly at pre-launch from a tiny link on a page. 

So what's the differences between the 2 types changes: the Apple type where I'm unhappy about the change and need a workaround vs the other types that people adopt and love?

Before we address that, remember: people change because they want to change. You can't force people to change anything or decide to do any action. If your users don't want to do something in a new way - it won't happen. They will complain and find reasons not to do it. The drive to use your product needs to be stronger than the drive to not use it (the Facebook example - the drive to be connected is stronger than confusion about the new feature.).

But there is a silver lining regarding change. When people complain about a change, it's usually not about the change. It's about a related issue: transparency, insecurity, loss of control, etc. It is regarding something about them and their outlook on their own life. There are a number of articles about change and why it may not work and often, it's for personal reasons. It's where their heads are at regarding the change.

I complain about the phone jack because of the technology of it - or more accurately because I am too lazy to find a new headset. Which is true! 

 

Why are some changes successful? It meets these three criteria:

• The feature added was something that the customer wanted already. This is often the case for fast acceptance. If a customer has a particular feature or business fix in mind, he more easily accepts its implementation. It doesn't take long for that customer to jump right in and start using it. The new feature is a win for the customer.

• The feature was familiar (intuitive) to use. No training was necessary. This is also key for fast adoption. Someone can start using a tool right away because he doesn't need training for how to use it.

• Users are already thinking differently about how to use a feature or app. Sometimes people are using an app in a new way already. Or they have already thought about that feature and are counting the days for it to be implemented. The company is catching up to what the users need.

These are the traits that exist for every successful change I have observed. And by success, I mean seamless adoption and increased usage Day One and beyond.

Yes, this can and does happen frequently.

 

Why would people push back on a change? 

This has nothing to do with training or an explanation for the changes. Rejection of change is never about the actual change. It's about issues surrounding the change. People blame the feature or other reasons as to why it didn't work, but that's not the reason why someone isn't using a tool. People say they hate Facebook and still use it. People say they are sick of Apple but still buy the phone.

There are deeper reasons at hand as shown in an HBR article, a Forbes article, and this other article to explain change, but to sum it up:

• Loss of control. People had an understanding of how things worked before (or thought they did, which is a different problem for a different day) and now they don't. That's a scary place to be. Their world has shattered.

• Fear. This includes fear or failure, success, unknown, looking stupid, more work, unexpected impact, something new. Fear is a key factor for rejecting anything. And we're not talking about technology rejection because the technology simply isn't there. We are talking about a change that is valid and makes sense and it being rejected because...great question! There is no reason for the rejection. What is FEAR again? False Evidence Appearing Real. Yes.

• Unprepared to think in a different way. Change always requires a new way to think about the world. It sounds severe, but it's true. A change may include a different way to think about a product and how it works - and if you aren't ready to think about a product that way, then this new view won't happen and change is rejected. 

 

What are signs of rejecting an app or feature beyond the obvious of not using the new solution?

• Making fun of the new solution - who created it, who is managing it, what it does. Making fun of something is a coping mechanism to make it seem lesser and you greater so your ego can accept the circumstances. It's psychology.

• Raising constant excuses and roadblocks as to why a feature is a bad idea. And each roadblock is a reason that can be easily cured or resolved with a conversation or other item. This is about finding reasons not to do something, not trying to improve a situation.

• Blaming an individual for the change. I would blame Steve Jobs for features I didn't like on an Apple product. Or Bill Gates for Microsoft. Or Mark Zuckerberg for Facebook. Is that realistic? NO! A team created and implemented a feature. Not just one person wanted it - many people agreed with it and made it happen. Thinking that many people wanted a feature makes you feel small and insignifiant and possibly wrong. And who wants to be wrong? It's easier to blame one person for a change.

 

How can you reduce these challenges if you have a public app?

Usability testing and monitoring metrics. Get the feedback directly and fix the problem. A fairly straightforward approach. Users will be honest with you and let you know where their head is at with the change.

If you are implementing an internal tool and there is change how can you reduce this? There really isn't any predictor. Sure, there are things you can do - bring people along during the change management process, show them what you are doing, keep them involved. But even that doesn't guarantee that change will be accepted. People have emotions. Even if everyone agreed that the change was a great idea in the first place doesn't mean that when it is implemented that everyone will feel great about it. There may be insecurities, fears, loss. Those emotions need to be addressed during the process. It is less about the change itself and more how people feel about that change.

To sum up, change only happens when people (users) want the change as much as you do. And they may not want the change because of fear, they don't want to think about the situation in a new way, or they want to keep control over their own world. Change happens when people have made up their mind that a new feature or idea makes sense - or they have already made a decision that the feature or idea needs to be implemented now for them to be successful. 

Successful change is about where someone's head is at.  

JP Penney - a case study in customer research, or rather, listening

I will be writing about this more, but this is a great example as to why you really need to know your customer/user. If you don't understand how they think, you can't get them to buy. There are many ways to achieve sales, and some great ways to do it, but it's key to understand who you are selling to.

The irony here is that the CEO came from Apple, which is NOTORIOUS for not testing ideas. I've had many an argument with Apple employees about testing, research and user feedback. The thing is, products in Apple are designed for their users, who are a tech savvy group of people, similar in profile to those creating the products. One could say (ok, I'm saying) that they generally design for themselves - and that's super easy to do.

One article is on Huffington Post. I'm sure there are other articles out there, and most likely an HBR case study is in the works. 

Being successful and having sales is about listening and having a dialog with your customers. Gee - maybe I don't need to write a new blog entry about this - I already did!

 

How much data is enough? Enough to have a conversation with your users.

"Let's look at the data."

I wholeheartedly believe that you need to review data before making business decisions - actually I prefer to make decisions based on 80% data, 20% intuition. I know I'm stating the obvious, but sometimes people don't really look at the data before they make a decision. I know - that sounds weird, doesn't it? It's just as weird as some people who look at TOO MUCH data before making a decision. Yes - that's right - too much data. 

Looking at too many data sources can honestly cloud decision making. Analysis paralysis. We have all experienced this. Study after study to prove what we intuitively know is right and what we see as trends as being real trends. Almost like having research to prove that the sky can be blue. Unnecessary and redundant, and this type of practice prevents people from getting things done - or working - what we get paid to do.  

So how much data is enough to make a decision? This is a question I've seen at every organization. And the exact balance comes from perceiving data as the other sides of a conversation created from product development - there is the side of the company creating the product, and feedback from the industry and customers (or data).

 

I believe in learning by doing. Doing, or working, is the only way to make progress and it's better to do and fail than not do at all. Data, or feedback, should help drive doing. And feedback should keep the conversation going with product and be enough to feed the work. 

Too much feedback from a product is like learning about Christianity by being in Church and listening to a preacher rather than being in the outside world, being generous to one's fellow man and just living. Listening all the time is certainly not very fulfilling (why Church only lasts for a few hours one day per week), nor is it a way to learn anything. You only learn from experience. This is also why children have homework - it's nice that a child understands how addition works, but that doesn't mean the child knows how to add.

Feedback includes the audience's response to a product as well as knowing what your competitors are doing. Once you make something, you need to understand how people use it, what they do with it, why they like it (or not like it).

Types of feedback:

• Web stats/metrics
• A/B Testing (almost like asking a direct question to get quick, direct feedback)
• Talking to customers
• Surveys
• Competitive analysis (helps for trends)

In addition to understanding what your customers like is just as important as understanding which features the non-customers use at other sites. If you pay attention to the feedback, it really is like having a dialog with your users and prospects. They respond when you make changes - either they buy more or less, buy from you or someone else.

 

Web stats/metrics

This is probably the easiest type of user feedback a product manager or user experience professional can access to keep a product conversation going. It's readily available and can be reviewed at any time to help keep the work moving and respond to users. The metrics which are most meaningful:

• Heat maps - see what users click on
• Pathing reports - how users navigate the site
• Dropoff reports - where do users lose interest and simply leave
• Numbers OUTSIDE of metrics - purchase rates, call rates, sales rates - basically, any actions that the Web drives users to interact with the company in real time

You get a pretty good idea of what users do from metrics alone and are able to identify quickly what you need to do to keep that conversation going. There are never enough Web metrics.

 

A/B Testing

If you can't talk to customers directly and you have a specific question about which approach works better - have an A/B test.

What makes a great A/B test? I know there are a number of articles out there that address this and I may write an entry on this itself, but to sum it up, key areas to test:

• Language - new positioning/messaging, calls to action, how to phrase things
• Page layout - how does the presentation of the product make a difference in an interaction
• Colors and page elements - buttons, text color and fonts, headline colors, content colors
• Images - what images and angle resonate with users

You can never have enough good A/B tests (well, you can if they are not effectively planned out).

 

Talking to Customers

This is the best way to keep conversations going with users. You can learn about issues from metrics and surveys, but it is only from talking directly to customers that you learn the "why" of what they do. This is key to keep the conversation meaningful and get customers and prospects to use your product more.

How much data is enough here?

• Regular usability tests with customers (informal studies every other week; minimally 2 times per quarter)
• Focus groups/innovation games at least once per quarter
• Talk to salespeople regularly to get insights about customers

Salespeople are a great resource for B2B marketers - and these people just aren't leveraged enough. Even people who work in stores could provide great feedback about the types of people who come into their store. These people who work directly with customers know what will get a customer to buy, how to approach a customer, what resonates with them, which messages work. They understand customer motivations and know what will work and won't work.

You can never talk to salespeople or customers enough. If you could get weekly or quarterly feedback from sales and customers - you are doing great!

 

Surveys

Surveys provide helpful feedback because you can ask users direct questions, but they come with some risk. If you don't ask the right questions, you won't useful answers to guide your new projects. Again, feedback is about maintaining a conversation, and surveys give limited feedback to keep a conversation going. If you get a trend of confusing responses, you can't directly ask participants why they chose as they did - and that why question always gives the best insight. Without the why, you can only speculate why users gave unexpected answers - there is no hard proof. If anything you need a second study to confirm your speculations (this isn't like confirming that the sky is blue - this is confirming which insight or musing about the participants is accurate. With no data, the ideas of marketers is literally musing). To put it simply, the challenge of surveys is that surveys beget surveys.

Surveys also support an older view of users and customers - demographics. I have found that using demographic data to create a picture of users typically confirms stereotypes and creates new ones. It places a label on people that has no real correlation to anything.  

Here's an example - I read the article that outlines demographics about health issues across the nation. It has an interesting graph and shows some trends. What is disturbing about the analysis is the generalizations that are made about people who suffer from specific illnesses:

For instance, residents of counties such as East Baton Rouge, La., in the "Minority Central" segment are more likely than average to suffer from Irritable Bowel Syndrome (IBS). While this disease tends to afflict white Americans more than black Americans, a study funded byGlaxoSmithKline and RTI Health Solutions found a correlation between socioeconomic status and the disease. Those with lower education levels and income were more likely to be afflicted. While consumers in these areas tend to have higher-than average concentrations of blacks, and lower than average groupings of Hispanics, overall they fall into a lower socioeconomic status than our other segments, possibly putting them at higher risk for IBS. 

-from The Demographics of Health Care

Now, I'm not just disturbed by the corporate funding of the survey (that's a whole different story) - it's the correlation that was made between socioeconomic status and disease. Sure, that's a great jumping off point, but what does it tell me about a problem - nothing! It gives me no insight except that maybe diet or environment may contribute to this. Then again, the issue could be genetics. Saying poor people suffer from IBS is creating a stereotype at some level. Data like this to me, unless it is used for diving into further research (and that isn't clear from this article), doesn't create a picture beyond creating and confirming stereotypes. And this is feedback, but again, it's not working towards a conversation. It positions the participants as lab rats being observed rather than people who participate in the world and different organizations and systems.

Ethnographic data is far better way to position users. It gets to the motivation of the individual - the why. Why do poorer people suffer from IBS? Why do people do what they do? What is the motivation? Again, if you don't have the why and understand people's motivations, you aren't making conversation. Users don't make decisions only based on their economic background, gender or race - these may be factors, but often they don't make a bit of difference - they often make decisions based on their value systems. Understanding the value systems (motivations) of people is missing and so needed for a successful dialog and feedback for a product.

Use surveys sparingly - demographic data doesn't really give you why answers; surveys give answers to direct questions. Only use when you have specific inquiries that don't require a why answer.

 

Competitors and Trends

Trends are key to understand what's going on with competitors. A trend is defined as "The general direction in which something tends to move." General direction - that's key here. It's basically what is your competition doing? What's the new level of parity? To get an idea of the new parity, you don't need to examine 20 sites - 5-6 will do. You are looking for trends, and if 3-4 of your 6 competitors are doing the same thing, that means that either:

• They tested an approach and it works
• There is a technical reason they are doing what they are doing
• Their users asked for the feature
• They are just copying each other because it is easier

You can't assume that what you see in a trend is truly based on what users like - even if you see that trend across multiple sites. Users only use things based on their experience and what they are used to - almost like creatures of habit. New ideas take a while to get used to. Further, most users don't know what to ask for - they don't do this type of work for a living. They depend on us in product to create something for them and solve their problems - they only know what they don't like (and where they spend their money proves that). Sites will often copy each other with the erroneous thought that a feature or approach has been usability tested, when it may have just been the easiest way to do something quick.

Getting feedback from competitors keeps the product conversation going. 

Know 5-6 competitor sites deeply - more than enough to get information about a trend and what others are doing. More than that, and you are just seeing more of the same. It's always good to look at competitors and use their products, but spreading out interest just won't get you new information unless there is a market disruptor who is doing something in a brand new way.

 

Interpreting Data

Numbers may be numbers that you can't really argue with, but depending on your interpretation of them, they can tell very different stories. You can look at the number 5 as being halfway to 10 or half less than 10; it's the same number, but the interpretation of 5 is slightly different with different meaning. The interpretation of a number is always up for debate and can only be clarified within context. And this gets back to looking at too much data and what that means.

This is why there has to be some level of intuition in interpreting the data and keeping product dialog alive. The military is encouraging intuitive thinking:

The U.S. military also pointed to studies suggesting a sixth sense can arise from "implicit learning" — absorbing information without being aware of the learning process — rather than building up expertise through years of practice. 

-from U.S. Military Seeks Sixth Sense Training

And there is also:

Intuition is tactical – tactical meaning the reflexive and mostly quick reaction to a given event. Analytics is strategic – strategic meaning planned course of action. It’s easy to see which would save lives on live battles and accidents.

-from The Intuitive vs The Analytical Mind

From these statements, it is easy to see that some level of intuition is key to be responsive to user feedback and requests at a faster rate with product updates. If you get too analytical/strategic - then you won't get anything done and stop the dialog/feedback loop of product and use. Or worse, you respond too late because you are stuck creating a plan. 

Analyze the data enough to make a decision. And yes, your intuition at some level is enough to give you insights to make changes. Spend enough time to understand what users are "saying", and then react. If you are wrong, you can always make changes again - at least you tried. Creating the "perfect plan" for users won't get you anywhere.

 

There is a delicate balance of intuitive and analytics/strategy when keeping a conversation. With conversations, sometimes you need to just respond with your gut and intuition; sometimes you need to think about what you need to say in response. In all cases, you need to be informed about a subject to have a meaningful conversation and respond intelligently (not fantastically). However, in all cases, your response needs to be timely to keep that conversation going. Taking a step back to look at the data and analyze the facts may be a good strategy if things are disastrous in a conversation, but generally, you want to respond promptly - or else a conversation won't continue.

80% data and 20% intuition keeps a solid conversation going between products and the market (competitors and users). It allows you to get data and then work to respond to what the data is telling you and keep a product evolving. Like a great conversation, each side needs to be informed (the data) and contribute (work and interaction). In today's competitive world, there is no room for analysis paralysis and carefully made plans that may miss an opportunity. You really need to trust in the intuition required in successful conversations.

Published on UX Magazine's site

Here is the article that I wrote at UX Magazine (published just before the holidays). Enjoy!

http://uxmag.com/articles/usability-testing-includes-users-as-stakeholders

What is a satisfying usability test

I was in a test where a fellow UX professional wanted "meaty" feedback. I was curious as to what she meant by that. I thought we learned after observing and questioning just 2 users that checkout paths are pretty much commodities -- users knew approximately what to expect, they wanted better ways to do things like split shipping and they wanted to have a clearer breakdown of charges. A couple of times, users told us that if they didn't know the real total of the purchase, they would have abandonned the site and have just gone to the store. To me - those were huge findings mainly because they covered the little things that honestly push users just over the edge to stop shopping and just give up. Basic, but huge. To me that meant - make this as close to everyone else as possible so I don't need to think (and if we want to stray from that - we need to rethink the process and review it with them - they may not want that).

Are we always looking for the big revelation? I hate to say this - general experiences for usability are pretty - well - common and straightforward. Sometimes, almost intuitive. I guess this comes to the question - what are we learning from usability? Are we looking for those seemingly tiny pebbles that turn out to be boulders or are we looking for the big ah-has? Are we looking for problems or are we getting insights into what makes these users tick and do what we want them to do?

So back to my question - what is a substantial result of a usability test and considered to be satisfying? From my perspective, the goal of testing is learning how the users think - not just about the to-dos in the experience, but what they are willing to tolerate in order to achieve their goals (in this case, buying something). I see testing as an opportunity to try to understand what matters to users (what's hard and easy), and that is what gives us insight into what's usable. I want to leave a test understanding what will push a user to walk away as well as what will keep a user fixed to the screen. Mainly, I want to understand what makes them tick.

I think it also matters what the background of the user is. Is the user a frequent buyer? Does he use a lot of sites for the same thing? What does he expect? What has he seen that he likes? Today, we are meeting users who are generally highly experienced and competent on the Web - it's not new, it's part of their daily lives. Just looking at how information gets distributed for news and notices, we know people are online - a lot - and have very strong opinions about what they like and want. Usually, the devil is in the details - a slow page load time, a cumbersome way to do something they have done before elsewhere. Those are the things that will make them say, "This site sucks."

Then we come back to users knowing what they know. They don't develop products. Most likely, they won't give you an innovative idea or way to approach a process. This isn't meant to knock users - it's just not their job and they don't think that way. To them, it's about comparisons - Site A does this well, Site B does this poorly, Site C is just cool. They don't examine what goes through their minds when they buy a shirt - they just go online and get the shirt or go to the store and put down the credit card. if they have problems of any sort - then they abandon ship and the business loses the money. They don't examine the details of why there is a problem. There just is.

Sure, you could give users a survey to get their feedback on what could be interesting, but does that make a difference? On surveys, a person's commitment to give an accurate answer is dependent on that person's investment in the survey. Most times, people give a quick answer to get the job done and get their reward or people can give an answer to improve a business or hotel or destination they frequent (they are invested in seeing their input used). In testing, we need to get inside their heads and figure out how they are thinking and making decisions. As UX professionals (and we all know this), we need to figure out how users may approach an offline process online. Does a general usability test and review achieve this? Sometimes, it does. Often, we're back in the land of comparisons.

I guess if the goal of the test is to get that new insight, you need to present the users with the innovation you want to make - that big ah-ha! It's not up to them to give you that feedback. If anything, you should be able to get that out of the discussion. Questions and comments like these should be commonplace:

• Question: What are you thinking when you enter in your credit card number?
• Question: Should the site have this already because you logged in and saved this or used this 800 times?
• Comment: I want control of what I enter so don't pre-populate it and it's creepy for you to store that
• Comment: This is just a necessity - let's get it over with

So back to what's meaty. I guess meaty to me is getting inside the user's brains and getting feedback as to what they want to do and what's easy for them. If you want to get that big innovation - you have to have it on the table. It's up to the designers and product managers and developers to come up with that based on the feedback. The users will tell you how it compares to what they have used and how they process that flow of information. I guess meaty comes from how you listen to the users.

User as removed stakeholders - we gotta fold them into the process

I came across this blog entry the other day, which was pretty interesting: http://www.measuringusability.com/blog/self-reporting.php

An article like this one is awesome - it backs up the movement for UX professionals to give users a more active voice and role in the development process. Users always have great feedback - but it's hard for them to voice their concerns about a site because they are directly experiencing it. It's like why some people will see a therapist. A therapist gives a third party, unbiased perspective of the life you are living. UX professionals give an unbiased perspective of the experience a user is having.

Sure, this makes the UX professional the middleman, yet again. But at the same time, with this unbiased perspective, it almost makes the UX person be like a UN representative - part of the group of users, but removed enough to have a more unbiased perspective of what they want. Well, that is given that the UX professional hears the user's voice.

We need more direct user feedback - not less. And if we can get direct user feedback that covers 50-75% of the problems at a site per iteration, then honestly, we're doing pretty good.

The other core problem with user feedback can often be how they communicate it - they need to be trained. Software development professionals have their own vocabulary which often doesn't mirror the users. But even if we in a product team just get raw feedback more often, and don't overly filter through the responses, we are getting more valuable information than if we didn't ask a user any question at all.

There are two sitautions where I see direct user feedback failing. If someone is doing user acceptance testing on an application, and he wasn't directly involved in the requirements gathering, will he necessarilly know if he is seeing a problem or if the system was designed to be that way? I think that is a challenge in general with working with users in unfinished applications (not a bad thing, just something that needs to be considered). How would a user know if something was built correctly or not? And if the real purpose isn't clear, then is it really not working as expected? More of a metaphysical question, but I hope you see my point.

Also, users have a high tolerance for dysfunction if they have a need to use a system to complete key tasks. I have seen this directly in usability testing (just the other day, someoene who does online shopping said they would continue in a painful shopping process if that meant she could buy what she wanted) and have some links floating around to help prove this, but in general, it's like people having a high tolerance for pain if they need to get something done. So if a system developed is critical for users to complete a key activity, then the users will tolerate a slightly less usable application, just because they can basically do what is needed. And users may not probe themselves to troubleshoot - they will find workarounds to get done what is needed.

Even with these two concerns (and I'm sure there are more as I think about it) - the users need to speak directly and have a more active voice during the process. Whether they have a "therapist" or UX professional to interpret the results, they need to add their voice so the team knows they are designing and developing and prioritizing what is needed.

 

 

Innovation Games - Get Your Team Communicating

So let's say you go to a requirements gathering meeting. You sit down with some internal stakeholder and hear the complaints and issues and try to understand everyone's feedback and perceptions. After about an hour or two or more of discussion, the group seems to gradually stop talking and winding down the meeting. It seemed to go well - you got a lot of information, so why complain?

Then about a week later there are changes, which are expected because business priorities change. However there are some new issues that, well, have a massive impact on the project and scope. Then you are sitting there going - "Man, couldn't we have learned that all a little earlier?"

Let's face it - talking is easy, listening is harder, but real communication is just plain difficult. How often are we in conversations where we say something, we hear the other person's response, and then a little later we wonder - what did we just talk about? It's hard to get out there what's important to you. And I think it's even harder for users of systems - users know what they want, but they don't write requirements for a living so they don't communicate them in terms of stories or bullet points or just plain needs.

So how do you get those requirements communicated?

Well, Luke Hohmann has a way with Innovation Games. I attended a two-day training seminar to learn how to get the right information out of people using games. Of course, we played the games too - probably the best part.

When we walked in the door, we designed a name tag. Well, using Mary logic, I wanted to make it pretty and girly. So I found a pink Sharpie (I didn't know that they existed), some glitter (ok, lots of glitter), and drew swirls and all that. I was honestly at a bit of a loss when I went to present my card (everyone had a great story, while I just wanted to make a pretty, girly card) - but man, I learned so much more personal information about the people there than i planned - it was awesome. I'm honestly bad with names (I need to meet you 2-3 times before i remember what your name is) - but at least I could remember people as "You are the guy who is from Canada" or "The guy with the eyeball on his tag."

By nature, I'm not a talker (you would never know this from my lengthy blog entries or from the fact that in my second life I'm a belly dancer - you'd think I'm extrovert extraordinaire...but it's true.). I'm happy sitting in the back of the room saying nothing for 2-3 days. Hell, I'll say nothing while working at home for a few days except for some IM conversations. But in this class - that won't work. You can't hide - Luke makes you get involved and get active.

I liked the case studies we had in class to learn about game application - they brought to mind some situations I was in where I had to get information out of people. For example, there is this game "Speed Boat" - a great way to have people talk about their "anchors" or complaints. Everyone loves to vent - making it a great game. Then there is product box where you get all scrapbook-y and pull together a box of what you think the product is and how it should be sold. And then a timeline game where you get people to list all of the times and events they use a product. And so much more.

It was an exciting couple of days - we all got involved and really learned more about how to use the games.

I mean, using games provide people a way to communicate their priorities, what is meaningful to them, their complaints, their relationships and heck - just get it all out on the table so you know what you are dealing with. What I liked most about the games - no more than 8 people can participate at a time. This makes the whole process more intimate and lets even the most introverted participate - I mean, it's pretty easy to get to know 8 people and a moderate just can't miss you if you don't talk. Heck, I was even talking a lot for me by the end of the class.

I can't wait to use the games with product users at usability testing or general sessions. Sure, it gets a little "arts and crafts-y" at times, sure it's super casual - but that's the best way to get people to open up, lose their guard, participate and communicate. This way, people are out of the lab and involved in their environment and discussing what really matters.

That's what these meetings should be all about, right?

 

Iterative Usability Testing or the Usability Hokey Pokey

Jakob Nielsen admits in his article, "Why you only need to test with 5 users" that 5 test subjects can find 80% of the usability issues; 15 users will find all issues. I'll be honest, based on my experience, that extra 20% of usability issues are usually so minor that they don't have an impact on the main flow. So if 5 test subjects find 80% of the usability issues, why do we keep having tests with 10 people or more? Sure, we want to find as many issues as we can with an online application or site, but why be so complete right out the gate?

A huge risk that should be considered during testing is what if the system as designed just generally doesn't work - people don't get it, they don't like it, find it problematic, etc. and you have to redesign it entirely. At that point, knowing about misspellings and incorrect page space usage doesn't matter. So let's say you have a very complete test group of 15 as Jakob notes in his article, and you find out that the general design doesn't work. You go back to redesign your application. In theory - you should test it again (most times, this does not happen. But ideally, it's a must happen because it is a new system and it's not tested - it's a result of a failed system in the first place). But there is the next risk for the UX - will the business fund testing with another 15 users?

Let's say instead, you approach this as an iterative test as Jakob suggests. You do some work, create a prototype or some type of sketch to get feedback and show it to 5 people. Let's say it's a bust and you go back to the drawing board. You do the same thing again and test with 5 more people. Let's say this time it works, but needs some work. You refine it and then test it again with 5 more people. Let's say there are minor problems to fix. You could do at least 1 more test (if not 3 more) based on the original budget for testing to capture all usability problems. This is far more efficient and effective to product a UX that users will find helpful and they will use.

Every time you make a change in the test, in a way, it's a brand new test because there are different risks and factors in the design that need to be reviewed. I think this is a better method for testing mainly because you can experiment and get constant feedback from your users. Also, you start treating users as more of a stakeholder in your project/product. Right now, users are treated as a visitor - someone who comes in and uses what you made here and there. They usually are involved only 2-3 times in a project. It's up to the UX professional to know their needs or raise questions to get the business owner to know the needs. In a way, that's just a ridiculous notion - we aren't mind readers and we need the direct input of users - just a better communication chain.

The best way to get feedback from users is to include them in reviews, in the same way that the developers have reviews with the business owners. This is the 5 person usability test process. I wouldn't suggest having a board or panel to do the reviews regularly - you could do that to help get new features to add, but usability works best with new eyes. Automated testing with thousands of users is nothing more than A/B testing - and in a way, that is measuring preference (I'll be writing more about this later - but this is also a type of usability that can be tested). Doing an online test for the initial testing sessions is more than sufficient. At those stages of testing the goal is mainly to understand if the flow works or if people can find what they need to complete a task. If you feel the need to have in-person sessions, I would suggest to leave it at the end when you want to see how the user responds to the look/feel of an application. But in general, if you can see the face of a person and their facial expressions, have that person test on a computer he plans to use at the time of day that person will use it, then you should be ok.

Basically to get requirements from users you need to do the hokey pokey - put a hand in (test), pull it out (refine) put a foot in (test), shake it all about (larger test or maybe bigger test). This is probably one of the easiest ways to get the requirements out of users and have a successful product.