One of my favorite things about being an entrepreneur is that every year the center of gravity of the company changes and so does my focus. So every year or so I change hats, from marketing to product, from sales to management. The last year has definitely been ‘the year of the product’. I changed lots of my current product management perspectives and finally said goodbye to some methods that should have been long gone.
Why using personas and interviewing users can lead your product to fail?
One of the main frog leaps I had in marketing was understanding that people are complex. Much more than we can predict. There’s no way we can actually place them in boxes.
Marketers love spending days (and nights) sharpening the messaging, picking the right words to tell the story of the product. The basic assumption is that people want a product which will help them to do X. The reality is that people act from much deeper motives – and it’s usually not just what they need.
In marketing, you don’t really have to understand what makes people tick. There are A/B tests and the ability to run multiple experiments, even with limited resources.
Product managers are not as fortunate. Every feature is too “expensive” and we have to get it right as soon as possible.
For those of you who don’t know “Persona” methodology well – This is one of the most common PM methods to define a product or a feature. The idea behind personas is to come up with a few ‘imaginary’ people who represent your target audience and to create a profile for each one. When designing the product, the personas are used to understand how features will be used and how people will react to them. A persona can be, for example –
Mike is a Hipster graphic designer, he’s 33 years old. He lives in NYC, earning $70K/ year, commutes on the Subway where he listens to podcasts. He has a vegan diet (but not too strict about it). He usually buys eco-friendly products…
The problem, in my opinion, is rooted in the attempt to oversimplify people and real-world problems. Even if the PM will write a 10 pages description about Mike, it will still be far from capturing who he really is and what affects his decisions. We should define our target audience and their needs, but, we can’t ‘program’ or ‘invent’ our users.
How do astronauts scratch their nose?
This is a great product example. In astronauts’ space suits there a small, simple piece of scotch on the inner part of the helmet. It’s used… Well… you guessed it already, to scratch their nose. It makes sense that inside a sweaty suit, your nose might start itching. And nothing can be more upsetting than an itchy nose while walking on the moon. No persona would have an itchy nose in its description. On no interview with a future astronaut about his needs, scratching one’s nose would have been an issue. But, when viewing a real person in this scenario for a few hours, no one would have missed this need.
The most significant observation I had this year is that product managers keep ‘inventing’ people when they’re surrounded by real ones.
If Personas is wrong, what’s right?
I try avoiding playing imaginary games as much as possible and design my product based on real data. I’ll share more about the process using Oribi as an example – we develop an analytics tool which helps companies easily understand how to improve their main KPIs. We have a few types of potential customers from different company types and sizes. We could have easily built Personas. Instead, we asked 20 potential customers to install a small script we developed to extract all their data. We started monitoring every piece of relevant data – how often it changes, what are the most interesting insights and which parts don’t provide enough value.
The path to a good product goes through endless Excel files
Every feature we decided to develop had to go through ‘the real data test’. For example, if we thought about a trends feature, it was built based on data from dozens of sites where we manually identified what usually caused trends, which indications are actionable and which shift actually mean something for the users. The prototyping process usually starts by manually improving our data. We take our basic feature design and start ‘pouring’ some real data in. Then, we’re able to see what doesn’t work well and which important aspects we didn’t think through. We make some changes to the product and run the data into it again. The final algorithm we use is some sort of ‘reverse engineering’ of the manual work. For example, instead of deciding that we’re going to alert users on every change higher than 10%, we’re going to look at the data, see which changes really matter to the users and based on that set the alerts threshold.
Products’ failure can be rooted in building them on an artificial environment. When designing for imaginary people with predicated needs, going out to the real world can be overwhelming. While I personally love low touch and high scale products, I think there’s a big advantage when building Enterprise products. Enterprise products are usually based on tight interaction with ‘real’ customers from day one, with real needs and a real character. They might not represent the entire market but working with them usually leads to at least an OK product and not a failing one.
And what about user interviews?
The devil probably invented the “So, what other features would you like?” question. If people love your product and desperately need more features, you’ll hear about it. If they don’t, adding more features is probably not the answer.
We speak a l-o-t with our users. all the talks are with existing users. It ranges from users who adore what we do to users who stop using the product after their first session. We focus on what worked well for them and what didn’t. We don’t ask people who never used the product what they’d like. Over the years, after hundreds of talks to users and potential users, I can say for sure that people enter a different state of mind during interviews, they usually tend to say the ‘right’ thing rather than what they’d actually do. Think about interviewing people and asking them, when given a $20 bill, what they’d buy on the Supermarket. The answers would usually be the ‘right’ ones – vegetables, fruits, bread, dairy products. Very different than the results you’d probably get when analyzing real shoppers.
On my previous company, Takipi (now OverOps) I held many user interviews. We developed a technology to help developers solve complex problems in production environment. On interviews, people always focused on the ‘spicy’ and exciting stories – the complex bug who crushed all their servers, the security issue caused by a new employee who touched the wrong place on his first day. No one mentioned the everyday problems, those which eventually led people to use and pay for the product. If will go back to the astronaut example – they talked about walking on Mars rather than scratching their nose.
If I’d go back to the initial plan of the same product, I’d take from about 10-20 companies all their log files and documentation of their production issues during the last couple of weeks. Based on it I’d try to understand what are the low hanging fruits, what are the most critical issues and which problems repeat most often.
How to design a UI that looks good after the prototyping stage?
Using real-world data is also an important principle to keep when designing the product interface. One of the main ‘fail points’ of products is the gap between how the design looks like when leaving the designer’s desk to the way it looks when it had to handle real data. We’ve all been there, the amazing design soon turns into a battlefield of long labels, unstructured data, texts in unexpected languages and renegade graphs. At Oribi, when we design the product interface, we work with some of our customers actual data from the get-go. We don’t use ‘well behaved’ data, fictional names and nice graphs. We skip the ‘pretty’ stage but make sure everything will still function well when meeting reality.