Emmett Shire: How to conduct interviews with users

Stanford course CS183B: How to start a startup . Started in 2012 under the leadership of Peter Thiel. In the fall of 2014, a new series of lectures by leading entrepreneurs and Y Combinator experts took place:



Sam Altman: Good afternoon. Our guest today is Emmett Shir. Emmet is the CEO of the Twitch service, acquired by Amazon, where he currently works. Emmett will talk about how to conduct interviews with users. This part of the course "How to Start a Startup" will be devoted to communication with customers - it should be really useful. Thank you so much for visiting us.
')
Emmett Shire: Thank you, Sam. I started my first startup with Justin Kan back in college. We called our company Kiko Calendar. But things did not go very well. Actually, it wasn't all that bad. We built the company and sold it on eBay. Let this not the result that you probably hope to get from their startups.

These were good times. We learned a lot. We learned a lot about programming. But we knew nothing about calendars. None of us used calendars. Moreover, we did not even bother to talk with those who used them. That was a mistake. We did not take into account the fundamental element when creating our startup. The element of communication with users.

When launching our second startup, we avoided talking to users using a typical trick: we ourselves were consumers of the product.

We had an idea to create a television show Justin.tv, namely, a reality show about the life of Justin Kahn. We have developed a technology and website for this reality show. We ourselves were users of our product. One way to cheat and avoid talking to many other clients is to create something for yourself, literally. Thus, you do not have to discuss the project with anyone else, because you know what you want and what you need. But this technique is not for all startups. In most cases [unfortunately] startups are not created for those who use them. Occasionally, when creating such a startup, luck may smile at you, and you will be one of many people who also want to use what you use. But most often, such projects fade into the background and do not lead to anything.

For some time we continued to work on the Justin.tv project and really succeeded, because, as it turned out, there were people who also wanted to do this - broadcast their lives via the Internet. The only thing that prevented Justin.tv from reaching unprecedented heights was that we could not figure out how to go beyond the scope of this initial television show. We have created a wonderful product. If you wanted to launch a 24-hour reality show about your life, then our site is for you. We had just what you needed, but we wanted more. We wanted to multiply the number of users and increase the capabilities of our product, but had no idea how to do this, because we were not these users.



So, we decided to convert Justin.tv. We decided that we need to move in a different direction. Although we developed a very valuable technology, we did not know how to use it so that the project would become really popular. Two directions seemed the most promising: mobile and gaming. I headed the game direction in the company. We did what we had never done before - talked to users. Despite the fact that I enjoyed watching videotapes of game moments, neither I nor my colleagues knew anything about broadcasting video games. I was inspired by the content of our idea. I thought this was a promising area. At that time, not everyone understood how much pleasure you get from watching video games.

Raise your hands, who among those present know about watching video games on the Internet? If you do not know, you should read about it in order to understand what I will talk about. I thought the idea was awesome, but I didn’t know anything about one important aspect, namely, the acquisition of content that will broadcast the startup. We conducted a large number of interviews with users. We interviewed many people, and the data obtained formed the basis of all decisions taken regarding the technical characteristics of Twitch over the next three years. We continued to talk with users, moreover, the whole branch of our company was engaged in such communication. We did not have such a division on Justin.tv. No one in our company was responsible for communicating with the most important users.

Guys, I want you to have an idea of ​​what it means to talk to users. We realized that in our case, the most important people are streamers, since it is up to them that determines why people choose a particular broadcast or a particular website. They just follow the content. You choose the content that you like best, that is, choose a specific tape drive.

In fact, in interviews with users, it’s very important who you speak with, and not just what questions you ask and what you want to extract from this.

If we interviewed only the viewers, we would get completely different answers than if we interviewed streamers. Thanks to communication with the latter, we understood how to create something useful for them. And it turned out it was a strategically correct decision. Of course, I would like to share with you a universal way of determining the target user of your product, but there is no universal way. You will have to think carefully and decide for whom your product is actually intended.



And now I want to do something interactively. You suggested a few ideas, and I chose one of them. I want you to take the first step of this process [of studying users] right now. Who would you interrogate, where would you go to figure out what you need to create? The idea I chose is a note taking application.

The idea is as follows: the level of note-taking technique is far from perfect, and I want to create an note-taking application that will enhance it. With this application it will be easier to take notes in class. Maybe there will be opportunities for teamwork in it, or it will somehow increase your concentration. Or maybe it will expand multimedia capabilities [lecture notes].

Any specifications are allowed. That is the idea. So, you have 120 seconds to think not about what questions to ask or what properties are needed by this application, but about who you will talk to. Whose answer will help you understand whether to do this or not? It would be better if you do not just think about it, but write down five types of people with whom you would talk: indicate which of them, in your opinion, would be most important to you.

There is nothing better than trying something first hand. In fact, the best way to do this is to think. I'm glad to hear the sound of the keyboard. If you are listening to a lecture from home - think about it too. The first question at the start of almost any startup is who to talk to. You need an answer to the question: who is my user, and where can I find it?

I will ask a volunteer to come up from the hall and tell us who he would talk to. And we will discuss this.

Student 1: First of all, I would definitely talk with students from college because we attend a bunch of lectures. In particular, I would talk with students studying different subjects. To see the difference in the note-taking technique between those who study English and those who study mathematics or computer science.

Emmett Shire: So you want to talk to college students. Can you choose a specific group of students? After all, we will not interview all students in college.

Student 1: I want to talk with students from college and then break them into groups depending on the subjects studied. This would be appropriate for people who use different techniques in their studies, as some write a lot of notes. Others do not write detailed notes, but still make some notes.

Emmett Shire: Very good for a start. Obviously, these are the users you want to chat with, especially if you see them as a target consumer. In this case, you will need to find out how they are making notes now, and what might interest them.

When selling something to students, one of the main problems is that they do not spend a lot of money. It’s very hard to get you guys to fork out, especially on things related to learning. People do not want to buy even textbooks. You all probably use checks, debit cards, or borrow money from friends. Thus, if you confine yourself to only students, then you will miss the one who is most important for this application. If you really had such a college note-taking application, then the college’s IT department would be the most likely buyer.

Presumably, in most cases you will need to sell the software to students, as well as to people working in the field of education, that is, the school administration. This is one of the possible approaches. I assume that if you talk to students in college, you will find out that they will not buy any note taking application now. But maybe I'm wrong, and they will buy it. That is why you need to go and talk to users.

But after that you should talk with other groups. I would also talk to programmers. This is another fairly promising category. You can talk to your parents.

Who spends money on the education of their children? Who is ready to open your wallet? Parents of first-year students whose children go to college for the first time. Your child needs this application for academic performance so as not to fly out of college. There are many groups of people who are not direct users of your product, but play a potentially crucial role in its success. When your startup is at the initial stage, when you just have this amazing idea, you want to reach as wide an audience as possible. You do not want to be limited to only one category of people. You want to know about everyone who can contribute.

Let's invite someone here and pretend to be interviewing the user. We will talk with a college student and find out what we need to create, what we need to add to this application for taking notes. So, we need another volunteer for the interview. Hey.



Student 2: Hi. My name is Stephanie.

Emmett: Hi Stephanie.

Stephanie: Nice to meet you.

Emmett: Thank you for agreeing to participate in the interview. I would like to know about your note-taking method. How did you take notes today?

Stephanie: I take notes in different ways. It’s easier for me to take notes on a laptop because of speed and productivity, and also because I can go back to the note later and add it. Most of the abstracts are based on text, as, for example, on history pairs. But if this is a physics lesson, then I have to draw more complex diagrams.

Emmett: So what are you using today?

Stephanie: Just a pen and paper.

Emmett: You use pen and paper. So you combine. You take notes with a pen, and sometimes you use a computer.

Stephanie: Yes.

Emmett: When you make all these notes do you look at them at the end? Honestly! Do you really go back to them?

Stephanie: Pencil synopsis - not really. But I look through the ones I keep on the computer. They are easier to find, easier to share with classmates and friends, to cooperate.

Emmett: What programs did you use to make a summary today? Audience Member 2: Google Docs and Evernote.

Stephanie: Google Docs and Evernote.

Emmett: Why two at once?

Stephanie: If I just make notes for myself, then it's easier to use Evernote. You can share these notes, but Google Docs is more suitable for this. If my friend has already created a folder in Google Docs, I just need to add a document to this folder when, for example, we are doing a group project. If this is for my personal use, then I prefer Evernote.

Emmett: It seems you are actively interacting with other guys.

Stephanie: Yes, I want to combine our efforts.

Emmett: Tell me more. Are you trying to gather as much information as possible from the abstracts of other people? Or do you mostly look at your notes at the end of the semester? How does this happen?

Stephanie: Basically, I read my own notes, because I'm pretty picky about the design. I pay attention to design, formatting, even color. The font we use greatly influences the way I learn. So I try to personalize my notes, even after merging with others.

Emmett: So you take other people's notes, but then you redo them in your own way. Awesome! If you have notes in Evernote, and Google Docs, and hand-written on paper, do you come back to them again when the semester ends? Are you reviewing your notes at the end of the semester?

Stephanie: I do not reread the abstracts made in class. But if we are talking about seminars such as this one, or preparing for an interview, I usually return to them from time to time to refresh my knowledge. It helps me to prepare for the upcoming events.

Emmett: Interesting. Be good, tell me about it. You take notes not only in class?

Stephanie: I take notes to highlight key points. For example, inspirational quotes from seminars such as this. If I'm going to meet with someone, the notes will help me remember what was being said.

Emmett: Great. Under other circumstances, I would have gone deep into this question. I still have a lot of open questions after this conversation. Who do you work with? How long are your notes? How much time do you spend on note-taking? I would take a closer look at all these aspects, but in order not to waste your time and not dwell on one example, we will move on. Thank you very much, Stephanie.

Stephanie: Thank you.

Emmett: You noticed that we do not discuss the content of the application at all. I am not particularly interested in its properties. I'm not interested in what specific feature set is in Google Docs or Evernote. Perhaps, I will dwell in more detail on those functions [of these applications] that we actually use. If Stephanie is constantly sharing information, how does this happen? I heard an interesting thing: "We use a folder."

When conducting the first interview it is not at all necessary to ask questions related to how to streamline the flow of customers. Or questions about the characteristics of the application and so on. This can be confusing because potential users think they know what they need. You will get the “horseless carriage” effect: you will be asked to have a faster horse instead of asking for a real solution to the problem.

Therefore, try to keep the discussion of the functionality as far as possible, since most of what customers say [on this topic] will seem realizable to you. When a real user asks you to create a certain function, it is very difficult to refuse, because you have a person who really has this problem. They say: "Make me this function." But only by interviewing many other people and getting an idea of ​​their problems will you find out if this is really a promising area or not.

I have not always seen the problem in what the users said at these interviews. At least, this problem was not so global that it was worth creating an absolutely new product to solve it. I didn’t identify too many serious problems that would really complicate the process. Until I have a good idea, I would take it as the absence of any results.

This does not mean that you can not move on and continue to interview more and more people. The fact that the first interview did not bring results does not mean that the same will happen in the future. As soon as you interview 6-8 people, you usually stop at this. With this approach, you can hardly gather enough of the necessary information. Therefore, it is important to talk to different people, whose answers will be different. Because the answers of six students at Stanford University will be different from the answers of six high school students or six parents.

Based on this, you may have some ideas. You have received information about how someone makes notes. You have thoughts about how to create something cool. If you only needed to add one feature to Google Docs, what would it be? For a new product like the one we are talking about, this is a good way to grow. How can we at least improve this program, which is being used so much now? What really interests the user, what will give you an advantage?

Think a couple of minutes, what kind of functionality it can be. Based on the story of Stephanie, try to come up with something that will convince her to abandon the usual teamwork in Google Docs in favor of your new application with the same characteristics as Google Docs, plus with your new feature. This should be a special useful feature that will convince users to stop working with what they are using now.

Student 3: The reason she uses Evernote is a brief note. With their help it is more convenient to write down some separate thoughts. It seems to me that there are no such small documents in Google Docs. I think it would be useful to create a mobile version of the application that would not be so cumbersome.

Emmett Shire: Great. Good observation. This is exactly what you learn from interviews with users. Now you have this idea. You have received a response from the user. What if Google Docs had opportunities not only for collaborative and group work, but also such small notes? There would be a product that is more designed for short records. The following question arises: is this idea enough for you? Do people really bite on this advantage?

There are two ways to check this. First, if you are well versed in programming, you can literally go and create this function, launch it into the world and see what happens. If you succeed, it will be a great way to solve the problem. But very often it takes months to create this small testimonial. Therefore, you want to test the idea before turning it into reality.

You can review your idea, depict workflow diagrams, show people how this functionality will look like in the end. Just do not ask users if they are interested in one or another function. This is a trap, because by asking a similar question, you often get the answer: "Yes, of course." And it seems to you that the idea is very successful. But when you implement it, it turns out that no one really is going to use it. So do not ask the question "Is this feature really useful?"



Sam Altman: Is it possible to get at least some guarantees before starting to create a function?

Emmett Shire: So what can be done to make sure, considering that you cannot just come and ask clients: “Is this function useful or not?” It depends on the specific function. Usually the best thing you can do is quickly create a prototype. If you want to create an application based on Google Docs, do not try to make Google Documents, but for taking notes. Try to write a browser extension that implements this small function and see if it will be useful for users. Finding a way to cheat is what you need to do, because it’s very difficult to find out how good a function is if you don’t have the opportunity to show it to people.

If we are talking about larger projects aimed at making a profit, then everything is easier. If you sell your product, that's great. Sales is the solution. Convince people to give you their credit cards, and I guarantee they will be interested in your function. Buyers willing to pay you in advance is the best guarantee for your product. That is why it will be quite difficult to implement the idea with the note-taking application for students.

A trial version of your application may be free - unless you are a staunch supporter of immediate monetization. Attempting to convince people to invest in your idea will not necessarily help you get important information. But if someone comes and says, "Hey, take my money!", This is a very good sign, an indication that people are interested in your product. If you are not ready to give five dollars for this idea, perhaps you don’t really like it.

The last thing I want to do is once again go through what happened in Twitch. I brought a few slides that I would like to show you. Here are some examples of Twitch user reviews. Initially, the document had 26 pages, but I decided that it would be a bit tiring to read all the reviews, besides, I would not be able to keep up with one lecture. Therefore, I previously reduced their volume for you.



Before launching Twitch, we spoke with some streamers on Justin.tv and asked them questions about the broadcast. What do they like in the broadcasting process, why do they do it, what do they usually talk about and what else do they do besides broadcasting. If you talk with users of your product, you will receive from them very detailed information about the functions, as they directly use them. Here you need to read between the lines. We were asked to clear the blacklist in the chat. This is a fairly ordinary request, because our chat worked in a special way. Users asked us to create the ability to edit ready-made headers. And these requests were really justified.

We surveyed about 12-14 streamers from the game direction on Justin.tv and received all these reviews. “Your competitors have all these cool features, like polls and scrolls. And in their program, I can change the settings in the chat. " Later we received some positive feedback. “Yes, you do not have this annoying advertising and it's cool. And you can "ban" trolls. , . . , Justin.tv.

, . , , . , . , , , , «» , , . – .



, . , , . . revshare (. , ). , . .

. . [ ] . , , , , . , , .

, , , . , , , . , , .



, , , , . . – , – Google, : , , «-». «-» .

«-» . , . , , , , .

, , . . , . , , .

? « ». « 12 ». « . YouTube». « -, ». « . «» , . , – . ?! ».



, , , . . . , Xbox PlayStation4, . , .

, , , . , .

. . . , . , . . , , , , : « , . , ».

, . , : « , », , , . , .

, . , . , . , . Justin.tv -.

Justin.tv , , Google Analytics, Mixpanel . , , , . .

, . , . Justin.tv, , . , . .

You will have a great idea, and you want to tell about it, but it turns out that users are not interested. They need completely different characteristics. They do not care what seems important to you. And it's a little sad, but think about what it will be like in four months when you start this functionality, but no one wants to use it.

[ Answers to questions from students of Emmett’s lecture ]