In 2003, David Hauser and his co-founder Siamak started Grasshopper, and thus began their journey in bootstrapping a business that would go on to generate $30 million a year in revenue. I'm Courtland Allen, and I've interviewed a lot of founders for IndieHackers.com, but none who've achieved that level of success, especially without raising any money whatsoever from venture capitalists. In this converation with David, I really wanted to pick his brain about the kinds of things that early stage founders and aspiring entrepreneurs could do to raise their chances of building a hugely successful business like he did. We talked about everything from coming up with a solid initial business idea and how to find your customers in places nobody else is looking to managing your work-life balance, or as he calls it, your work-life integration. David has a ton of insightful knowledge to share, and I know you guys are going to enjoy this interview and learn a lot, so I present to you David Hauser, the founder of Grasshopper. Before we get started, let me tell you about a group of guys that I got to know a couple of months ago. They reached out to me as fellow MIT alums, and I ended up interviewing them about one of their profitable side projects on IndieHackers.com. Dixon & Moe is a boutique digital agency. They're technical designers with a focus on business and marketing and they specialize in helping bootstrapped founders grow their products. They're based in San Francisco, and they work with founders from all over the world. And they're big advocates of the indie hacker movement, both as participants, having made several revenue-generating projects in-house, and as design and development consultants who work with an array of founders and products. For example, they helped the founder of WisePops grow his revenue from 10 to $50,000 a month and quit his job at Amazon to go full-time on his side project. They helped Alex from GrooveHQ, who by the way has David Hauser as his lone investor, to reposition his help desk software from a scrappy app for startups to a professional tool used by companies big and small. He's now making $500,000 a month. Dixon & Moe are looking to take on two new projects this year and would love to see where they can help. You can chat with them over email or phone about any design, development or marketing problem that you're struggling with. Reach out to Moe, that's M-O-E, at<EMAIL_ADDRESS>That's dixonandmoe.com. And make sure to tell them that Courtland from Indie Hackers sent you. Hi, everybody, this is Courtland from Indie Hackers and I'm sitting down today with Dave Hauser, the founder of Grasshopper and Chargify. How you doing, Dave?
Good. Hey, Courtland, thanks for having me.
Yeah, thanks for being on the show. So you've done a lot of things in your career, including Grasshopper and Chargify, and I am personally the most familiar with Grasshopper. So we'll probably spend a little bit more time talking about that today, but can you tell us a little bit about what Grasshopper is and how it works?
Sure, yeah, so myself and my business partner, Siamak, started Grasshopper more than 12 years ago now. And the simplest way to describe it is we provide a virtual phone system for entrepreneurs. So what does that mean? It means, you know, the phone system you'd expect in a big office, press one for sales, two for support, on-hold music, transferring, you know, all those features just totally virtual, so no hardware, no software to install, all web based, but you can also still keep using the cell phone that you have or if you do have a VoIP line or anything else, so it's pretty flexible. And we designed for that, because we knew that entrepreneurs really have lots of varying degrees of offices and setups and even moreso now, remote employees and things like that. So I mean, that's kind of what we did, and we served over 250,000 customers over that period of time, and, you know, learned a lot along the way.
I bet, that's a huge number of customers. And you ended up selling Grasshopper in 2015 to Citrix. Just how big had Grasshopper grown?
Yeah, so we had bootstrapped Grasshopper to 30 million in revenue and just under 40 full-time employees, mostly in our office in Massachusetts, just outside of Boston. So, pretty sizable company, a solid management team that was really running a lot of the business, you know, a lot of those things, it made a good acquisition for Citrix.
Yeah, at $30 million a year is by far the most money that anybody I've ever interviewed has made. So that's humongous, and I also had no idea that you guys are outside of Boston. What part?
So we were in Needham. We moved around a little bit, but we started in Needham and finished in Needham. And that's where we kind of stayed, and that's where the company still is today.
Cool, you guys should've come out to the MIT Career Fair or something, because I was there between 2005 and 2009. So I probably would've seen you.
Yeah, we had done a few things, a bunch of stuff, interns with Northeastern and other things, but that was towards the later years, when we had more kind of junior hiring needs. When we were first kind of starting, it was hard to hire more junior people. We just kind of needed more senior people, so it was kind of away from that group.
Makes sense. So let's go back to 2003 and talk about the founding of Grasshopper. I think a lot of listeners are at that beginning stage, where they're thinking about starting a business, but they're maybe not quite sure how to come up with a good idea or what it is they should be working on. So I'm curious how did you at Grasshopper decide to build a virtual phone system. Where did the idea come from? And what motivated you to get started?
So myself and my business partner had both worked on previous businesses, and we had experienced the same problem so many times, which is how do you sound professional when you don't have an office necessarily, right? We didn't want people calling, you know, our cell phone, and we'd just pick up at random times. We needed a professional image. And we went out there and looked for this, and there weren't really many options, right. The options that were available were either very expensive or not flexible and the technology itself was actually quite easy, right. This wasn't a magical technology that we created. It'd been around for a long time. But we knew the problem was. It wasn't packaged correctly for what we, at the time, called small business users. And we kind of, over time, learned that the better term, that people identified with was entrepreneurs. But it just wasn't packaged and priced for them. So we spent the time to create the solution and package and price it for the right market.
Yeah, there's a lot of uncertainty in the beginning for most people, and it sounds like what you're saying is that nobody was doing what you were doing in the way that you wanted to do it, which was to sell to small business users and entrepreneurs. But there was really no guarantee at that point that it was gonna work out, right, because maybe you could've tried doing it, and there would be enough people who wanted a professional virtual phone system or some other problem. So what did it take to validate your idea early on?  Or were you confident from the very beginning?
So I think we were kind of young and stupid in that we were confident from the very beginning, but I think it came from the fact we were solving a problem we had, right. It wasn't something that we heard from other people necessarily or kind of magically created. This was a problem we had, right, directly. And I think some of the best products are really created this way, because you could can somewhat skip a little bit of the step of talking to users right away, because you at least know the base of what you're talking about, right. And that's not to negate, like, it is super important to get real-user feedback, but we knew we were solving a problem. And I think we ignored a lot of the other things. And then we got to selling right away. Like, rather than talk about it and see and kind of pontificate about why it should work or wouldn't, we're like okay, let's just sell this to people. And we started selling way earlier than we should've, but I think it's the best thing we ever did.
Yeah, I remember in your tech space interview, you talk about, and speaking of selling too early, you talk about doing customer support in the early days. And customers would call you, and while you were on the phone, you would be writing SQL queries to look up the customers in your database, because you hadn't written any software to actually look them up.
Yeah, I mean, look, the priority was very simply like getting people sold and paying us money. And it was still important to support them, but we didn't think about an interface to do it or a CMS or a support ticketing, like, people would call. We'd answer the phone, and we would be literally writing SQL statements to find the customer in the database, right.
Yeah.
But the important part was they had paid us, and they were a customer, and they were using the system.
Interesting. So I talked to a lot of people who are starting businesses, especially developers, and what you guys did is almost the exact opposite of what most developer founders do. The temptation for programmers oftentimes is to just build, build, build, before talking to anybody, and often they spend weeks or even months just building out their software without talking to a single person about it. Well, you guys are the opposite. Your whole viewpoint was that the system was crucial, but you could build it out later and just wing it for now. And for now, selling is the number one priority. So why were you guys so focused on selling when it's not the intuitive way to go for a lot of people?
Well, I mean, I think there's two factors. One, we didn't have very much capital, so we had to bootstrap the business. We just had a little bit of money from past ventures and such, and it wasn't a tremendous amount. And this was way before, you know, kind of the true cloud computing. There was no Twilio for telecom. We were having to buy physical equipment and run T1 lines and do these things. So the cost of doing so was just very fixed, right. And we knew that we weren't going to get very far if we spent a lot of time doing that and not selling. And then both of us, although I'm technical, right, I can write code, and I have written code. I'm not great at it. You know, now, today, I can probably review it better than I can write it. Both of us kind of came from more business backgrounds. So all we really knew was, like, let's market and sell this. Like, that's what we knew. We both went to Babson College, and that's the background we had.
Yeah, it's a background that a lot of programmers don't have, including myself, because I basically had to spend the last 10 years in the school of hard knocks, just trying to figure out how to do some of this stuff. But it's really interesting to look back at the time period where you were creating Grasshopper and to look at the differences, because there was no cloud computing. There was no Twilio or AWS. And so in a way a lot of the things that you could do were very limited. You just didn't have that many tools to build on top of, which seems like a disadvantage, at first glance, but it's also advantageous, in a way, because it forced you guys to get in a mindset of selling immediately.
Yeah.
And of course, the disadvantage is that things are super expensive and hard to do. So how did you afford your infrastructure? 'Cause even just getting service or hosting, back in those times, was a pretty tough ordeal.
Oh yeah, I mean, we were buying servers from Dell and racking them and stacking them in a data center, and you know, like, what people wouldn't even imagine you would do today, right. And I think, two things we did, right. We used the capital that we had as efficiently as possible. So we negotiated terms with every vendor we worked with, some vendors extending us terms that they probably shouldn't have, right, because they believed in what we were doing, right. We sold them on the vision of, like, we're building something for entrepreneurs, and that's who we are. We believe this is gonna work, and here's why. And they probably gave us terms that they hadn't given others, and they shouldn't have given us, right. So we took probably about 200, $250,000 and ended up, you know, putting together an infrastructure that should've cost us probably about $1 million. From servers, Oracle database(s), like, those are things that are quite expensive and then also bringing in physical T1 and then as we got bigger, DS3 lines and stuff to these data centers. Like those are hard costs that today, people don't even think about.
So not only were you selling to customers from day one, but you're also selling to vendors, selling them on your idea and selling them on your vision. And I can't imagine that was easy to do, because it was 2003, I mean, not that long beforehand, pretty much every tech company had just been wiped out of existence, so how did you negotiate with these people?
Yeah, it was a hard time. You know, we sold them on our vision. We sold them on who we were. And I think it really came down to people believing that what we were telling them actually was a need. So that was some validation even before we were able to sell, that other people saw this as a need, especially as we talked to people in telecom, right, because had to negotiate contracts to buy phone numbers and minutes and things like that, right. And we got feedback like, you know, we're gonna start referring people to you right away, because we get customers that are too small, that always want this, right. So I think that was good. And we stuck with those same vendors for a long time and probably longer than we should've from a cost perspective. We were loyal to them, because they helped us start. And over time, obviously, we did start to migrate to larger tier one vendors compared to tier two vendors and things like that, but look, I think they made a good investment, and it paid off for them.
Yeah, it sounds like it was a win for everybody involved. It also sounds like you were dealing with a little bit smaller companies in the beginning. Did that make it easier for you to negotiate these deals?
Oh yeah, we were talking to their founders or presidents, right. These were, you know, in telecom, very small companies, but way bigger than we were at the time, which was nothing.
Until you eventually grew to the huge numbers, like you were saying, 250,000 customers served and making $30 million a year. One constant that I've seen in talking to different founders and entrepreneurs is that there's always a difference between what founders believe and what the actual reality of the world is. For example, you, as a founder, might have ideas about who your customers are or what they want or the best place to reach them or what messages they'll respond to, and you might not be accurate in all those assumptions. I assume that you guys weren't perfect in the beginning, but you grew so much in the end that you had to have eventually successfully aligned your beliefs and hypothesis with the reality of the world. So what are some of the things that you learned, and also what methods did you find the best for learning what product to build and how to reach customers?
Yeah, I think that's a great question. And quite honestly, I think we got all of it wrong at the beginning, and maybe towards the end, we got a little bit right. But I think it's an ongoing battle in both directions, right, in that, you know, when you're starting, you have this vision that no matter what, you're gonna persevere through everything, right. And it's like all green in front of you, right. In kind of the middle, you worry a little bit more, and you're like okay, now there's some reality setting in. Like, it's a little bit harder to sell than I thought. It costs more to market, right. And I think that kind of continues. And if you lose that, that's when you start losing, and competition comes in and kills you, right. And I think one of the things we learned towards the middle as well which was we thought we had kind of capped out on the market.  We're worried that we're starting to kind of plateau in growth. We had many plateaus in growth, and then we had growth spurts, right. And we kind of started to believe this ourselves. So we spent time focusing on other products, and we built other things. And that's when we built Chargify, and a bunch of other stuff that failed. Chargify actually did well. And we lost focus on the business, and looking back on it, what we really realized is we were wrong. We got bored, and we should've stayed focused.  We just weren't looking in the right places.
Yeah, it's a really tough line to straddle, this balance between when to quit, or at least, when to change it up, and when to persevere, kind of seemingly against all odds. And you hear stories on both sides, of people pivoting into a really great idea that was way better than what they were working on at first. Or people, like for example, the Airbnb founders, who keep working on the same thing over and over, even though it's failing for years, and it ends up working out.
Yeah, I mean, I think that's what it is to be a founder, right. We, as founders, kind of persevere through that, and probably in most cases are too positive and wait too long, right. But that's what creates success, where I think, those that fail didn't have that same drive, or, you know, if you just hire a manager to run a business, right, they're much more pragmatic about it, right. Founders have that vision, like, "I don't really care what else is happening, but I believe this."
Right, and on the flip side, there are times where the thing that you're working on, as a founder, truly is not worth working on, or maybe you're just going in a different direction than you should be. So are there any examples, and I'm especially interested in the early days, but are there any examples where there's something that you learned from talking to customers and trying to sell that shifted your direction?
Yeah, I mean, unfortunately, in the beginning days, we didn't really talk to customers very much, was probably the biggest problem. This was well before it was kind of a popular thing to even, you know, in the tech community, say you talk to customers, right. So we didn't understand that at all. So we built and packaged the product in a very complex way that we understood, right. And this came out in user churn, right, which was we were onboarding customers that were not as technology savvy as we were, right, and either would call up, and it would cost us a lot of money in support time and cost, because we always provided 24/7, US based support, not outsourced to another country, right, and true 24/7, not like open a ticket and wait. Like, call us and we answer the phone, right. So we saw it in churn and cost. And that's when we realized uh oh, like, we have a problem here that yes, we've built something that people want, because we've shown that we can get people to sign up, effectively, at good CPAs, but now, we're having people churn out, and we're having onboarding problems, right. And that's when we started to say okay,
Cool, and what kind of tools did you use to start talking to your target market? Was it email surveys or--
You know, initially, it was really talking to people, because we were doing the customer support ourselves, so that meant like I was on the phone actively talking to someone who was struggling with a problem. And, you know, quite honestly that was the best thing we could possible do, compared to some, like, random user group or whatever. This was someone who paid us money, signed up, and said, "Yes, I want this." And they're telling me they have a problem.
There's just no better source of feedback than someone who's actually a paying customer.
Right, and we tried to encourage this in later years too, where, you know, I think, as you grow, and you have 10 people and then 20 and then 30 and 50 or whatever, right, you lose touch with the customer. and especially, you know, developers, 'cause they kind of get insulated from support, right, because it's, quote-unquote, a distraction. So we had a program where everyone had to make seven phone calls to new customers, everyone, no matter what position you were, right. And it was just a welcome, but like, very quickly, a welcome turned into hey, I love this, I hate this. Why is there a problem with this? Right, like, it was instant feedback and just seven phone calls. It only took a little bit of time, but it kind of brought back that touch.
Smart.
And we did that for a long period of time.
That's really smart, and honestly, I think it's funny that you mention that you weren't talking to customers in the beginning, and that it wasn't as well known of a mantra back then in the startup community that you needed to actually talk to customers, because in my experience, even today, people don't understand this. People drastically underestimate how wrong their guesses and hypothesis can be about what people need. And they underestimate how effective it is to actually talk to a customer.
It's one of the reasons I hate the kind of movement to all-online, ticket based support. I think there's just this inherent value in talking to someone on the phone. You get much more instant and direct feedback. You can ask questions right away. And you can make a real connection with a customer. And what we found was it costs us a lot of money, like, no question, right. However, for sure, this contributed to our extremely high word-of-mouth referrals, which we maintained at between kind of 22 and 30% until we sold the company, right. Which as you're scaling marketing is very hard to do.
Yeah, that's huge.
But like when someone calls up with a problem, we consistently turned them into raving fans, because we solved it on the phone, in real-time, when they needed it, right. So to some extent, we wanted people to have problems, so we could have that touchpoint, because we saw what happened, and we listened to the calls. And I think it's just so critical, and so many people bypass that today.
Right, and what about in the early days, before you guys had really started talking to customers, and you just had this sense of okay, we know that we have this problem. We need a virtual phone system for ourselves. Other people have to have this problem, too. How did you find these other people?
Yeah, so we were very lucky when we started, that this was before AdWords, but it was at the beginning of Omniture, who's selling, you know, kind of click traffic. And we were buying traffic really cheap, like cents per click, right. Or even in some case, a cent a click.
Were these banner ads?
Yeah, we used banner ads, but most of the paid traffic came from search paid traffic with Omniture, which was the precursor to AdWords. So we did that. We did some print stuff, which sounds crazy, but there's a clear target for SMBs. At the time there was Entrepreneur Magazine, Fortune, you know, a few that were very targeted. So we played with print. And those were the initial channels. The other one, we spent a lot of time on was organic SEO. It was very clear that there was a need for people searching for these terms. And those people were converting at much higher rates, because it was intent purchases, right. So all of our initial targeting, like from day one to the first 12 months, was like someone is looking for small business phone system, virtual PBX, professional phone, those types of terms, and we just converted them as best we could.
Yeah, so it sounds like you were paying for a lot of your traffic, and SEO worked out great, too, which is interesting, because those are just two advantages that, I mean, like super cheap, paid traffic paying for ads for cents and also getting to the top of Google was probably just way easier in 2003 than it is now.
Oh, it was way easier. But I mean, I think, the advantages are different today, and it's just a different platform, right. So if I'm starting today, I'm looking at, like, how am I buying traffic on Instagram and Pinterest and places like that, where there is still intent. It's not as direct as search, right, but there's still intent, and I can buy traffic very cheap, because there are newer platforms that are still trying to prove themselves, right.
Right.
And the companies that I talk to now, and I work with, those are the areas that they're seeing an advantage in, right, where big companies are ignoring that.
So it sounds like you guys did very well with paid advertising, and you also got a lot of word-of-mouth referrals, which to be honest, are kind of the Holy Grail of growth in marketing. Were there any other marketing strategies that worked well for you guys later on? Because I'm sure in the 12 years that you ran Grasshopper, you probably tried all sorts of strategies and channels. Was there ever a magic bullets of sorts that worked better than anything else?
Yeah, I think unfortunately we were always searching for the magic bullet, and in our kind of naive, young selves, we're like look, we're gonna find this channel where we're gonna go from, you know, 25,000 customers to a million customers, guys. Like, don't worry, we're gonna find this channel. Right? And I think just it doesn't exist, right. And people always hear about these stories where the companies we know, quote-unquote, or that are success stories, yes, they have millions of customers, but there was no magic channel that got them there, but that's what the story usually is, right. So we tested everything, right, and we failed at most of it, but along the way, we found things that worked for us, right. So along the way, we found that satellite radio worked for us. We were the first advertiser with XM, well, at the time, Sirius, right, when Howard Stern moved over, right. So we signed up very early, and we scaled that channel to well over 50 or $100,000 a month. Then we had lots of failures. We tried tray table ads on planes. We tried--
Wow.
Anything you could imagine where our target market might be.
Yeah.
And then as we continued to grow, I think what happened was, you know, these little tests became bigger and bigger, right. So when we were small, a test might've been $100 or $1,000, right. When we were growing, a test might've been $1,000 or $10,000. As we kind of matured a little bit, and we got bigger, and we were starting to get closer to 30 million in revenue, tests became, you know, a few hundred thousand dollars. Or in our, you know, the year before we sold, we had a million-dollar test in radio, right. So we decided that terrestrial radio was the right approach. We had all the right metrics, but the cheapest you can really test terrestrial radio in three demographics and do it the right way is about $1 million. So that was a test, right. We were pretty confident in it, to put that type of money up, but it was still a test. It worked and we scaled that channel to $12 million, you know, very quickly.
Yeah, that's a lot of money, and those are economies of scale that worked, because obviously a smaller business can't afford that, even though it might be helpful, so they just can't even compete on that front. But it's cool that you guys are able to consistently test and figure out what appealed to your customers over time, 'cause it sounds like right up until the end, you guys were still experimenting and finding out new ways to grow.
I think one of the things that happened, you know, quite a number of years ago, now, is we really learned that testing was critical, and this was early on in A/B testing tools, and we had a few people come in and really help us and build a culture of testing. And this culture of testing started to really go throughout the organization, not just in marketing, not just in conversion, but A/B testing onboarding tools. Literally, the question was have you tested this, right, internally? And yeah, I think it helped us tremendously, and we tested pricing plans, pricing increases. We did different tests on how we increased plans on our user base or not on it. Everything was tested.
Yeah, testing is a smart way to do it, because obviously, then you know what's working, instead of just blindly assuming that all the things that you're doing are working. And speaking of blindly assuming, I want to talk about how people will look at an established business, like Grasshopper, in maybe its 10th or 11th year, and they see what it looks like, and they see that things that it's doing, and then they assume that it's always been this way. And so they try to copy it. And they don't realize that the assumption that they're making is that the things that are important to a huge $30 million business aren't necessarily important for getting a fledgling business off the ground. So what are some of the ways that Grasshopper today, or when you sold it in 2015, differed from when it started?
Yeah, so I think a lot of things changed, right, but when you're small, I think the things that don't matter as much are kind of structure and process. When you get bigger, those are critically important, because you can't be looking and touching everything at the time, right. We made a lot of mistakes, but I think one thing that is ignored early on, and I think is critical, is culture and core values and core purpose and really having a true culture that really exists. I wish we had done that earlier.
And just for some context, when did you guys make your first hire?
Probably in month three, from when we kind of started.
Okay, so you guys started growing almost immediately.
And that hire was someone to help with customer support.
Right, and so from the beginning, you were saying that you guys didn't focus on culture as much as you probably should have.
No, we didn't. We just hired. We had no idea what we were doing, right. And we got to a point where we had about 50 people. The culture clearly wasn't right. It just didn't feel right. And we spent a solid year figuring out what our core values were, talking about them, okay, making sure the right people were on the bus. That took us more than a year. And that was in the middle of trying to scale a business, We made lots of mistakes doing it, but we could've saved that by really doing that much earlier. And we did that at Chargify much earlier, right.
I think just the concept of having a company culture with values that you want people to operate on is fascinating, because when you're only one or two people, then culture just sort of evolves naturally out of whatever your personality is, because you represent such a high percentage of the people there, but if you grow, and you don't set a culture deliberately, then one kind of forms on its own, and it might be unhealthy or something that you don't want.
Absolutely.
So what goes into setting a culture, like intentionally, deliberately creating a culture at a company with more people, 'cause I know it's easy to have a list of values or to write some words down on a page, but how do you actually instill those values in people and make them a reality, rather than just a cheesy collection of words.
Yeah, it takes a lot of time and a lot of effort. And I think actually getting real core values is actually very hard, right. Lots of people can write honesty and respect and things like that, but having real core values that are meaningful and actually represent not just what you want but what you are, and you're right, it comes from the founders. So our core values were go above and beyond, always entrepreneurial, radically passionate and your team. It spelled out GARY. We had fun with it, right. But we pounded people over the head with this. In the hiring process, this was the very first questions, right. Like, how do you apply yourself to these core values, right, and give stories around them. In the rewards and recognition process, we had quarterly, and then we moved to yearly reviews. Long story why we did that, it's not really relevant until you're kind of 30-plus people. But, you know, every period where we reviewed each other, we talked about core values. Our monthly meetings, we presented gifts and rewards around the core values, who showed this the most, right. We had an I-caught program, where people could catch other people showing these core values, right.
So it was just everywhere, all the time?
Yeah, we publicly talked about 'em. Customers knew 'em. So when we gave a credit, right, or we did something, we matched it to a core value. No matter what it was, it always went back to that.
And there's a related topic that I want to talk about, that's kind of similar but perhaps more individualistic, but it's the psychology behind being a founder, which is questions like how do you manage your mental state and make sure that you're the best that you can be? And I think a lot of people underestimate how much of a role this plays, maybe just 'cause it's hard to measure and that's why I love talking about it, 'cause it's so rarely discussed, but there are all sorts of things for developing this kind of personal culture. Like, what's your philosophy on work-life balance, for example, and how do you deal with scheduling your work? How do you maintain motivation and get yourself up in the morning? Or on the flip side, how do you deal with burnout or depression? So what are your thoughts. Did you have any guiding personal philosophies for how to manage your own psychology as a founder?
Sure, yeah, so I mean, I think there's a lot in that question, and so yeah, first, I don't believe that there is such thing as work-life balance, I think there's work-life integration, right, which means, you know, I work at times that doesn't make sense, right. Like, maybe I'll interrupt dinner, or I'll work at home, or whatever it is, right, or on vacation, because I think as you integrate things, you have more flexibility and freedom, compared with regimenting things and saying I can't work after 7 p.m. Or, you know, I'm going on vacation, and I'm not gonna look at emails, right. I think those extreme things really don't help.  No, like, real people that have to get real things done. Do look at it more and should look at it more. Sorry, right. So I think that it's about integration. I think personal health is very important. If you can't treat your personal health and eating right, which initially I was all wrong with. I was eating all the wrong foods. I wasn't exercising. I was gaining weight, right, because I'm in the office. I'm working. I'm stressed. So I made major changes there. That had a clear impact. I think waking up early is the biggest advantage any entrepreneur could ever have, right. If I wake up at 5 a.m. and everyone else wakes up at 8 a.m., I gained three hours on everyone, right. And that's a magic three hours, where A, there's much less distraction, right. There's less emails. There's less phone calls. There's less people asking for things. But most other people are not getting up at 5 a.m. and working, right. And that's how you win, plain and simple, right. Like, that's how you win. There's no magic. There's nothing else. So I think that's very important. I think the last piece to your question, and I'm kind of trying to remember each piece. So if I forget something, definitely ask me.
Yeah, it was a heavy question.
Yeah, I think the other piece is as entrepreneurs, we have two selves, right. One which is our public self, which we have to present to our employees, our partners, our vendors, which is overly optimistic, right, no matter what, We're gonna persevere through anything, right, even when the real reality might not match that, but we have to, we need to face the real reality, but we need to present an image of we're gonna get past this and here's why and here's my plan, right, with here may be times where you go through this roller coaster, and you're at the bottom of that kind of slide, and things are very negative, or you really don't believe that the next step is possible, and I think the only way to deal with that is to have peers around you that you can discuss those things with, and they're other entrepreneurs, right, because it's hard to discuss it with your family, because there may be real financial impact to sharing something like, "Oh my God, I can't believe we might not be able to meet payroll next week," right. So it's hard to share it with family. It's very hard to share with employees sometimes, right, until there's more maturity in the business. But in the early days, like, I couldn't imagine sharing those problems, right. But finding a peer group of other entrepreneurs facing the same problems and being able to comfortably share them, I think that's how you get through the non-public image of entrepreneurship.
Everything you said is awesome. And I just love talking about this stuff. The first thing that you said, it sounds like you're really focused. All right, it was when you were talking about work-life integration, rather than work-life balance, which got me thinking about the fact, and not everybody is like this, maybe it's just me, but for me, one of the things that I prioritize the most in my life is flexibility. Because what your business needs, for example, is not always going to fit on this perfect nine-to-five schedule, and it's similar with your mental state. What you feel motivated to do at any given time is not going to always fit into this perfectly organized box.
Right.
So for me, it comes down to doing what feels right at the right time.
Absolutely.
With Indie Hackers, for example, there are gonna be times where I really feel like editing the text interviews, and there's gonna be times where I really don't want to do it, and the thought of it makes me sick. And I'm gonna be a lot happier as a person, and a lot more productive as a founder, if I don't try to force myself to do everything on a rigid schedule every day, no matter what.

Exactly, and, you know, there is actually such a thing as being a workaholic, or being addicted, and I understand why people want to have these strict separations, but I think sometimes, it's kind of an overreaction, right. You're trying to live your entire life on a rigid schedule with strict delineations and time blocks, with your work between 10 and six, and you leisure time on Saturdays at 2 p.m. It just, for me at least, just doesn't work that well. It just doesn't naturally fit into how I feel and in what my business needs. But another thing that you talked about, that I thought was interesting, was waking up early. In most of my previous projects, pretty much for the last 10 years of my life actually, I've had this night owl programmer schedule, where I'll stay up late coding and wake up late. But with Indie Hackers, I wake up super early, and it's a different business, to be honest, because I spend so much of my time talking to people, answering emails, tweeting, sales calls with sponsors, et cetera, and I think part of me waking up early is that I moved in with my girlfriend who likes going to bed early and waking up early, et cetera, but also part of it is that I'm on the West Coast, and if I wake up at 10 a.m., then half the day's already gone on the East Coast effectively. So it's kind of a natural thing. If I'm actually doing my job right and talking to people, then I'm gonna feel pressure to wake up early anyway.
Yeah, I think it's a great point about time zones, right. I experienced this when I moved from the East Coast to the West Coast, but all of my staff and the company was on the East Coast, right. So a 10 a.m. meeting was a 7 a.m. meeting for me, right. And so even more so, waking up early was important, because now everything had shifted by three hours naturally, so now if I wanted time that was more quiet or focused, I had to wake up even earlier, right.
Right.
And I think there's not better advice, I think, for an entrepreneur, than doing that. There's no other solution.
Another thing that you talked about was having peers around you who were entrepreneurs themselves, because you can share with them a lot more easily what's going on, and they can understand you better than other people who haven't really been in the trenches. And I think a lot of people listening, including myself, are solo founders. We don't have co-founders. That's not really even an option. And a lot of us don't even have employees. So I think that having some sort of other source for a community is crucial. And what's helped me personally is this trend towards transparency. So for example, I can hop on Twitter, and you certainly could not have done this in 2003, 2004, but I can tweet out a decision like, "Hey, I'm recording a podcast. Which of these topics sound the most interesting to you guys?" Or last week, I tweeted, "Hey, I'm having trouble with my affiliate marketing program. What am I doing wrong?" And the amount of amazing support and feedback that I got from people was staggering. Everyone jumped in and had different opinions for what I could do. And I think from a practical standpoint, obviously, getting this kind of advice is good, but even from a psychological or an emotional standpoint, just having other people who've been through what you're going through makes it way easier when something goes wrong, because it doesn't feel like you're completely alone. And I think a lot of founders who haven't found a community, who aren't on the Indie Hackers forum or aren't on Twitter and don't have people to talk to, end up getting sad or quitting or getting depressed, because they just don't have anyone to share with. On a different note, I'm really curious about how you sold Grasshopper, because when it comes to selling, there's always two questions that I have. Now you ended up selling to Citrix in 2015, and one question is why Citrix? And the other question, I think, is a little bit more interesting, but it's why sell at all? Because at this point, you've been running Grasshopper for 12 years.  I've only been running Indie Hackers for five months, and I already feel like it's part of my identity, so I can't even imagine 12 years. What kind of framework did you have in your mind for determining when to sell?
Yeah, great question. So first, from day one, we never had an exit plan. People kind of asked us this question, and they kind of thought we were crazy. We were like "What do you mean exit plan?" Like, we're building a business, because we want a great place to work. And we think we're solving a real problem. Like, why do we need an exit plan, right? Part of this was we were also bootstrap, so we were never pressured by VCs or others to have a, quote-unquote, exit plan. So we never expected to sell the business, quite honestly. Citrix approached us. The process took over a year from when they first approached us to when we sold it. There were a lot of reasons that it was a good fit. I think the timing made sense. As founders, we had been in the business for 11 years. All of our personal wealth we had generated was in the business, right. We had no reason we needed to sell. You know, we drove the cars we wanted, or car, I should say. We lived in the house we wanted to live in. We didn't have extreme lifestyles, but we had the stuff we wanted, right, and, you know, I think that's, as being an entrepreneur, that's kind of part of the goal, right, is to have the flexibility and to have what you want, not to have crazy things but just have what you want, right. So there was no reason to sell. But as we went through the process, we found more and more of a fit with Citrix. You know, culturally, they believed in keeping the name Grasshopper, that brand which we cared about. They cared about the employees and keeping them. And end of the day, the monetary and structure of the deal made enough sense that it was impossible to say no to, right. And I think that the best outcome is when you're not looking for that. And it worked out very well for us, and the companies still thrives today within Citrix.
What's your role been like since then? Have you been helping out with the company at all, or is it kind of a clean break?
It was actually 100% clean break, so we closed in the middle of May in 2015. And the next day, I'm like, "Holy shit, "I need a new email address, because-- They're like, "Your email's gonna forward for a few months, but you got to figure something out."
Right.
So like, you know, every user name I had for every account, like, you know, everything, right, but I think it worked out well. You know, we had a management team of five other people that were running the business, and that's part of the reason Citrix didn't need us to stay. And it worked out very well for us, and it worked out well for them, where they didn't need to find a place for two founders and give them a salary for no reason. So yeah, it was pretty instant. It was like okay, that's it, done.
I think it's pretty impressive that you guys built a business that could completely run without the two of you.
Yeah, I think partially, intentionally done it, and partially, you know, by mistake. We hired people to fill in the gaps for the things we didn't like doing. Over time, those people got better and better at doing those things, so there was no reason to have us around as much. I then moved out of the office, so being remote kind of forced more of that. And, honestly, the team we had was just an amazing team, and I love all of those people that have been involved in really set us up for success and did the things that, as founders, we could never have done, brought structure to the business, brought process, brought years of experience in other organizations and what we really needed to be able to scale to that point.
So what are you up to now? 'Cause obviously you're in a financial situation that is, I'm assuming, pretty good, and in some ways, that limits what you can do. It limits what's even worth working on, but obviously, it also gives you the ability to work on things that you never could've earlier.
Yeah, so I spent a year kind of thinking about that, doing a lot of different stuff. I spent some time with 500 Startups and helping out with their Mountain View accelerator program, so working with some companies there. I've made a ton of angel investments, both prior and-- And now really, I've come to the realization, about a year-and-a-half after the sale, what we're gonna do is we want to buy a SaaS business, preferably in the SMB space, but we'll look at any SaaS business that's doing between half-a-million to a million or more in annual revenue. And we're gonna buy it and scale it, like we have. So that's gonna be our next step, and we'll still do investments and other stuff, but honestly, I've slowed down my angel investing and really targeted very specifically to companies like Groove, that I made an investment in, who is both profitable and didn't need the investment but was looking for both advice and structure in certain areas, and, you know, different, very specific things, and in doing so, those are the types of investments that I'll be making more of.
It sounds really fun, I think.
Yeah, I mean, hopefully, it is. Ask me in a few years and then see. Maybe it's totally changed, and I decide to do something different, but, you know, after a kind of year of thought and testing different things out, this seems like what is most interesting to me.
I want to wrap things up by hearing kind of your general advice for people who are indie hackers or aspiring entrepreneurs. What resources did you fine helpful, and what stories are inspiring to you, and what do you think somebody who's just starting out should know about?
Yeah, so I get this question a lot, and I've thought about it so many times. And one part of me felt like my advice should change over time maybe, right, like, because things change, industry changes, but I keep coming back to the same thing, which is I think the best piece of advice is just go out and do something, anything, right. Rather than talk about it, rather than, you know, wait and see what users think, just do something, right, because I would prefer to see an entrepreneur fail at something than fail at doing nothing, right. And I think that today we spend so much time reading, and there's so many resources, right. I could say go here, go there. There are so many great resources, blogs and podcasts and everything else. We can consume so much information that we get into this paralysis of like okay, like, all right, I need to know my next seven steps. Like, no, just do one step and do something, right. If it's build a product, if it's talk to a customer, if it's sell something, collect a credit card, like just each little step, just keep doing them, right.
Yeah, I meet a lot of people who are, they've been contemplating starting a business forever, and they're just consuming things, and they never really get around to actually starting. And so I think in terms of bang-for-your-buck advice, that's probably the most valuable just because of the sheer number of people who are on the fence.
Yeah, I wish I had better things, like, yo, go do these three hacks or go do this, right. But in reality, I think if you're doing something, you're getting up early in the day, and you're working hard, you'll find something that you both love and is successful. It may take a few failures. It may take time. There's no magic to it, but if you combine those things, you're at least on the right path. If you're not doing anything, and you have a schedule, and you're not eating well, like, you know, like nothing's gonna work. It doesn't matter if you're an entrepreneur or not.
Agreed. Well, cool, it was great having you on the podcast, David. I know we didn't get to talk about a lot of different stuff, Chargify, for example, and I've got about 20 other questions here that I did not get to ask, but maybe one day in the future, you can come back on.
Yeah, absolutely, and if anyone has any specific questions, feel free to reach out. I'm on Twitter. My email address is readily available online. I'm happy to talk to entrepreneurs that take the time to, you know, proactively reach out and have a real question or something I can help with.
All right, thanks, David.
Appreciate it, Courtland.
If you enjoyed this conversation, you should join David, me and other indie hackers and aspiring entrepreneurs in discussing this episode on the Indie Hackers forum. Just visit www.indiehackers.com/forum. Also, make sure to subscribe, so you know exactly when I release new episodes in the future.


Board Games are now a USD 15 billion market
Gone are the days where playing board games meant playing Scrabble with the grandparents, or Monopoly with kids in the neighborhood. Board games are now actively played across all ages and is a hobby that is growing at 15% a year.
Board Game Guru was built to help board gamers new and old find games that are right for them. With almost 100,000 games listed on BoardGameGeek.com alone, it is a daunting task to try and pick one to play. Board Game Guru aims to help anyone interested in playing board games to discover new games suited to their playstyle in an easily navigable way.
We have built multiple recommendation engines using deep learning models that can cater to both people who are new to board games and enthusiasts. On top of this we have built result filtering directly into the user experience, recognizing the fact that users are being presented with too many options rather than too few.
As a result, Board Game Guru is unlike any other recommendation engine online right now with a focus on user friendliness and a curated set of results:
The Dataset
The data powering our recommendation engine was scraped from BoardGameGeek.com, the foremost meeting place for board game enthusiasts online. With profiles for board games, user profile, and reviews, the site is a treasure trove of information for those looking to learn more about board games.
The initial dataset had 108,571 different games with 47,623,627 user reviews from 513,378 real users powers the recommendation engine.
To improve speed and reduce noise, we pared this down to a working dataset of 9,596 games that had at least 750 ratings.
This still left us with 486,469 users with over 40 million reviews to work with.
Information such as game type, category, number of players, playing time, recommended age, and designer/artist/publisher information is all included in the dataset to allow for more granular model buidling:
Using this data we built filters into our user interface that allowed users to narrow down recommendations by number of players, playing time, or recommended age, all according to how the underlying data was bucketed:
Using this data we built filters into our user interface that allowed users to narrow down recommendations by number of players, playing time, or recommended age, all according to how the underlying data was bucketed:
The Modeling
Board Game Guru uses two types of models to produce recommendations for users: Matrix Factorization and Item KNN.
For users with existing BoardGameGeek.com profiles with games already rated on it, a Matrix Factorization model is used. This is an algorithm used in many existing recommendation engines that uses Collaborative Filtering to discover relationships between users and board games through their user reviews, to make recommendations based on the reviews of similar users.
For users who are new to board games or do not have profiles on BoardGameGeek, an Item KNN model is used. This model computes the similarity between board games by using the observations of users who have interacted with both items.
The Site Architecture
The web app has been written in Python and is powered by a Flask app. Everything is hosted on AWS' Elastic Beanstalk, with surveys powered by Typeform:
Say Hi to our dedicated engineers who made the site!
This web application was completed as part of a W210 Capstone Project by students in the Master of Information and Data Science (MIDS) program at the University of California, Berkeley.
Shujing Dong
Vidhu NATH
Mike SHUM
Anu Yadav


| Day Ahead Market | |
| Offered (MUs) | Cleared (MUs) |
| No internet Connection | no internet connection |
| Real Time Market | |
| Offered (MUs) | Cleared (MUs) |
| No internet Connection | no internet connection |
| Merchant Power (DAM) | |
| Offered (MUs) | Cleared (MUs) |
| No internet Connection | no internet connection |
| DAM Results |
| RTM Results |
| RTM vs DAM Chart |
| RTM vs DAM Comparision |


class: center, middle ### W4995 Applied Machine Learning # Model Interpretation and Feature Selection 03/04/20 Andreas C. Müller ??? Alright, everybody. Today we will talk about Feature selection and model interpretation, look at how to do this in general, and how to doit with scikit-learn. We're also going to look into a couple of other Python libraries that help doing these two things. FIXME motivation for model interpretation! FIXME random forest code is confusing because I tried to simplify it. Also: we haven't seen those yet?! FIXME use / explain permutation importance FIXME use seaborn clustermap for correlation matrix plot FIXME add diagram for permutation importance FIXME explain when and how to interpret coeffients FIXME mutual information needs better explanation FIXME lasso show random between correlated, same for tree, compare to RF FIXME IceBOX example with cross? FIXME should be two lectures really! FIXME maybe drop shap and shaply FIXME better explanation for ice-box and partial dependence FIXME partial dependence definitely needs a process diagram FIXME remove boston FIXME icebox do real simpsons paradox example FIXME unsupervised feature selection terrible? FIXME why is shap in the wrong direction? Why spurious importances? --- class: spacious # Model Interpretation (post-hoc)? - ## Not inference! - ## Not causality! - ## still useful(?) ??? Possibly useful for feature selection, feature engineering, model debugging, explaining predictions. Alternatively: use model that are easy to interpret! OR: derive a simpler explainable model from a non-explainable one (model compression) Caveat: if your model is bad, interpreting it makes no sense! --- # Types of explanations ##Explain model globally - How does the output depend on the input? - Often: some form of marginals ## Explain model locally - Why did it classify this point this way? - Explanation could look like a "global" one but be different for each point. - "What is the minimum change to classify it differently?" ??? Marginals: how does the prediction change as function of a particular Features? --- class: spacious # Explaning the model $\neq$ explaining the data - model inspection only tells you about the model - the model might not accurately reflect the data --- # "Features important to the model"? Naive: - `coef_` for linear models (abs value or norm for multi-class) - `feature_importances_` for tree-based models Use with care! --- # Linear Model coefficients - Relative importance only meaningful after scaling - Correlation among features might make coefficients completely uninterpretable - L1 regularization will pick one at random from a correlated group - Any penalty will invalidate usual interpretation of linear coefficients --- # Drop Feature Importance `$$I_i^\text{drop} = \text{Acc}(f, X, y) - \text{Acc}(f', X_{-i}, y)$$` .smallest[ ```python def drop_feature_importance(est, X, y): base_score = np.mean(cross_val_score(est, X, y)) scores = [] for feature in range(X.shape[1]): mask = np.ones(X.shape[1], 'bool') mask[feature] = False X_new = X[:, mask] this_score = np.mean(cross_val_score(est, X_new, y)) scores.append(base_score - this_score) return np.array(scores) ```] .smallest[ - Doesn't really explain model (refits for each feature) - Can't deal with correlated features well - Very slow - Can be used for feature selection ] ??? on it's own not very useful, we'll see it later rebuilds many models, not explaining this particular model! --- # Permutation importance Idea: measure marginal influence of one feature `$$I_i^\text{perm} = \text{Acc}(f, X, y) - \mathbb{E}_{x_i}\left[\text{Acc}(f(x_i, X_{-i}), y)\right]$$` ```python def permutation_importance(est, X, y, n_repeat=100): baseline_score = estimator.score(X, y) for f_idx in range(X.shape[1]): for repeat in range(n_repeat): X_new = X.copy() X_new[:, f_idx] = np.random.shuffle(X[:, f_idx]) feature_score = estimator.score(X_new, y) scores[f_idx, repeat] = baseline_score - feature_score ``` -- - Applied on validation set given trained estimator. - Also kinda slow. --- # LIME .smaller[ - Build sparse linear local model around each data point. - Explain prediction for each point locally. - Paper: "Why Should I Trust You?" Explaining the Predictions of Any Classifier - Implementation: ELI5, https://github.com/marcotcr/lime ] .center[ ![:scale 60%](images/lime_paper.png) ] --- # SHAP - Build around idea of Shapley values - very roughly: does drop-out importance for every subset of features - intractable, sampling approximations exists - fast variants for linear and tree-based models - awesome vis and tools: https://github.com/slundberg/shap - allows local / per sample explanations --- class: middle # Case Study --- # Toy data ```python X.shape ``` ``` (100000, 8) ``` .wide-left-column[ ![:scale 100%](images/toy_data_scatter.png) ] .narrow-right-column[ ![:scale 100%](images/covariance.png) ] --- class: compact # Models on lots of data .smallest[ ```python lasso = LassoCV().fit(X_train, y_train) lasso.score(X_test, y_test) ``` 0.545 ```python ridge = RidgeCV().fit(X_train, y_train) ridge.score(X_test, y_test) ``` 0.545 ```python lr = LinearRegression().fit(X_train, y_train) lr.score(X_test, y_test) ``` 0.545 ```python param_grid = {'max_leaf_nodes': range(5, 40, 5)} grid = GridSearchCV(DecisionTreeRegressor(), param_grid, cv=10, n_jobs=3) grid.fit(X_train, y_train) grid.score(X_test, y_test) ``` 0.545 ```python rf = RandomForestRegressor(min_samples_leaf=5).fit(X_train, y_train) rf.score(X_test, y_test) ``` 0.542 ] --- # Coefficients and default feature importance ![:scale 100%](images/standard_importances.png) ??? - tree and RF have no directionality in importances - lasso (and to some degree tree) selected randomly among correlated - RF assigns high importance to random features --- # Permutation Importances .center[ ![:scale 50%](images/standard_importances.png) ![:scale 50%](images/permutation_importance_big.png) ] --- # SHAP values .center[ ![:scale 50%](images/standard_importances.png) ![:scale 50%](images/shap_big.png) ] --- class: middle # More model inspection --- # Partial Dependence Plots - Marginal dependence of prediction on one (or two features) `$$f_i^{\text{pdp}}(x_i) = \mathbb{E}_{X_{-i}}\left[f(x_i, x_{-i})\right]$$` - Idea: Get marginal predictions given feature. - How? "integrate out" other features using validation data - Fast methods available for tree-based models (doesn't require validation data) - Nonsensical for linear models. --- class: compact # Partial Dependence .tiny[ ```python from sklearn.inspection import plot_partial_dependence boston = load_boston() X_train, X_test, y_train, y_test = train_test_split(boston.data, boston.target,random_state=0) gbrt = GradientBoostingRegressor().fit(X_train, y_train) fig, axs = plot_partial_dependence(gbrt, X_train, np.argsort(gbrt.feature_importances_)[-6:], feature_names=boston.feature_names) ``` ] -- .center[ ![:scale 60%](images/feat_impo_part_dep.png) ] ??? But there's also something else that's quite interesting, which is called Partial Dependence Plots that's actually possible for all trees. Unfortunately, in scikit-learn, it’s available only for the gradient boosting. So the idea here is to not only see what parameters are important but how will they influence the target. And so after you fit your model, I'm using the Boston data set to the gradient boosting regressor, there's a thing called plot partial dependence, which gets the model and the training data set and then the features for which I want to make the partial dependence plots. So here, I'm looking at the six most important features. The feature importance is important according to the trees. So I sort this and take the six most important ones. And so this is what the partner dependence plots look like. So this is the most important feature, the second most important feature and so on. This is sort of the marginal contribution of each feature. Basically, in each tree, you're summing out the contribution of all the other features, and you look only at what is the contribution of this particular feature. In general, this would be hard to compute. For tree-based models you can compute is a very efficient way because you can basically just sum up over the whole space by traversing the tree. Here with increased room size, the price increases and you can see how it increases and you can see that there's like a big step function here. And you can see that for increased LSTAT, the price decreases. For the others, there aren’t any drastic effect. There seems to be some threshold for NOX. And something funky going on DIS. The question is, so I said for random forest, you can't find out the direction of which way the feature influences. But that was for the feature importance. So the feature importance tells the impurity decrease. So here I also look at the feature importance and they're just numbers so they don't give you the direction. For both gradient boosting and random forest, you can look at partial dependency plot, and they will give you a non-parametric model of how a single feature influences it. Unfortunately, it’s not available in scikit-learn. You could do the same thing for random forests. Because in the end, the way they predict is quite similar. --- # Bivariate Partial Dependence Plots .smaller[ ```python plot_partial_dependence( gbrt, X_train, [np.argsort(gbrt.feature_importances_)[-2:]], feature_names=boston.feature_names, n_jobs=3, grid_resolution=50) ``` ] .center[ ![:scale 80%](images/feature_importance.png) ] ??? You can also do this in bivariate. So here, looking at the two most important features in a bivariate plot, I'm actually not giving it a list of features, but I'm giving it a list of list of features. And so the two most important features, LSTAT, and ROOM. You can see there’s no very complex interaction. As you can see, the effect of these two features taking into account all the other features. This is something you can usually do for linear model easily. If you do this for the linear model, you would have lines everywhere but for more complicated models this is very hard to do. Way more tricky to do in neural networks. --- # Partial Dependence for Classification .tiny-code[ ```python from sklearn.inspection import plot_partial_dependence for i in range(3): fig, axs = plot_partial_dependence(gbrt, X_train, range(4), n_cols=4, feature_names=iris.feature_names, grid_resolution=50, label=i) ``` ] .center[ ![:scale 50%](images/feat_impo_part_dep_class.png) ] ??? We can do the same for classification. Here is a partial dependency plot for the iris dataset. As I said, for classification we’re usually using One Versus Rest. So here, we have three classifiers. One classifier for sertosa, one for Versicolor and one for virginica. You can see that for the first two features they're like flat. But for sertosa the decision function is high for small part petal length, medium peddling for versicolor and large for virginca. These are all put through logistic function and then normalized. --- # PDP Caveats .left-column[ ![:scale 100%](images/pdp_failure.png) ] -- .right-column[ ![:scale 100%](images/pdp_failure_data.png) ] --- class:compact # Ice Box .smaller[ - like partial dependence plots, without the `.mean(axis=0)` ] .center[ ![:scale 60%](images/ice_cross.png) ] .tiny[ - https://pdpbox.readthedocs.io/en/latest/ - https://github.com/AustinRochford/PyCEbox - https://github.com/scikit-learn/scikit-learn/pull/16164 ] --- class: center, middle # Feature Selection ??? Alright, so let's talk about automatic feature selection. I want to talk about what methods are there to determine what features are important for your dataset, what features are important for a particular model. --- class: spacious # Why Select Features? - Avoid overfitting (?) - Faster prediction and training - Less storage for model and dataset - More interpretable model ??? There's a couple of reasons why you might want to feature selection. One is to avoid overfitting and get a better model. In practice, I have rarely seen that happen. It's not usually what I would try to do to increase performance. If I’m only interested in performance I probably would not try to do automatic feature selection unless I think only a very small subset of my feature is actually important. There's a lot of increasing performance just by selecting only important features. What I think is more commonly, the reason to do automatic feature selection is you want to shrink your model to make faster predictions, to train your model faster, to store fewer data and possibly to collect fewer data. If you're collecting the data or to feature from some online process, maybe it means you need fewer features, you need to store less information. I think actually the top reason to do feature selection is to have a more interpretable model. If your model is smaller, if your model has 5 features instead of 500, it's probably much easier for you to grasp what the model does. If you can have a model that is as good with way fewer features, it will be much easier to explain and most people will be happy with it. --- class: spacious # Types of Feature Selection - Unsupervised vs Supervised - Univariate vs Multivariate - Model based or not ??? There's a couple of different types of feature selection that I want to talk about. You can have supervised and unsupervised feature selection. Depending on whether you consider the target or not. You can have Univariate versus Multivariate feature selection. Whereas Univariate looks at each feature at a time and determines if it’s important. Multivariate looks at interactions as well. And you can have a feature selection based on a particular machine learning model or not. --- class: spacious # Unsupervised Feature Selection - May discard important information - Variance-based: 0 variance or mostly constant - Covariance-based: remove correlated features - PCA: remove linear subspaces ??? So the simpler thing that you might try is to do unsupervised feature selection which means just discard some features based on the statistics. For example, you could remove features that are mostly constant or that have zero variance. Like if they're always constant, then definitely they're not going to help you and you can remove them. If they have very small variance, on the other hand, that might only mean the scale of the features is small and you should just re-scale the feature. So just because it has a small variance, it doesn't really mean anything. People often discard covariance features. But maybe just the difference between these two correlated features was the thing that's important for prediction. If you use any unsupervised method, then you don't know what are the things that are actually important are. Similarly, with PCA, it’s not really feature selection, because it doesn't select subsets of the original feature. PCA will always use all original features. But it will remove linear subspaces of the feature space. And again, a lot of people like this to reduce the dimensionality and you will get the same things like a smaller model, maybe not more interpretable. It might be that the information you discarded is just the information that's important. In a covariance sense, just because the data doesn't extend a lot in a particular direction doesn't mean that this direction isn't the most important one for your prediction problem. --- # Covariance .smaller[ ```python from sklearn.preprocessing import scale boston = load_boston() X, y = boston.data, boston.target X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0) X_train_scaled = scale(X_train) cov = np.cov(X_train_scaled, rowvar=False) ``` ] .center[ ![:scale 32%](images/img_17.png) ] ??? Here's what the covariance matrix looks like for the Boston housing data set. The way you could do covariance based feature selection is you look at the features that have the highest covariance and just drop one of them or you could sum over all the features and look at which features most correlated with the other ones and drop that. --- ```python from scipy.cluster import hierarchy order = np.array(hierarchy.dendrogram( hierarchy.ward(cov),no_plot=True)['ivl'], dtype="int") ``` .center[ ![:scale 90%](images/img_19.png) ] ??? So if you look at covariance, it should probably try to sort your features using some clustering. Here I'm using hierarchical clustering from scipy. On the right, it’s the covariance matrix and on the left, it's also the covariance matrix where I reordered the columns and rows. You can see that there are three correlated blocks. You could do this automatically, or you could look at it and see what you think how many features are there, and how many of them are correlated. I really urge you, whenever you look at correlation, never look at correlation without resorting the columns. On the right-hand side here, maybe I can see that these two are correlated, but I can't see anything else. Whereas on the left hand I can see much more clearly what the structure of the data is. What it did just was, it did clustering on the rows and columns to resort them. --- class: center, middle # Supervised Feature Selection ??? --- # Univariate Statistics .smaller[ - Pick statistic, check p-values ! - f_regression, f_classsif, chi2 in scikit-learn ```python from sklearn.feature_selection import f_regression f_values, p_values = f_regression(X, y) ``` ] .center[ ![:scale 60%](images/img_20.png) ] ??? So the simplest way is univariate features selection without models. The most classical one is to use statistical test and see the ones that are significantly related to the target. Depending on whether you look through classification or regression, you can add a t-test, f-test or chi-squared test and you can say which are the features that are significantly related with a target and I'm just going to select these. So this is again here for the Boston housing dataset, using F regression and F test. You can see here F values and the P values and you could use either of them to select a subset of the features. So for example, the number of rooms and LSTAT had very high F values, very small P values. These are certainly the most important features. While this one here doesn't seem very important. One reason why this is not very important is because this is the binary variable and so this assumes linear regression model and the linear regression model is not very good at exploiting the binary variable. This is a super quick test to do. It's very fast, it's probably something you want to look at. But it assumes a linear model, which you might not want to assume. --- .smaller[ ```python from sklearn.feature_selection import SelectKBest, SelectPercentile, SelectFpr from sklearn.linear_model import RidgeCV select = SelectKBest(k=2, score_func=f_regression) select.fit(X_train, y_train) print(X_train.shape) print(select.transform(X_train).shape) ``` ``` (379, 13) (379, 2) ``` ] -- .smaller[ ```python all_features = make_pipeline(StandardScaler(), RidgeCV()) np.mean(cross_val_score(all_features, X_train, y_train, cv=10)) ``` 0.718 ] -- .smaller[ ```python select_2 = make_pipeline(StandardScaler(), SelectKBest(k=2, score_func=f_regression), RidgeCV()) np.mean(cross_val_score(select_2, X_train, y_train, cv=10)) ``` 0.624 ] ??? If you want to use these univariate statistics in scikit-learn there’s a couple of tools to select features based on this. In feature selection, there's a whole bunch of them. There select k best which selects the K best feature, you can specify the number of features you want. Select percentile, which selects a percentile that you want and then there's select FPR, it controls for the false positive rate, it basically does the multiple hypothesis testing adjustments to make sure that you false discovery rate of thinking the features are significantly important is low. These are scikit-learn transformers, you can instantiate them. By default, the parameter they all use a score function for classification. So if you want to do regression, you need to set the score function to F regression because you need different tests for regression classification. I said linear regression is not for binary features, maybe I shouldn't have formulated in that way. Linear regression doesn't allow you to do interactions, which is if you have multiple binary features would be the only issue. It's more sort of that this linear test will not put a lot of emphasis on a binary feature because the way the test works. I can obviously also put this in a pipeline. Here, I’ve used a centered scale for ridge in a pipeline for a Boston housing data set and here, I’ve used two best features. You can see it actually got much worse because two features are not enough to select or to express all the information. --- # Mututal Information ```python from sklearn.feature_selection import mutual_info_regression scores = mutual_info_regression(X_train, y_train, discrete_features=[3]) ``` .center[ ![:scale 90%](images/img_22.png) ] ??? Another univariate statistics that you can use which is a little bit more complicated, which is called Mutual Information. There's a version for regression classification in scikit-learn. This doesn't use a linear model. It uses a non-parametric model using nearest neighbors. So basically, this also works if the interaction is nonlinear and it works on discrete and continuous features, but you have to tell us which features are discrete. So here, in this case, I tell it the feature number three is discrete and then this gives me some scores telling me what's the mutual information between this feature in the target. Here I'm comparing the F values off the standard regression tests with the mutual information and they are sort of similar, but not entirely. If you think there are nonlinear interactions, and you have a model that can capture nonlinear interactions, then doing feature selection like taking these nonlinear features into account is good. This is much more computationally intensive than doing the F statistics. But this is still Univariate, looking at one feature at a time. --- class: spacious #Model-Based Feature Selection - Get best fit for a particular model - Ideally: exhaustive search over all possible combinations - Exhaustive is infeasible (and has multiple testing issues) - Use heuristics in practice. ??? So now let's look at multiple features at a time. Most of the things that look at multiple features at a time are model-based. Usually, model-based feature selection finds the subset of features on which this model performs best. So giving them a particular model like a linear model, or random forest, I want to find a subset of features for which this model performs best in terms of cross-validation performance. Ideally, to do that, I would do an exhaustive search over all possible subsets of the features. But that's like exponentially many. Also, if I do so many models fits I might overfit. So instead, there are several heuristics you can use to basically shrink or grow the sets of features that you're using. So first you fix the model. For models that give you some measure of feature importance, there's a very simple technique which just looks at how important the models feature is. --- # Model based (single fit) .smaller[ - Build a model, select "features important to model" - Lasso, other linear models, tree-based Models - Multivariate - linear models assume linear relation ] .smaller[ ```python from sklearn.linear_model import LassoCV X_train_scaled = scale(X_train) lasso = LassoCV().fit(X_train_scaled, y_train) print(lasso.coef_) ``` [-0.881 0.951 -0.082 0.59 -1.69 2.639 -0.146 -2.796 1.695 -1.614 -2.133 0.729 -3.615] ] .center[ ![:scale 55%](images/img_23.png) ] ??? Using, say, a linear model, you fit the single model, and you discard all the features that the model doesn't think are important. So this works for linear models really well and for tree-based models. This allows you to take linear interactions into account and with trees it allows you to take arbitrary interactions into account. So for example, I can use lasso, I can fit the lasso on my model, and I can look at the coefficients and the things that have small coefficients are less important in some sense, and so I could discard some of them. Here again, I plot the F values versus the coefficients of lasso. For example, here you can see for this variable, Lasso thinks its way less important than Univariate selection. Maybe because it was explained already by a combination of the other features. So if you have very co-related features Univariate selection will give all of them the same importance whereas lasso will usually pick only one of them. If you have many co-related features lasso picks only one of them at random. So it doesn't mean that other features are not important. Question is what does this purple don't represent? It represents that both the points are overlapping entirely. Here I use lassoCV which means it adjusted the regularization parameter in Lasso to make the optimum predictions. --- # Changing Lasso alpha .smaller[ ```python from sklearn.linear_model import Lasso X_train_scaled = scale(X_train) lasso = Lasso().fit(X_train_scaled, y_train) print(lasso.coef_) ``` [-0. 0. -0. 0. -0. 2.529 -0. -0. -0. -0.228 -1.701 0.132 -3.606] ] .center[ ![:scale 80%](images/img_24.png) ] ??? If I look at the different alpha, it might be quite different. For example here with a default alpha of one, a lot of the coefficients are exactly zero and you can see it would only select five. How to select the alpha in lasso is important for the feature selection and so you might want to grid search. Now, let’s say we want to use this and so we want a scikit-learn estimator that does this. So we can put it in a pipeline. --- class: smaller # SelectFromModel ```python from sklearn.feature_selection import SelectFromModel select_lassocv = SelectFromModel(LassoCV(), threshold=1e-5) select_lassocv.fit(X_train, y_train) print(select_lassocv.transform(X_train).shape) ``` ``` (379,11) ``` -- ```python pipe_lassocv = make_pipeline(StandardScaler(), select_lassocv, RidgeCV()) np.mean(cross_val_score(pipe_lassocv, X_train, y_train, cv=10)) np.mean(cross_val_score(all_features, X_train, y_train, cv=10)) ``` ``` 0.717 0.718 ``` -- ```python # could grid-search alpha in lasso select_lasso = SelectFromModel(Lasso()) pipe_lasso = make_pipeline(StandardScaler(), select_lasso, RidgeCV()) np.mean(cross_val_score(pipe_lasso, X_train, y_train, cv=10)) ``` ``` 0.671 ``` ??? SelectFrom model will work with any model with coef_ or feature_importances_. So that’s any linear model or tree based model. It uses this to select the features. So this will fit lasso and then if you call transform, it will discard all the features that lasso thought are not important. And you can change the threshold here, for example, so in this case, here I want to discard everything that lasso put to zero so I put a very small threshold in there. I could also set the threshold to be the median which selects 50% of the features. Here we are possibly discarding multiple features at once. SelectFromModel fits a single model, gets the feature importance was from this model and then drops according to the feature importance. And so here is how it looks in the pipeline, for example, you can see here that lasso with these parameters, drop two of the features and 11 after 13 remains. If I put it in a pipeline the performance is about the same. --- class: spacious # Iterative Model-Based Selection - Fit model, find least important feature, remove, iterate. - Or: Start with single feature, find most important feature, add, iterate. ??? But maybe dropping multiple features at once is not a good idea because the importance of the features might change once we dropped some of them. What we can do is, we can iteratively build models. And we could either start with a single feature then add more important ones or we could start with all features and discard some. We'll talk about these two strategies which are slightly different. --- class: spacious # Recursive Feature Elimination - Uses feature importances / coefficients, similar to “SelectFromModel” - Iteratively removes features (one by one or in groups) - Runtime: (n_features - n_feature_to_keep) / stepsize ??? So the next step from what we just saw would be recursive feature elimination. So if we use SelectFromModel it will drop all the unimportant features at once, in a recursive feature elimination usually drops one feature at a time or as a parameter step size it drops step size features at a time. So you fit the model, you discard the least important feature, you fit the model, you discard the important feature and so on until you have as many features left as you want again. Again, this needs the model to meet some measure of feature importance so you can use this with the linear model or tree-based model. An example where you cannot use this is, at least not in scikit-learn is Kernel SVM or neural networks, because they don't really give you a measure of feature importance easily. For each step feature that we remove, we need to train a new model. So this is much more expensive because we need to retrain the model many times, but we're sort of being more careful in how we remove the features. --- .smaller[ ```python from sklearn.linear_model import LinearRegression from sklearn.feature_selection import RFE # create ranking among all features by selecting only one rfe = RFE(LinearRegression(), n_features_to_select=1) rfe.fit(X_train_scaled, y_train) rfe.ranking_ ``` array([ 9, 8, 13, 11, 5, 2, 12, 4, 7, 6, 3, 10, 1]) ] .center[ ![:scale 95%](images/img_27.png) ] ??? This is implemented in RFE, for recursive feature elimination, it works similar to SelectFromModel. To do RFE, then the model that you want to use and then you can specify how many features you want to select. Here I'm comparing the ranking according to RFE. So when it dropped out the features to the linear regression coefficients, which is basically what I would use if I drop them off all at once, and you can see that they're sort of similar. So there's probably not a whole lot of difference, at least in this case. You need to specify how many features you want to select. But if you want to select only one feature, you have to build many models and drop all the other features and so on the way you tried out the model for keeping five features and so on. So if you want a grid search this, doing this independently would be a whole waste of time because they would do the same thing over and over again. --- # RFECV .smaller[ ```python from sklearn.linear_model import LinearRegression from sklearn.feature_selection import RFECV rfe = RFECV(LinearRegression(), cv=10) rfe.fit(X_train_scaled, y_train) print(rfe.support_) print(boston.feature_names[rfe.support_]) ``` ``` [ True True False True True True False True True True True True True] ['CRIM' 'ZN' 'CHAS' 'NOX' 'RM' 'DIS' 'RAD' 'TAX' 'PTRATIO' 'B' 'LSTAT'] ``` ```python pipe_rfe_ridgecv = make_pipeline(StandardScaler(), RFECV(LinearRegression(), cv=10), RidgeCV()) np.mean(cross_val_score(pipe_rfe_ridgecv, X_train, y_train, cv=10)) ``` ``` 0.710 ``` ] ??? So there's a thing called RFECV that allows you to do efficient grid search for the number of features to keep. This basically has built-in cross-validation, provides the number of features to keep. I’ve done RFECVC with linear regression and set it to do 10 fold cross-validation and then it will do the recursive feature elimination inside cross-validation and so it goes down from all features to just having one feature and with cross-validation, it will select the best number. --- .smaller[ ```python pipe_rfe_ridgecv = make_pipeline(StandardScaler(), RFECV(LinearRegression(), cv=10), RidgeCV()) np.mean(cross_val_score(pipe_rfe_ridgecv, X_train, y_train, cv=10)) ``` ``` 0.710 ``` ```python from sklearn.preprocessing import PolynomialFeatures pipe_rfe_ridgecv = make_pipeline(StandardScaler(), PolynomialFeatures(), RFECV(LinearRegression(), cv=10), RidgeCV()) np.mean(cross_val_score(pipe_rfe_ridgecv, X_train, y_train, cv=10)) ``` ``` 0.820 ``` ] ??? If we want to predict with the same model as used for selection, RFECV can be used as the prediction step. Could also use RFECV as transformer and use any other model! --- class: spacious # Wrapper Methods - Can be applied for ANY model! - Shrink / grow feature set by greedy search - Called Forward or Backward selection - Run CV / train-val split per feature - Complexity: n_features * (n_features + 1) / 2 - Implemented in mlxtend ??? So as I set recourse to each elimination. So this is more careful and it requires the model that gives you feature importance. There's sort of more general reprimand sets that are, in a sense, even more careful, but also more expensive, but it can be applied to any model. These are sequential feature selection. The idea is to either start with zero features, and add the most important feature, or start with all features and remove the least important feature at a time. And you do this by not using a feature importance by the model. But actually each step you basically do like one step looking at search. So let's say I started with all the features, I leave out the first feature, build the model and but I don't only build a model, I do cross-validation of the model on the subset so I get some accuracy or R squared value. And I do this for every single feature. So I leave each feature out and look at the cross-validate accuracy. And I dropped the one that gives me the highest cost-validated accuracy. So now basically, this is even more expensive than doing the recursive feature elimination because I do a one step, look ahead. So now the runtime is quadratic in the number of features. Also for each iteration, I not only need to train a single model, I need to do cross-validation. This is like a pretty standard method, because it's like very general, and you can apply it to any model. It's like a brute force search. Right now it's not in scikit-learn, but it's in a package called mlxtend which has a couple other tools. Mlxtend is luckily fully scikit-learn compatible, so you can put it in a pipeline and everything's fine. --- # SequentialFeatureSelector .smaller[ ```python from mlxtend.feature_selection import SequentialFeatureSelector sfs = SequentialFeatureSelector(LinearRegression(), forward=False, k_features=7) sfs.fit(X_train_scaled, y_train) ``` ``` Features: 7/7 ``` ```python print(sfs.k_feature_idx_) print(boston.feature_names[np.array(sfs.k_feature_idx_)]) ``` ``` (1, 4, 5, 7, 9, 10, 12) ['ZN' 'NOX' 'RM' 'DIS' 'TAX' 'PTRATIO' 'LSTAT'] ``` ```python sfs.k_score_ ``` ``` 0.725 ``` ] ??? So from mlxtend, you get feature selection, sequential feature selector, you put into the model, you tell it whether you want to do forward or backward. Forward equal to false means I started with all and I prune features one by one. Forward equal to true means I start with zero features and I use the one that gives me highest accuracy and then I add one by one. As I said with sequential feature selector, you need to specify the number of features and it does internally cross-validation and it optimizes accuracy for classification models and r square for regression models and does this stepwise selection. One thing I should have mentioned if you do feature selection this way, theoretically, you can have a model for feature selection that's different from the model for prediction. In here, if you look at the top for the recursive feature elimination, I use linear regression. But actually, the model I fit in the end was a ridge model. So I could also use, let's say, a tree-based model for feature selection and then the linear model for prediction if I wanted to. It's not entirely clear if that helps or makes sense, but it's sort of an additional degree of freedom. And for example, if I want a very interpretable model, I want my model that does the predictions to be linear so I can explain to my boss what it means. But I can also still do the feature selection using a more complicated model and just tell my boss “Oh, only these features are important and here's how I make the prediction” --- class: center, middle # Questions ? ??? The question is if I do forward method or backward method with the same number of features will the same happen? They both are sort of one step look ahead approximations to trying out all subsets and there's no guarantee we would get the same thing.


|
My Project
|
Inherits MonoBehaviour.
Inherited by MegaNode.
Public Member Functions | |
| Vector3 | CalcRepulse (Node n) |
| void | ApplyForce (float dt) |
| void | AddEdge (Edge e) |
| void | UpdatePos () |
| void | clearVel () |
| void | recordHistory () |
| Vector3 | difFromHistory () |
Public Attributes | |
| int | id |
| int | index |
| int | megaID |
| float | mass |
| List< Edge > | edges |
| List< int > | neighbors |
| Transform | t |
| bool | simulating |
| Vector3 | force |
Static Public Attributes | |
| static float | forceScale |
Node is the basic graph node. It keeps track of edges and neighbors. Has some simple physics calculations
| void Node.AddEdge | ( | Edge | e | ) |
Just adds an edge to our list. Edges are created and initialized in Controller. Adds the neighbor the edge connects to.
| e | The edge to be added. |
| void Node.ApplyForce | ( | float | dt | ) |
Applies the previously calculated force to self, resets it for the next frame Contains some checks to stop degenerate behavior
| dt | The time step we are simulating (delta time). |
| Vector3 Node.CalcRepulse | ( | Node | n | ) |
Calculates a repulsion force from another node
| n | The node we're recieving a repulsive force from. |
| void Node.clearVel | ( | ) |
Sets the speed to zero
| Vector3 Node.difFromHistory | ( | ) |
Returns the difference between history position and current posisiton (local)
| void Node.recordHistory | ( | ) |
Saves current local position as history
| void Node.UpdatePos | ( | ) |
Actually moves the node using the velocity calculated with ApplyForce Lowers velocity as well. This helps reduce degenerate behavior, though it means the simulation takes longer.
| List<Edge> Node.edges |
List of all edges connected to this node
| Vector3 Node.force |
Contains the force that a node is affectd by each update tick
|
static |
Gets set by Controller. A multiplier on force
| int Node.id |
Identifier according to input file row number
| int Node.index |
index in Controller's list of nodes
| float Node.mass |
Should always be 1 for non-mega nodes. For force calculations
| int Node.megaID |
If added to a mega, it's the index in Controller.megas of the mega
| List<int> Node.neighbors |
Node indexes of nodes connected through edges
| bool Node.simulating |
If false, it won't be updated. When part of an active mega node, will be false.
| Transform Node.t |
The transform of the node


Connecting Irssi To Twitch.tv IRC
A couple of days ago Brandi decided to try streaming her latest obsession, Animal Crossing: New Horizons on Twitch. As the unofficial game of the coronavirus quarantine, it seemed like a good idea. When she told me, I decided to do the comfortable thing and turn on her stream from the Twitch app on my Amazon Fire TV Stick so I could watch it on my TV. This works well for most of the streams I might happen to watch because I never actually care about interacting with the chat. While the app on my Fire TV Stick does allow me to view the chat, trying to type anything via a remote’s D-pad and an on-screen keyboard on my TV would be painful to say the least.
In this particular instance, though, I wanted to be able to chat with Brandi and a few other friends who popped into her stream. I initially had her stream open on my phone and was chatting there, but it was annoying to have to keep waking my phone up to look at the chat. Then I opened her stream in a browser on my Pinebook Pro. I simply paused the stream since I didn’t need it running twice, but this left a tiny sidebar for the chat that was less than ideal.
Being a regular user of IRC, I was aware that Twitch chat is just an IRC channel under the hood; this is pretty obvious given that the bots most decent-sized Twitch streamers employ in their chats are just IRC bots. I’d know since I once wrote a really shitty IRC bot for an IRC server I ran for my team at a previous job. As such, I figured I could just connect a regular IRC client to Twitch’s IRC servers, find Brandi’s channel, and chat that way. Given that I already use IRC regularly, I have a server that I normally connect from via my favorite client, Irssi.
Getting the connection right took a little doing as finding up-to-date information on Twitch’s IRC setup is a bit spotty. I ended up combining information from a bunch of different sites before I got everything to work properly.
I’m not going to cover how to get and access Irssi; if you need help with that you can use their own documentation. Assuming you’ve got access to Irssi and you’ve just launched the application, the first thing to do is to define a new network for Twitch; for any of the below examples just replace what appears in [brackets] with your own information:
/network add -nick \[username\] Twitch
This defines a new IRC network in Irssi and states what your nickname will be for it, which is just your Twitch.tv username. You could technically name the network anything you like, you don’t have to call it “Twitch”, but simplicity is generally the best approach. The next step is to define the server information for that network.
/server add -auto -ssl -network Twitch irc.twitch.tv 6697 \[oauth password\]
This defines the server of irc.twitch.tv
for the Twitch IRC network, specifies the port to use, and gives the password to authenticate your account. Note: Your OAuth password is not the same as your regular password to log in to Twitch! You must set that up at the OAuth password setup page for Twitch. Then copy and paste the password value you’ve generated into the command above. You must leave it prefaced with “oauth:” just like how Twitch provides it to you in the command; don’t strip that off the front.
With all of that done, now you can simply connect to your newly defined IRC network via:
/connect Twitch
This should show you a message letting you know that you’ve successfully connected. The channel to join is simply the name of the Twitch account prefaced by a hashtag just like every other IRC channel. So to join a particular channel you would just type:
/join #favoriteStreamerName
That’s all there is to it! You can now happily chat with a stream from Irssi while watching the actual video feed on another device. I assume the steps above could be pretty easily translated into a different IRC client if you prefer something other than Irssi.
There are, however, just a couple of caveats to keep in mind. As you might imagine, there is no support for Twitch emotes within IRC. When someone else in the chat uses an emote, for example, you will simply see the plaintext rendition of it. So if someone uses the BibleThump emote, you will see that text rather than the character from The Binding of Isaac.
The other main caveat is that as a guest in the channel you cannot see the other participants of the chat from IRC if you check who is in the channel. You can, however, still see whatever messages they send, and Irssi will still auto-complete their names if you want to mention them. For example, when running Irssi’s /names
command to see a list of channel members, I see only myself:
As you can see in the screenshot above, a friend of ours is also actively using Twitch chat, but I am the only member listed in the channel. This isn’t a big ordeal once you realize it (unless you’re really jonesing to see everyone’s name for some reason), but it did trip me up initially since I had assumed that I was not connecting to the proper channel. I eventually left both IRC and Twitch chat open in a browser until someone else typed something so that I could verify the same message appeared in both places.
On the whole I found this to be a streamlined, elegant way to actually participate in Brandi’s Twitch chat from my low-end laptop while watching her actual video stream on my TV from the comfort of my couch. Plus, you just get to feel awesome for using IRC.