Screaming in the Cloud

Kelsey Hightower joins Corey on Screaming in the Cloud to discuss his reflections on how the tech industry is progressing. Kelsey describes what he’s been getting out of retirement so far, and reflects on what he learned throughout his high-profile career - including why feature sprawl is such a driving force behind the complexity of the cloud environment and the tactics he used to create demos that are engaging for the audience. Corey and Kelsey also discuss the importance of remaining authentic throughout your career, and what it means to truly have an authentic voice in tech.

About Kelsey

Kelsey Hightower is a former Distinguished Engineer at Google Cloud, the co-chair of KubeCon, the world’s premier Kubernetes conference, and an open source enthusiast. He’s also the co-author of Kubernetes Up & Running: Dive into the Future of Infrastructure. Recently, Kelsey announced his retirement after a 25-year career in tech.

Links Referenced:

• Twitter: https://twitter.com/kelseyhightower

Transcript

Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.

Corey: Do you wish there were cheat codes for database optimization? Well, there are – no seriously. If you’re using Postgres or MySQL on Amazon Aurora or RDS, OtterTune uses AI to automatically optimize your knobs and indexes and queries and other bits and bobs in databases. OtterTune applies optimal settings and recommendations in the background or surfaces them to you and allows you to do it. The best part is that there’s no cost to try it. Get a free, thirty-day trial to take it for a test drive. Go to ottertune dot com to learn more. That’s O-T-T-E-R-T-U-N-E dot com.

Corey: Welcome to Screaming in the Cloud. I’m Corey Quinn. You know, there’s a great story from the Bible or Torah—Old Testament, regardless—that I was always a big fan of where you wind up with the Israelites walking the desert for 40 years in order to figure out what comes next. And Moses led them but could never enter into what came next. Honestly, I feel like my entire life is sort of going to be that direction. Not the biblical aspects, but rather always wondering what’s on the other side of a door that I can never cross, and that door is retirement. Today I’m having returning guest Kelsey Hightower, who is no longer at Google. In fact, is no longer working and has joined the ranks of the gloriously retired. Welcome back, and what’s it like?

Kelsey: I’m happy to be here. I think retirement is just like work in some ways: you have to learn how to do it. A lot of people have no practice in their adult life what to do with all of their time. We have small dabs in it, like, you get the weekend off, depending on what your work, but you never have enough time to kind of unwind and get into something else. So, I’m being honest with myself. It’s going to be a learning curve, what to do with that much time.

You’re probably still going to do work, but it’s going to be a different type of work than you’re used to. And so, that’s where I am. 30 days into this, I’m in that learning mode, I’m on-the-job training.

Corey: What’s harder than you expected?

Kelsey: It’s not the hard part because I think mentally I’ve been preparing for, like, the last ten years, being a minimalist, learning how to kind of live within my means, learn to appreciate things that are just not work-related or status symbols. And so, to me, it felt like a smooth transition because I started to value my time more than anything else, right? Just waking up the next day became valuable to me. Spending time in the moment, right, you go to these conferences, there’s, like, 10,000 people, but you learn to value those one-on-one encounters, those one-off, kind of, let’s just go grab lunch situations. So, to me, retirement just makes more room for that, right? I no longer have this calendar that is super full, so I think for me, it was a nice transition in terms of getting more of that valuable time back.

Corey: It seems to me that you’re in a similar position to the one that I find myself in where the job that you were doing and I still am is tied, more or less, to a sense of identity as opposed to a particular task or particular role that you fill. You were Kelsey Hightower. That was a complete sentence. People didn’t necessarily need to hear the rest of what you were working on or what you were going to be talking about at a given conference or whatnot. So, it seemed, at least from the outside, that an awful lot of what you did was quite simply who you were. Do you feel that your sense of identity has changed?

Kelsey: So, I think when you have that much influence, when you have that much reputation, the words you say travel further, they tend to come with a little bit more respect, and so when you’re working with a team on new product, and you say, “Hey, I think we should change some things.” And when they hear those words coming from someone that they trust or has a name that is attached to reputation, you tend to be able to make a lot of impact with very few words. But what you also find is that no matter what you get involved in—configuration management, distributed systems, serverless, working with customers—it all is helped and aided by the reputation that you bring into that line of work. And so yes, who you are matters, but one thing that I think helped me, kind of greatly, people are paying attention maybe to the last eight years of my career: containers, Kubernetes, but my career stretches back to the converting COBOL into Python days; the dawn of DevOps, Puppet, Chef, and Ansible; the Golang appearance and every tool being rewritten from Ruby to Golang; the Docker era.

And so, my identity has stayed with me throughout those transitions. And so, it was very easy for me to walk away from that thing because I’ve done it three or four times before in the past, so I know who I am. I’ve never had, like, a Twitter bio that said, “Company X. X person from company X.” I’ve learned long ago to just decouple who I am from my current employer because that is always subject to change.

Corey: I was fortunate enough to not find myself in the public eye until I owned my own company. But I definitely remember times in my previous incarnations where I was, “Oh, today I’m working at this company,” and I believed—usually inaccurately—that this was it. This was where I really found my niche. And then surprise I’m not there anymore six months later for, either their decision, my decision, or mutual agreement. And I was always hesitant about hanging a shingle out that was tied too tightly to any one employer.

Even now, I was little worried about doing it when I went independent, just because well, what if it doesn’t work? Well, what if, on some level? I think that there’s an authenticity that you can bring with you—and you certainly have—where, for a long time now, whenever you say something, I take it seriously, and a lot of people do. It’s not that you’re unassailably correct, but I’ve never known you to say something you did not authentically believe in. And that is an opinion that is very broadly shared in this industry. So, if nothing else, you definitely were a terrific object lesson in speaking the truth, as you saw it.

Kelsey: I think what you describe is one way that, whether you’re an engineer doing QA, working in the sales department, when you can be honest with the team you’re working with, when you can be honest with the customers you’re selling into when you can be honest with the community you’re part of, that’s where the authenticity gets built, right? Companies, sometimes on the surface, you believe that they just want you to walk the party line, you know, they give you the lines and you just read them verbatim and you’re doing your part. To be honest, you can do that with the website. You can do that with a well-placed ad in the search queries.

What people are actually looking for are real people with real experiences, sharing not just fact, but I think when you mix kind of fact and opinion, you get this level of authenticity that you can’t get just by pure strategic marketing. And so, having that leverage, I remember back in the day, people used to say, “I’m going to do the right thing and if it gets me fired, then that’s just the way it’s going to be. I don’t want to go around doing the wrong thing because I’m scared I’m going to lose my job.” You want to find yourself in that situation where doing the right thing, is also the best thing for the company, and that’s very rare, so when I’ve either had that opportunity or I’ve tried to create that opportunity and move from there.

Corey: It resonates and it shows. I have never had a lot of respect for people who effectively are saying one thing today and another thing the next week based upon which way they think that the winds are blowing. But there’s also something to be said for being able and willing to publicly recant things you have said previously as technology evolves, as your perspective evolves and, in light of new information, I’m now going to change my perspective on something. I’ve done that already with multi-cloud, for example. I thought it was ridiculous when I heard about it. But there are also expressions of it that basically every company is using, including my own. And it’s a nuanced area. Where I find it challenging is when you see a lot of these perspectives that people are espousing that just so happen to deeply align with where their paycheck comes from any given week. That doesn’t ring quite as true to me.

Kelsey: Yeah, most companies actually don’t know how to deal with it either. And now there has been times at any number of companies where my authentic opinion that I put out there is against party line. And you get those emails from directors and VPs. Like, “Hey, I thought we all agree to think this way or to at least say this.” And that’s where you have to kind of have that moment of clarity and say, “Listen, that is undeniably wrong. It’s so wrong in fact that if you say this in public, whether a small setting or large setting, you are going to instantly lose credibility going forward for yourself. Forget the company for a moment. There’s going to be a situation where you will no longer be effective in your job because all of your authenticity is now gone. And so, what I’m trying to do and tell you is don’t do that. You’re better off saying nothing.”

But if you go out there, and you’re telling what is obviously misinformation or isn’t accurate, people are not dumb. They’re going to see through it and you will be classified as a person not to listen to. And so, I think a lot of people struggle with that because they believe that enterprise's consensus should also be theirs.

Corey: An argument that I made—we’ll call it a prediction—four-and-a-half years ago, was that in five years, nobody would really care about Kubernetes. And people misunderstood that initially, and I’ve clarified since repeatedly that I’m not suggesting it’s going away: “Oh, turns out that was just a ridiculous fever dream and we’re all going back to running bare metal with our hands again,” but rather that it would slip below the surface-level of awareness. And I don’t know that I got the timing quite right on that, I think it’s going to depend on the company and the culture that you find yourself in. But increasingly, when there’s an application to run, it’s easy to ask someone just, “Oh, great. Where’s the Kubernetes cluster live so we can throw this on there and just add it to the rest of the pile?”

That is sort of what I was seeing. My intention with that was not purely just to be controversial, as much fun as that might be, but also to act as a bit of a warning, where I’ve known too many people who let their identities become inextricably tangled with the technology. But technologies rise and fall, and at some point—like, you talk about configuration management days; I learned to speak publicly as a traveling trainer for Puppet. I wrote part of SaltStack once upon a time. But it was clear that that was not the direction the industry was going, so it was time to find something else to focus on. And I fear for people who don’t keep an awareness or their feet underneath them and pay attention to broader market trends.

Kelsey: Yeah, I think whenever I was personally caught up in linking my identity to technology, like, “I’m a Rubyist,” right?“, I’m a Puppeteer,” and you wear those names proudly. But I remember just thinking to myself, like, “You have to take a step back. What’s more important, you or the technology?” And at some point, I realized, like, it’s me, that is more important, right? Like, my independent thinking on this, my independent experience with this is far more important than the success of this thing.

But also, I think there’s a component there. Like when you talked about Kubernetes, you know, maybe being less relevant in five years, there’s two things there. One is the success of all infrastructure things equals irrelevancy. When flights don’t crash, when bridges just work, you do not think about them. You just use them because they’re so stable and they become very boring. That is the success criteria.

Corey: Utilities. No one’s wondering if the faucet’s going to work when they turn it on in the morning.

Kelsey: Yeah. So, you know, there’s a couple of ways to look at your statement. One is, you believe Kubernetes is on the trajectory that it’s going to stabilize itself and hit that success criteria, and then it will be irrelevant. Or there’s another part of the irrelevancy where something else comes along and replaces that thing, right? I think Cloud Foundry and Mesos are two good examples of Kubernetes coming along and stealing all of the attention from that because those particular products never gained that mass adoption. Maybe they got to the stable part, but they never got to the mass adoption part. So, I think when it comes to infrastructure, it’s going to be irrelevant. It’s just what side of that [laugh] coin do you land on?

Corey: It’s similar to folks who used to have to work at a variety of different companies on very specific Linux kernel subsystems because everyone had to care because there were significant performance impacts. Time went on and now there’s still a few of those people that very much need to care, but for the rest of us, it is below the level of things that we have to care about. For me, the signs of the unsustainability were, oh, you can run Kubernetes effectively in production? That’s a minimum of a quarter-million dollars a year in comp or up in some cases. Not every company is going to be able to field a team of those people and still remain a going concern in business. Nor frankly, should they have to.

Kelsey: I’m going to pull on that thread a little bit because it’s about—we’re hitting that ten-year mark of Kubernetes. So, when Kubernetes comes out, why were people drawn to it, right? Why did it even get the time of day to begin with? And I think Docker kind of opened Pandora’s box there. This idea of Chef, Puppet, Ansible, ten thousand package managers, and honestly, that trajectory was going to continue forever and it was helping no one. It was literally people doing duplicate work depending on the operating system you’re dealing with and we were wasting time copying bits to servers—literally—in a very glorified way.

So, Docker comes along and gives us this nicer, better abstraction, but it has gaps. It has no orchestration. It’s literally this thing where now we’ve unified the packaging situation, we’ve learned a lot from Red Hat, YUM, Debian, and the various package repo combinations out there and so we made this universal thing. Great. We also learned a little bit about orchestration through brute force, bash scripts, config management, you name it, and so we serialized that all into this thing we call Kubernetes.

It’s pretty simple on the surface, but it was probably never worthy of such fanfare, right? But I think a lot of people were relieved that now we finally commoditized this expertise that the Googles, the Facebooks of the world had, right, building these systems that can copy bits to other systems very fast. There you go. We’ve gotten that piece. But I think what the market actually wants is in the mobile space, if you want to ship software to 300 million people that you don’t even know, you can do it with the app store.

There’s this appetite that the boring stuff should be easy. Let’s Encrypt has made SSL certificates beyond easy. It’s just so easy to do the right thing. And I think for this problem we call deployments—you know, shipping apps around—at some point we have to get to a point where that is just crazy easy. And it still isn’t.

So, I think some of the frustration people express ten years later, they’re realizing that they’re trying to recreate a Rube Goldberg machine with Kubernetes is the base element and we still haven’t understood that this whole thing needs to simplify, not ten thousand new pieces so you can build your own adventure.

Corey: It’s the idea almost of what I’m seeing AWS go through, and to some extent, its large competitors. But building anything on top of AWS from scratch these days is still reminiscent of going to Home Depot—or any hardware store—and walking up and down the aisles and getting all the different components to piece together what you want. Sometimes just want to buy something from Target that’s already assembled and you have to do all of that work. I’m not saying there isn’t value to having a Home Depot down the street, but it’s also not the panacea that solves for all use cases. An awful lot of customers just want to get the job done and I feel that if we cling too tightly to how things used to be, we lose it.

Kelsey: I’m going to tell you, being in the cloud business for almost eight years, it’s the customers that create this. Now, I’m not blaming the customer, but when you start dealing with thousands of customers with tons of money, you end up in a very different situation. You can have one customer willing to pay you a billion dollars a year and they will dictate things that apply to no one else. “We want this particular set of features that only we will use.” And for a billion bucks a year times ten years, it’s probably worth from a business standpoint to add that feature.

Now, do this times 500 customers, each major provider. What you end up with is a cloud console that is unbearable, right? Because they also want these things to be first-class citizens. There’s always smaller companies trying to mimic larger peers in their segment that you just end up in that chaos machine of unbound features forever. I don’t know how to stop it. Unless you really come out maybe more Apple style and you tell people, “This is the one and only true way to do things and if you don’t like it, you have to go find an alternative.” The cloud business, I think, still deals with the, “If you have a large payment, we will build it.”

Corey: I think that that is a perspective that is not appreciated until you’ve been in the position of watching how large enterprises really interact with each other. Because it’s, “Well, what customer the world is asking for yet another way to run containers?” “Uh, this specific one and their constraints are valid.” Every time I think I’ve seen everything there is to see in the world of cloud, I just have to go talk to one more customer and I’m learning something new. It’s inevitable.

I just wish that there was a better way to explain some of this to newcomers, when they’re looking at, “Oh, I’m going to learn how this cloud thing works. Oh, my stars, look at how many services there are.” And then they wind up getting lost with analysis paralysis, and every time they get started and ask someone for help, they’re pushed in a completely different direction and you keep spinning your wheels getting told to start over time and time again when any of these things can be made to work. But getting there is often harder than it really should be.

Kelsey: Yeah. I mean, I think a lot of people don’t realize how far you can get with, like, three VMs, a load balancer, and Postgres. My guess is you can probably build pretty much any clone of any service we use today with at least 1 million customers. Most people never reached that level—I don’t even want to say the word scale—but that blueprint is there and most people will probably be better served by that level of simplicity than trying to mimic the behaviors of large customers—or large companies—with these elaborate use cases. I don’t think they understand the context there. A lot of that stuff is baggage. It’s not [laugh] even, like, best-of-breed or great design. It’s like happenstance from 20 years of trying to buy everything that’s been sold to you.

Corey: I agree with that idea wholeheartedly. I was surprising someone the other day when I said that if you were to give me a task of getting some random application up and running by tomorrow, I do a traditional three-tier architecture, some virtual machines, a load balancer, and a database service. And is that the way that all the cool kids are doing it today? Well, they’re not talking about it, but mostly. But the point is, is that it’s what I know, it’s where my background is, and the thing you already know when you’re trying to solve a new problem is incredibly helpful, rather than trying to learn everything along that new path that you’re forging down. Is that architecture the best approach? No, but it’s perfectly sufficient for an awful lot of stuff.

Kelsey: Yeah. And so, I mean, look, I’ve benefited my whole career from people fantasizing about [laugh] infrastructure—

Corey: [laugh].

Kelsey: And the truth is that in 2023, this stuff is so powerful that you can do almost anything you want to do with the simplest architecture that’s available to us. The three-tier architecture has actually gotten better over the years. I think people are forgotten: CPUs are faster, RAM is much bigger quantities, the networks are faster, right, these databases can store more data than ever. It’s so good to learn the fundamentals, start there, and worst case, you have a sound architecture people can reason about, and then you can go jump into the deep end, once you learn how to swim.

Corey: I think that people would be depressed to understand just how much the common case for the value that Kubernetes brings is, “Oh yeah, now we can lose a drive or a server and the application stays up.” It feels like it’s a bit overkill for that one somewhat paltry use case, but that problem has been hounding companies for decades.

Kelsey: Yeah, I think at some point, the whole ‘SSH is my only interface into these kinds of systems,’ that’s a little low level, that’s a little bare bones, and there will probably be a feature now where we start to have this not Infrastructure as Code, not cloud where we put infrastructure behind APIs and you pay per use, but I think what Kubernetes hints at is a future where you have APIs that do something. Right now the APIs give you pieces so you can assemble things. In the future, the APIs will just do something, “Run this app. I need it to be available and here’s my money budget, my security budget, and reliability budget.” And then that thing will say, “Okay, we know how to do that, and here’s roughly what is going to cost.”

And I think that’s what people actually want because that’s how requests actually come down from humans, right? We say, “We want this app or this game to be played by millions of people from Australia to New York.” And then for a person with experience, that means something. You kind of know what architecture you need for that, you know what pieces that need to go there. So, we’re just moving into a realm where we’re going to have APIs that do things all of a sudden.

And so, Kubernetes is the warm-up to that era. And that’s why I think that transition is a little rough because it leaks the pieces part, so where you can kind of build all the pieces that you want. But we know what’s coming. Serverless also hints at this. But that’s what people should be looking for: APIs that actually do something.

Corey: This episode is sponsored in part by Panoptica.  Panoptica simplifies container deployment, monitoring, and security, protecting the entire application stack from build to runtime. Scalable across clusters and multi-cloud environments, Panoptica secures containers, serverless APIs, and Kubernetes with a unified view, reducing operational complexity and promoting collaboration by integrating with commonly used developer, SRE, and SecOps tools. Panoptica ensures compliance with regulatory mandates and CIS benchmarks for best practice conformity. Privacy teams can monitor API traffic and identify sensitive data, while identifying open-source components vulnerable to attacks that require patching. Proactively addressing security issues with Panoptica allows businesses to focus on mitigating critical risks and protecting their interests. Learn more about Panoptica today at panoptica.app.

Corey: You started the show by talking about how your career began with translating COBOL into Python. I firmly believe someone starting their career today listening to this could absolutely find that by the time their career starts drawing to their own close, that Kubernetes is right in there as far as sounding like the deprecated thing that no one really talks about or thinks about anymore. And I hope so. I want the future to be brighter than the past. I want getting a business or getting software together in a way that helps people to not require the amount of, “First, spend six weeks at a boot camp,” or, “Learn how to write just enough code that you can wind up getting funding and then have it torn apart.”

What’s the drag-and-drop story? What’s the describe the application to a robot and it builds it for you? I’m optimistic about the future of infrastructure, just because based upon its power to potentially make reliability and scale available to folks who have no idea of what’s involved with that. That’s kind of the point. That’s the end game of having won this space.

Kelsey: Well, you know what? Kubernetes is providing the metadata to make that possible, right? Like in the early days, people were writing one-off scripts or, you know, writing little for loops to get things in the right place. And then we get config management that kind of formalizes that, but it still had no metadata, right? You’d have things like Puppet report information.

But in the world of, like, Kubernetes, or any cloud provider, now you get semantic meaning. “This app needs this volume with this much space with this much memory, I need three of these behind this load balancer with these protocols enabled.” There is now so much metadata about applications, their life cycles, and how they work that if you were to design a new system, you can actually use that data to craft a much better API that made a lot of this boilerplate the defaults. Oh, that’s a web application. You do not need to specify all of this boilerplate. Now, we can give you much better nouns and verbs to describe what needs to happen.

So, I think this is that transition as all the new people coming up, they’re going to be dealing with semantic meaning to infrastructure, where we were dealing with, like, tribal knowledge and intuition, right? “Run this script, pipe it to this thing, and then this should happen. And if it doesn’t, run the script again with this flag.” Versus, “Oh, here’s the semantic meaning to a working system.” That’s a game-changer.

Corey: One other topic I wanted to ask you about—I’ve it’s been on my list of things to bring up the next time I ran into you and then you went ahead and retired, making it harder to run into you. But a little while back, I was at a tech conference and someone gave a demo, and it didn’t go as well as they had hoped. And a few of us were talking about it afterwards. We’ve all been speakers, we’ve all lived that life. Zero shade.

But someone brought you up in particular—unprompted; your legend does precede you—and the phrase that they used was that Kelsey’s demos were always picture-perfect. He was so lucky with how the demos worked out. And I just have to ask—because you don’t strike me as someone who is not careful, particularly when all eyes are upon you—and real experts make things look easy, did you have demos periodically go wrong that the audience just didn’t see going wrong along the way? Or did you just actually YOLO all of your demos and got super lucky every single time for the last eight years?

Kelsey: There was a musician who said, “Hey, your demos are like jazz. You improvise the whole thing.” There’s no script, there’s no video. The way I look at the demo is, like, you got this instrument, the command prompt, and the web browser. You can do whatever you want with them.

Now, I have working code. I wrote the code, I wrote the deployment scenarios, I delete it all and I put it all back. And so, I know how it’s supposed to work from the ground up. And so, what that means is if anything goes wrong, I can improvise. I could go into fixing the code. I can go into doing a redeploy.

And I’ll give you one good example. The first time Kubernetes came out, there was this small meetup in San Francisco with just the core contributors, right? So, there is no community yet, there’s no conference yet, just people hacking on Kubernetes. And so, we decided, we’re going to have the first Kubernetes meetup. And everyone got, like, six, seven minutes, max. That’s it. You got to move.

And so, I was like, “Hey, I noticed that in the lineup, there is no ‘What is Kubernetes?’ talk. We’re just getting into these nuts and bolts and I don’t think that’s fair to the people that will be watching this for the first time.” And I said, “All right, Kelsey, you should give maybe an intro to what it is.” I was like, “You know what I’ll do? I’m going to build a Kubernetes cluster from the ground up, starting with VMs on my laptop.”

And I’m in it and I’m feeling confident. So, confidence is the part that makes it look good, right? Where you’re confident in the commands you type. One thing I learned to do is just use your history, just hit the up arrow instead of trying to copy all these things out. So, you hit the up arrow, you find the right command and you talk through it and no one looks at what’s happening. You’re cycling through the history.

Or you have multiple tabs where you know the next up arrow is the right history. So, you give yourself shortcuts. And so, I’m halfway through this demo. We got three minutes left, and it doesn’t work. Like, VMware is doing something weird on my laptop and there’s a guy calling me off stage, like, “Hey, that’s it. Cut it now. You’re done.”

I’m like, “Oh, nope. Thou shalt not go out like this.” It’s time to improvise. And so, I said, “Hey, who wants to see me finish this?” And now everyone is locked in. It’s dead silent. And I blow the whole thing away. I bring up the VMs, I [pixie 00:28:20] boot, I installed the kubelet, I install Docker. And everyone’s clapping. And it’s up, it’s going, and I say, “Now, if all of this works, we run this command and it should start running the app.” And I do kubectl apply-f and it comes up and the place goes crazy.

And I had more to the demo. But you stop. You’ve gotten the point across, right? This is what Kubernetes is, here’s how it works, and look how you do it from scratch. And I remember saying, “And that’s the end of my presentation.” You need to know when to stop, you need to know when to pivot, and you need to have confidence that it’s supposed to work, and if you’ve seen it work a couple of times, your confidence is unshaken.

And when I walked off that stage, I remember someone from Red Hat was like—Clayton Coleman; that’s his name—Clayton Coleman walked up to me and said, “You planned that. You planned it to fail just like that, so you can show people how to go from scratch all the way up. That was brilliant.” And I was like, “Sure. That’s exactly what I did.”

Corey: “Yeah, I meant to do that.” I like that approach. I found there’s always things I have to plan for in demos. For example, I can never count on having solid WiFi from a conference hall. The show has to go on. It’s, okay, the WiFi doesn’t work. I’ve at one point had to give a talk where the projector just wasn’t working to a bunch of students. So okay, close the laptop. We’re turning this into a bunch of question-and-answer sessions, and it was one of the better talks I’ve ever given.

But the alternative is getting stuck in how you think a talk absolutely needs to go. Now, keynotes are a little harder where everything has been scripted and choreographed and at that point, I’ve had multiple fallbacks for demos that I’ve had to switch between. And people never noticed I was doing it for that exact reason. But it takes work to look polished.

Kelsey: I will tell you that the last Next keynote I gave was completely irresponsible. No dry runs, no rehearsals, no table reads, no speaker notes. And I think there were 30,000 people at that particular Next. And Diane Greene was still CEO, and I remember when marketing was like, “Yo, at least a backup recording.” I was like, “Nah, I don’t have anything.”

And that demo was extensive. I mean, I was building an app from scratch, starting with Postgres, adding the schema, building an app, deploying the app. And something went wrong halfway. And there’s this joke that I came up with just to pass over the time, they gave me a new Chromebook to do the demo. And so, it’s not mine, so none of the default settings were there, I was getting pop-ups all over the place.

And I came up with this joke on the way to the conference. I was like, “You know what’d be cool? When I show off the serverless stuff, I would just copy the code from Stack Overflow. That’d be like a really cool joke to say this is what senior engineers do.” And I go to Stack Overflow and it’s getting all of these pop-ups and my mouse couldn’t highlight the text.

So, I’m sitting there like a deer in headlights in front of all of these people and I’m looking down, and marketing is, like, “This is what… this is what we’re talking about.” And so, I’m like, “Man do I have to end this thing here?” And I remember I kept trying, I kept trying, and came to me. Once the mouse finally got in there and I cleared up all the popups, I just came up with this joke. I said, “Good developers copy.” And I switched over to my terminal and I took the text from Stack Overflow and I said, “Great developers paste,” and the whole room start laughing.

And I had them back. And we kept going and continued. And at the end, there was like this Google Assistant, and when it was finished, I said, “Thank you,” to the Google Assistant and it was talking back through the live system. And it said, “I got to admit, that was kind of dope.” So, I go to the back and Diane Greene walks back there—the CEO of Google Cloud—and she pats me on the shoulder. “Kelsey, that was dope.”

But it was the thrill because I had as much thrill as the people watching it. So, in real-time, I was going through all these emotions. But I think people forget, the demo is supposed to convey something. The demo is supposed to tell some story. And I’ve seen people overdo their demos with way too much code, way too many commands, almost if they’re trying to show off their expertise versus telling a story. And so, when I think about the demo, it has to complement the entire narrative. And so, sometimes you don’t need as many commands, you don’t need as much code. You can keep things simple and that gives you a lot more ins and outs in case something does go crazy.

Corey: And I think the key takeaway here that so many people lose sight of is you have to know the material well enough that whatever happens, well, things don’t always go the way I planned during the day, either, and talking through that is something that I think serves as a good example. It feels like a bit more of a challenge when you’re trying to demo something that a company is trying to sell someone, “Oh, yeah, it didn’t work. But that’s okay.” But I’m still reminded by probably one of the best conference demo fails I’ve ever seen on video. One day, someone was attempting to do a talk that hit Amazon S3 and it didn’t work.

And the audience started shouting at him that yeah, S3 is down right now. Because that was the big day that S3 took a nap for four hours. It was one of those foundational things you’d should never stop to consider. Like, well, what if the internet doesn’t work tomorrow when I’m doing my demo? That’s a tough one to work around. But rough timing.

Kelsey: [breathy sound]

Corey: He nailed the rest of the talk, though. You keep going. That’s the thing that people miss. They get stuck in the demo that isn’t working, they expect the audience knows as much as they do about what’s supposed to happen next. You’re the one up there telling a story. People forget it’s storytelling.

Kelsey: Now, I will be remiss to say, I know that the demo gods have been on my side for, like, ten, maybe fifteen years solid. So, I retired from doing live demos. This is why I just don’t do them anymore. I know I’m overdue as an understatement. But the thing I’ve learned though, is that what I found more impressive than the live demo is to be able to convey the same narratives through story alone. No slides. No demo. Nothing. But you can still make people feel where you would try to go with that live demo.

And it’s insanely hard, especially for technologies people have never seen before. But that’s that new challenge that I kind of set up for myself. So, if you see me at a keynote and you’ve noticed why I’ve been choosing these fireside chats, it’s mainly because I’m also trying to increase my ability to share narrative, technical concepts, but now in a new form. So, this new storytelling format through the fireside chat has been my substitute for the live demo, normally because I think sometimes, unless there’s something really to show that people haven’t seen before, the live demo isn’t as powerful to me. Once the thing is kind of known… the live demo is kind of more of the same. So, I think they really work well when people literally have never seen the thing before, but outside of that, I think you can kind of move on to, like, real-life scenarios and narratives that help people understand the fundamentals and the philosophy behind the tech.

Corey: An awful lot of tools and tech that we use on a day-to-day basis as well are thankfully optimized for the people using them and the ergonomics of going about your day. That is orthogonal, in my experience, to looking very impressive on stage. It’s the rare company that can have a product that not only works well but also presents well. And that is something I don’t tend to index on when I’m selecting a tool to do something with. So, it’s always a question of how can I make this more visually entertaining? For while I got out of doing demos entirely, just because talking about things that have more staying power than a screenshot that is going to wind up being irrelevant the next week when they decide to redo the console for some service yet again.

Kelsey: But you know what? That was my secret to doing software products and projects. When I was at CoreOS, we used to have these meetups we would used to do every two weeks or so. So, when we were building things like etcd, Fleet was a container management platform that came before Kubernetes, we would always run through them as a user, start install them, use them, and ask how does it feel? These command line flags, they don’t feel right. This isn’t a narrative you can present with the software alone.

But once we could, then the meetups were that much more engaging. Like hey, have you ever tried to distribute configuration to, like, a thousand servers? It’s insanely hard. Here’s how you do with Puppet. But now I’m going to show you how you do with etcd. And then the narrative will kind of take care of itself because the tool was positioned behind what people would actually do with it versus what the tool could do by itself.

Corey: I think that’s the missing piece that most marketing doesn’t seem to quite grasp is, they talk about the tool and how awesome it is, but that’s why I love customer demos so much. They’re showing us how they use a tool to solve a real-world problem. And honestly, from my snarky side of the world and the attendant perspective there, I can make an awful lot of fun about basically anything a company decides to show me, but put a customer on stage talking about how whatever they’ve built is solving a real-world problem for them, that’s the point where I generally shut up and listen because I’m going to learn something about a real-world story. Because you don’t generally get to tell customers to go on stage and just make up a story that makes us sound good, and have it come off with any sense of reality whatsoever. I haven’t seen that one happen yet, but I’m sure it’s out there somewhere.

Kelsey: I don’t know how many founders or people building companies listen in to your podcast, but this is right now, I think the number one problem that especially venture-backed startups have. They tend to have great technology—maybe it’s based off some open-source project—with tons of users who just know how that tool works, it’s just an ingredient into what they’re already trying to do. But that isn’t going to ever be your entire customer base. Soon, you’ll deal with customers who don’t understand the thing you have and they need more than technology, right? They need a product.

And most of these companies struggle painting that picture. Here’s what you can do with it. Or here’s what you can’t do now, but you will be able to do if you were to use this. And since they are missing that, a lot of these companies, they produce a lot of code, they ship a lot of open-source stuff, they raise a lot of capital, and then it just goes away, it fades out over time because they can bring on no newcomers. The people who need help the most, they don’t have a narrative for them, and so therefore, they’re just hoping that the people who have all the skills in the world, the early adopters, but unfortunately, those people are tend to be the ones that don’t actually pay. They just kind of do it themselves. It’s the people who need the most help.

Corey: How do we monetize the bleeding edge of adoption? In many cases you don’t. They become your community if you don’t hug them to death first.

Kelsey: Exactly.

Corey: Ugh. None of this is easy. I really want to thank you for taking the time to catch up and talk about how you seen the remains of a career well spent, and now you’re going off into that glorious sunset. But I have a sneaking suspicion you’ll still be around. Where should people go if they want to follow up on what you’re up to these days?

Kelsey: Right now I still use… I’m going to keep calling it Twitter.

Corey: I agree.

Kelsey: I kind of use that for my real-time interactions. And I’m still attending conferences, doing fireside chats, and just meeting people on those conference floors. But that’s what where I’ll be for now. So yeah, I’ll still be around, but maybe not as deep. And I’ll be spending more time just doing normal life stuff, maybe less building software.

Corey: And we will, of course, put a link to that in the show notes. Thank you so much for taking the time to catch up and share your reflections on how the industry is progressing.

Kelsey: Awesome. Thanks for having me, Corey.

Corey: Kelsey Hightower, now gloriously retired. I’m Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you’ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you’ve hated this podcast, please leave a five-star review on your podcast platform of choice along with an angry comment that you’re going to type on stage as part of a conference talk, and then accidentally typo all over yourself while you’re doing it.

Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.