lystn

About Corey QuinnOver the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group, the author of the weekly Last Week in AWS newsletter, and the host of two podcasts: Screaming in the Cloud and, you guessed it, AWS Morning Brief, which you’re about to listen to.Links CHAOSSEARCH @QuinnyPig TranscriptCorey: Welcome to AWS Morning Brief: Whiteboard Confessional. I’m Cloud Economist Corey Quinn. This weekly show exposes the semi-polite lie that is whiteboard architecture diagrams. You see, a child can draw a whiteboard architecture, but the real world is a mess. We discuss the hilariously bad decisions that make it into shipping products, the unfortunate hacks the real-world forces us to build, and that the best to call your staging environment is “theory”. Because invariably whatever you’ve built works in the theory, but not in production. Let’s get to it.This episode is brought to you by Trend Micro Cloud One™. A security services platform for organizations building in the Cloud. I know you're thinking that that's a mouthful because it is, but what's easier to say? “I'm glad we have Trend Micro Cloud One™, a security services platform for organizations building in the Cloud,” or, “Hey, bad news. It's going to be a few more weeks. I kind of forgot about that security thing.” I thought so. Trend Micro Cloud One™ is an automated, flexible all-in-one solution that protects your workflows and containers with cloud-native security. Identify and resolve security issues earlier in the pipeline, and access your cloud environments sooner, with full visibility, so you can get back to what you do best, which is generally building great applications. Discover Trend Micro Cloud One™ a security services platform for organizations building in the Cloud. Whew. At trendmicro.com/screaming.Welcome to the AWS Morning Brief: Whiteboard Confessional. I am Cloud Economist Corey Quinn, which means that I fix the horrifying AWS bill both by making it more understandable, as well as lower. On today's episode of the AWS Morning Brief: Whiteboard Confessional, I looked around the whiteboards in the backgrounds of the Zoom calls that I'm having with basically everyone these days because going to the office during a pandemic is and remains a deadly risk, and it's amazing how much you can learn about people's companies by what they leave on the whiteboards. Whether you happen to be visiting their office or left inattentively in the background because they forget that they didn't turn on the Zoom background. One of the most disturbing things that we'll see on any whiteboard in any company that you work at, ever, is an org chart. And what makes it disturbing, first off, is that when you see an org chart, that means that generally, someone is considering reorganizing, which is a polite framing of shuffling the deck chairs on the Titanic. It ties into one of the great corporate delusions that somehow you're going to start immediately making good decisions, and all of the previous poor decision making you've made is going to fall away like dew in the new morning. And the reason that that's the case is that everyone tends to be an optimist when looking forward because otherwise, we'd wake up crying and never go to work.Have you ever noticed that you can take a look at an org chart or an architecture diagram and remove all of the labels, and you've accidentally built the exact same thing just with different services rather than teams? Well, I'm certainly not the first person to make this observation. What I'm talking about is known as Conway's Law. Conway's Law is named after computer programmer Melvin Conway, who in 1967 introduced a phenomenal idea, for the time, that we still haven’t escaped from, specifically, any organization that designs a system defined broadly will produce a design whose structure is a copy of the organization's communication structure. Effectively what that means is you ship your culture as well as your org chart, and if we take a look at how different seminal software products of the ages have come out. It's pretty clear that there is at least some passing resemblance to reality. You take a look at Amazon; they're effectively an entire microservices company. They have so many different small two pizza teams building things, and sure enough, you take a look at AWS, for example, they have 200 some-odd services that are ideally production-grade, but again, it's a mixed bag. Because again, not every team is identical, and not every team has the same resources. So, as a result, though, you take a look at that, that is the good part of their culture. Well, what's bad? Well, anything that involves all of those teams to coordinate at once on something. Think of the billing system. Think of the AWS web console. You start to see where these things break down. These are the seams between services that AWS tends to miss out on. If you take a look at Google, for example, the entire model there, to my understanding, is you want to get promoted and you want to get a raise, and that all comes down to certain metrics that don't necessarily align with what people want to be working on. So, you see people instead focusing on things that are they're incentivized to do to go up in the org, and not maintain things that they built last year, which is why I suspect, at least, that we see this neverending wave of Google product deprecations. And the list goes on, and I'm certainly not a corporate taxonomist; I'm a cloud economist, so I'm not going to go into too much depth on what that looks like in different places, but it does become telling. Let's get into that a bit more. But first:This episode is sponsored in part by ChaosSearch. Now their name isn’t in all caps, so they’re definitely worth talking to. What is ChaosSearch? A scalable log analysis service that lets you add new workloads in minutes, not days or weeks. Click. Boom. Done. ChaosSearch is for you if you’re trying to get a handle on processing multiple terabytes, or more, of log and event data per day, at a disruptive price. One more thing, for those of you that have been down this path of disappointment before, ChaosSearch is a fully managed solution that isn’t playing marketing games when they say “fully managed.” The data lives within your S3 buckets, and that’s really all you have to care about. No managing of servers, but also no data movement. Check them out at chaossearch.io and tell them Corey sent you. Watch for the wince when you say my name. That’s chaossearch.io.Now, one thing that I sort of pride myself on being—because I have to be—is data center archaeologist. Frankly, these days it’s cloud archaeology. But when I go into a new client environment, I ask them to show me their architecture diagrams, and that always goes the same way. First, people apologize because the architecture diagram is out of date. Spoiler: everyone's architecture diagram is out of date. That is the nature of the universe. Smile, nod, and accept it. But it shows a lot about how they think about things; about how different components communicate with each other. And it really tends to lead to an interesting analysis in its own right because what you're really looking at is effectively a second-order effect of what their company culture is. This is why digital transformations are so freaking tricky. You can call them cloud migrations, you can call them digital transformations, you can call them anything as long as companies will sign the $20 million consulting project SOP. The reason that they're tricky is that you're trying to change the culture of the company. It's my belief, and I know that there's an entire industry who's going to argue with me on this point, that you cannot change the culture of a company externally. It has to be something that you do organically, and it has to come from the top. Something I've learned starting my own company is coming from when it was just me where the culture was a one-to-one match with my personality. Now we're 10 people; it absolutely has changed, and, “We're not going to have a formal organizational structure, just everyone's going to do the right thing.” Yeah, that breaks down right around the time there’s a second person involved. Coordination among organizations becomes a serious challenge in its own right. There's always going to be some efficiency loss due to communication friction as companies get larger and larger. At some point, this enters the many millions of dollars space and companies sort of take on their own inertia. Earlier this week, IBM announced their earnings, and they went up massively as far as stock value in the after-hours because they didn't lose quite as much money as analysts thought they were going to lose. I'm being slightly sarcastic; they did make a profit but it was declining quarter-over-quarter and year-over-year. And this is the problem. There's a sort of its own inertia. One of my jokes about IBM has been that they're the kind of company that could go out of business and five years later, some of their divisions might hear about that for the first time. Now, if you're big enough to have an environment where a digital transformation or a cloud migration is tricky—and spoiler: unless you're that one person startup, you're going to have some problems there—you're going to encounter an environment where shifting the culture is required in order to affect meaningful and lasting change in how your product or service gets delivered and built. Cultural change is hard because you're asking people to do things differently, and we are all creatures of habit. It's one thing to say, “Okay, we're going to use a different tool that everyone has to use.” You can get people to go along with that, grumbling. But you're going to communicate with one another differently? We're going to change our processes? And carrying those legacy processes from the pre-digital transformation era—we'll call it the analog years—forward into a scenario where you have to have a human being approve every step of the build release pipeline. Yeah, it turns out that it's hard to find someone to work in your environment whose name is Jenkins, so at some point, you have to be willing to let go. Well, some people also further identify the thing that they do as being core and central to their entire identity. The thing that they do defines their own self-perception. And that means that when someone comes in with a new way of doing things, that means that thing that you used to do all the time can now be automated away, or the scope of it can be dramatically reduced, what you're fundamentally trying to sell someone on is their own irrelevance. At least that's the perception. How do you manage that? Well, that's where the art of digital transformation comes in. I don't have any magic answers on this because I don't sell digital transformations. I sell cloud cost optimization and understanding. Now, this stuff is linked in many cases, but it's certainly not a one-to-one match. And success in shifting the culture is always going to be dependent, on some level, on who is buying into the vision, who has to shift and how, and ultimately the outcome hereafter. Entirely too often, the reason people go through digital transformations is that they hired a new CIO, and that person needs to have something to put on their resume when they get fired in 18 months and look for their next CIO job. At that point, I think I've gotten cynical enough that it's time to call this an episode.I’m Cloud Economist Corey Quinn. This is the AWS Morning Brief: Whiteboard Confessional. And if you've liked this podcast, please leave a five-star review on Apple Podcasts, whereas if you've hated this podcast, please leave a five-star review on Apple Podcasts along with a copy of your org chart slash microservices architecture diagram.Thank you for joining us on Whiteboard Confessional. If you have terrifying ideas, please reach out to me on twitter at @quinnypig and let me know what I should talk about next time.Announcer: This has been a HumblePod production. Stay humble.