Markdown source

Transcript: The Pipeline Is Dead - Iris ten Teije, Sky Valley Ambient Computing

Source Video

Local Cache

Transcript

As we're online today, you can't raise As we're online today, you can't raise your hands, but just nod at your screen. your hands, but just nod at your screen. your hands, but just nod at your screen. How many of you have spent a significant How many of you have spent a significant How many of you have spent a significant part of your career part of your career part of your career making making software move from one making making software move from one making making software move from one computer to another? computer to another? computer to another? CI pipelines, packagers, registries, CI pipelines, packagers, registries, CI pipelines, packagers, registries, container images, app store reviews. container images, app store reviews. container images, app store reviews. That entire stack exists to solve That entire stack exists to solve That entire stack exists to solve exactly one problem. exactly one problem. exactly one problem. Get a frozen artifact, that code, Get a frozen artifact, that code, Get a frozen artifact, that code, from machine where it was built to the from machine where it was built to the from machine where it was built to the machine where it runs, safely, machine where it runs, safely, machine where it runs, safely, reproducibly, once. reproducibly, once. reproducibly, once. And here's what I think everyone is And here's what I think everyone is And here's what I think everyone is missing. missing. missing. That entire stack is built around one That entire stack is built around one That entire stack is built around one idea that is so old that is so old that idea that is so old that is so old that idea that is so old that is so old that we stopped seeing it as a choice. we stopped seeing it as a choice. we stopped seeing it as a choice. One version of your software for One version of your software for One version of your software for everyone. everyone. everyone. We've shipped it that way for so long We've shipped it that way for so long We've shipped it that way for so long that almost nobody asks why anymore. that almost nobody asks why anymore. that almost nobody asks why anymore. I'm Iris. I'm one of the co-founders of I'm Iris. I'm one of the co-founders of I'm Iris. I'm one of the co-founders of Differ. Differ. Differ. My co-founder Noam was the first My co-founder Noam was the first My co-founder Noam was the first engineer at JFrog. He helped build the engineer at JFrog. He helped build the engineer at JFrog. He helped build the pipeline that I'm about to tell you is pipeline that I'm about to tell you is pipeline that I'm about to tell you is dying. dying. dying. I came at it from the other end. I came at it from the other end. I came at it from the other end. I spent a decade in fintech. I was early I spent a decade in fintech. I was early I spent a decade in fintech. I was early at a digital bank that we scaled and at a digital bank that we scaled and at a digital bank that we scaled and exited. exited. exited. Shipping software in the environments Shipping software in the environments Shipping software in the environments least willing to tolerate what I'm about least willing to tolerate what I'm about least willing to tolerate what I'm about to propose. So, I totally get any to propose. So, I totally get any to propose. So, I totally get any reservations and I've had them myself as reservations and I've had them myself as reservations and I've had them myself as well. well. well. However, I'm telling you it's coming However, I'm telling you it's coming However, I'm telling you it's coming anyway. And anyway. And anyway. And that's not a warning. It's the best that's not a warning. It's the best that's not a warning. It's the best thing that's happened to software in a thing that's happened to software in a thing that's happened to software in a long time. long time. long time. Every piece of distribution Every piece of distribution Every piece of distribution infrastructure that you've touched infrastructure that you've touched infrastructure that you've touched encodes the same assumption. encodes the same assumption. encodes the same assumption. Software is produced in one place. Software is produced in one place. Software is produced in one place. It runs in another. It runs in another. It runs in another. Anything in the middle, the artifact, is Anything in the middle, the artifact, is Anything in the middle, the artifact, is frozen. frozen. frozen. And that assumption was correct. For And that assumption was correct. For And that assumption was correct. For decades, it was just true. decades, it was just true. decades, it was just true. Why? Why? Why? Because producing a correct change was Because producing a correct change was Because producing a correct change was expensive. expensive. expensive. It took skilled humans hours or days. It took skilled humans hours or days. It took skilled humans hours or days. So, you did it rarely. It was a central So, you did it rarely. It was a central So, you did it rarely. It was a central event. You verified it and froze it. event. You verified it and froze it. event. You verified it and froze it. And you shipped that frozen thing to And you shipped that frozen thing to And you shipped that frozen thing to everyone. everyone. So, the one-way pipeline is not So, the one-way pipeline is not So, the one-way pipeline is not arbitrary. arbitrary. arbitrary. It's the direct consequence It's the direct consequence It's the direct consequence of production of software being of production of software being of production of software being expensive and risky. expensive and risky. expensive and risky. And And And of course, the frozen artifact has some of course, the frozen artifact has some of course, the frozen artifact has some advantages. You get advantages. You get advantages. You get reproducibility, previewability, reproducibility, previewability, reproducibility, previewability, rollback. rollback. rollback. And every guarantee that we lean on in And every guarantee that we lean on in And every guarantee that we lean on in production flows from one fact. production flows from one fact. production flows from one fact. There is one artifact and it doesn't There is one artifact and it doesn't There is one artifact and it doesn't change after we ship it. change after we ship it. change after we ship it. That's the deal. One version for That's the deal. One version for That's the deal. One version for everyone, frozen. everyone, frozen. everyone, frozen. We call it reliability. We call it reliability. We call it reliability. But the price was that the software But the price was that the software But the price was that the software couldn't really be for anyone in couldn't really be for anyone in couldn't really be for anyone in particular. particular. particular. And nobody ever made that decision. And nobody ever made that decision. And nobody ever made that decision. There was never a meeting where someone There was never a meeting where someone There was never a meeting where someone put one version for everyone versus a put one version for everyone versus a put one version for everyone versus a version for each person and picked the version for each person and picked the version for each person and picked the first one. first one. first one. And it wasn't because the second option And it wasn't because the second option And it wasn't because the second option if a version for each person was worse. if a version for each person was worse. if a version for each person was worse. It just wasn't an option. It just wasn't an option. It just wasn't an option. Giving to users different software meant Giving to users different software meant Giving to users different software meant forking the code base and hand forking the code base and hand forking the code base and hand maintaining both. maintaining both. maintaining both. A version per user at any real scale A version per user at any real scale A version per user at any real scale wasn't really a viable option. wasn't really a viable option. wasn't really a viable option. And one version for everyone never had And one version for everyone never had And one version for everyone never had to win an argument. It was just how it to win an argument. It was just how it to win an argument. It was just how it was. It was the only shape software was. It was the only shape software was. It was the only shape software could take. could take. could take. And we started treating it as a fact And we started treating it as a fact And we started treating it as a fact about software like gravity, like about software like gravity, like about software like gravity, like something that that is just true. something that that is just true. something that that is just true. But it was never really a fact about But it was never really a fact about But it was never really a fact about software. It was a fact about cost and software. It was a fact about cost and software. It was a fact about cost and budget and the economics and that cost budget and the economics and that cost budget and the economics and that cost just changed. Now, you might you might just changed. Now, you might you might just changed. Now, you might you might expect me to go into AI 10 code, but expect me to go into AI 10 code, but expect me to go into AI 10 code, but everyone has already said it and that's everyone has already said it and that's everyone has already said it and that's stable stakes and to me that's not stable stakes and to me that's not stable stakes and to me that's not really the interesting part. The really the interesting part. The really the interesting part. The interesting part is where and how cheap interesting part is where and how cheap interesting part is where and how cheap The cost of producing a correct and The cost of producing a correct and The cost of producing a correct and scope change scope change scope change is collapsing towards zero and is collapsing towards zero and is collapsing towards zero and just as importantly just as importantly just as importantly the production of the software no longer the production of the software no longer the production of the software no longer has to happen in one place up front has to happen in one place up front has to happen in one place up front before anyone runs it. before anyone runs it. before anyone runs it. Part of it can be run on the server, Part of it can be run on the server, Part of it can be run on the server, part of it on the client, part in the part of it on the client, part in the part of it on the client, part in the user's live session. user's live session. user's live session. And as each step stops being a decision And as each step stops being a decision And as each step stops being a decision you freeze at build time you freeze at build time you freeze at build time and and and and now starts becoming more of a and now starts becoming more of a and now starts becoming more of a real-time one real-time one real-time one that also means that that also means that that also means that each each each piece can be placed each each each piece can be placed each each each piece can be placed wherever it makes most sense including wherever it makes most sense including wherever it makes most sense including right in front of the user in their right in front of the user in their right in front of the user in their context. context. context. And so this whole one-way pipeline And so this whole one-way pipeline And so this whole one-way pipeline existed because making software was the existed because making software was the existed because making software was the expensive, central, and rare event and expensive, central, and rare event and expensive, central, and rare event and running it was cheap. running it was cheap. running it was cheap. So you separated development from So you separated development from So you separated development from distribution. distribution. distribution. But now as making a change becomes as But now as making a change becomes as But now as making a change becomes as cheap as running one and it can happen cheap as running one and it can happen cheap as running one and it can happen in the same place in the same place in the same place as um as um as um as where you run it, the reason to as where you run it, the reason to as where you run it, the reason to separate them separate them separate them is dissolving. So far I've talked about is dissolving. So far I've talked about is dissolving. So far I've talked about the supply side. the supply side. the supply side. Producing code has become cheap Producing code has become cheap Producing code has become cheap easy. We can now make a change on a per easy. We can now make a change on a per easy. We can now make a change on a per user basis. user basis. user basis. But I also want to address the demand But I also want to address the demand But I also want to address the demand side. side. side. People have always wanted software that People have always wanted software that People have always wanted software that fits them and we have decades of proof fits them and we have decades of proof fits them and we have decades of proof for that. for that. for that. It just wasn't really possible for most It just wasn't really possible for most It just wasn't really possible for most types of software because of cost. types of software because of cost. types of software because of cost. To start with one example, the forward To start with one example, the forward To start with one example, the forward deployed engineer. deployed engineer. deployed engineer. Enterprise software has always had a Enterprise software has always had a Enterprise software has always had a line item called professional services line item called professional services line item called professional services and a whole industry exists around that. and a whole industry exists around that. and a whole industry exists around that. If you are a big client of a company If you are a big client of a company If you are a big client of a company like Salesforce, you probably have like Salesforce, you probably have like Salesforce, you probably have consultants that are helping you consultants that are helping you consultants that are helping you implement implement implement custom setups, configurations. custom setups, configurations. custom setups, configurations. You have an engineer living in your You have an engineer living in your You have an engineer living in your Slack channel, and it's not that Slack channel, and it's not that Slack channel, and it's not that smaller customers smaller customers smaller customers can't benefit from this type of can't benefit from this type of can't benefit from this type of customization. customization. customization. It just didn't make sense financially It just didn't make sense financially It just didn't make sense financially until today. until today. until today. To give another example that might To give another example that might To give another example that might resonate with resonate with resonate with you as engineers, you as engineers, you as engineers, think about your dot files, your editor think about your dot files, your editor think about your dot files, your editor config, key bindings. config, key bindings. config, key bindings. You rebuild every tool you touch into You rebuild every tool you touch into You rebuild every tool you touch into your tool by hand on every machine. your tool by hand on every machine. your tool by hand on every machine. It's another example of a demand for It's another example of a demand for It's another example of a demand for personalized software. personalized software. personalized software. And lastly, Excel, the most successful And lastly, Excel, the most successful And lastly, Excel, the most successful business software ever created. And business software ever created. And business software ever created. And Excel isn't really Excel isn't really Excel isn't really a a a static program. static program. static program. It's millions of people that all build It's millions of people that all build It's millions of people that all build their own programs on top of it. So, that's where. Give people the power So, that's where. Give people the power to make software to make their software to make software to make their software to make software to make their software theirs, and they take it. Seen in the theirs, and they take it. Seen in the theirs, and they take it. Seen in the social feed that per user wins on wins social feed that per user wins on wins social feed that per user wins on wins from one size fits all on every metric from one size fits all on every metric from one size fits all on every metric that mattered. that mattered. that mattered. This is more of an example of content This is more of an example of content This is more of an example of content versus software. versus software. versus software. But now that we have But now that we have But now that we have uh better coding again now we have uh better coding again now we have uh better coding again now we have coding agents and better coding agents, coding agents and better coding agents, coding agents and better coding agents, we can move this also to the software we can move this also to the software we can move this also to the software layer. So, my point is none of this is layer. So, my point is none of this is layer. So, my point is none of this is really new demand. We've seen it for really new demand. We've seen it for really new demand. We've seen it for decades. There have been predecessors decades. There have been predecessors decades. There have been predecessors like feature flags, segmentation, AB like feature flags, segmentation, AB like feature flags, segmentation, AB testing. There's an enormous industry testing. There's an enormous industry testing. There's an enormous industry around uh uh around this, and around uh uh around this, and around uh uh around this, and we've been trying to make software we've been trying to make software we've been trying to make software diverge for many years. diverge for many years. diverge for many years. But But But we go forced into a specific shape, that we go forced into a specific shape, that we go forced into a specific shape, that of creating buckets and segments that of creating buckets and segments that of creating buckets and segments that you declare in advance. you declare in advance. you declare in advance. And And now, for the first time, we can make now, for the first time, we can make now, for the first time, we can make software truly adaptive. software truly adaptive. software truly adaptive. So, to get back to the title of this So, to get back to the title of this So, to get back to the title of this talk, when the agent is the runtime, talk, when the agent is the runtime, talk, when the agent is the runtime, when the thing that runs your software when the thing that runs your software when the thing that runs your software can also modify it, can also modify it, can also modify it, development and distribution stop being development and distribution stop being development and distribution stop being two phases. two phases. two phases. The boundary blurs and it's gone. The boundary blurs and it's gone. The boundary blurs and it's gone. And the shape that we bet on at Differ And the shape that we bet on at Differ And the shape that we bet on at Differ is that instead of one code base gated is that instead of one code base gated is that instead of one code base gated by flags and shipped to everyone, by flags and shipped to everyone, by flags and shipped to everyone, you deploy you deploy you deploy one canonical stem one canonical stem one canonical stem and every user runs her own divergence and every user runs her own divergence and every user runs her own divergence of it. of it. of it. Same origin, Same origin, Same origin, but individually adapted live. It's going from the least first version It's going from the least first version for everyone for everyone for everyone to the best version for anyone. Now, if you're any of such a person, you Now, if you're any of such a person, you might be a little worried. Your stomach might be a little worried. Your stomach might be a little worried. Your stomach might be turning because might be turning because might be turning because I have just deleted the frozen artifact I have just deleted the frozen artifact I have just deleted the frozen artifact and the frozen artifact was what was and the frozen artifact was what was and the frozen artifact was what was holding up the entire holding up the entire holding up the entire the the entire building. And I do we do the the entire building. And I do we do the the entire building. And I do we do get these objections, for example, from get these objections, for example, from get these objections, for example, from a call that I had with a CTO recently a call that I had with a CTO recently a call that I had with a CTO recently who is right to be skeptical. What he who is right to be skeptical. What he who is right to be skeptical. What he said roughly was, said roughly was, said roughly was, "I can already barely reason about one "I can already barely reason about one "I can already barely reason about one AI-generated code base. AI-generated code base. AI-generated code base. And you want me to run millions of And you want me to run millions of And you want me to run millions of these? You're not describing a these? You're not describing a these? You're not describing a capability. You're describing my worst capability. You're describing my worst capability. You're describing my worst problem multiplied." problem multiplied." problem multiplied." And if that's your reaction, that's not And if that's your reaction, that's not And if that's your reaction, that's not surprising. It's the right instinct, but surprising. It's the right instinct, but surprising. It's the right instinct, but perhaps aimed at the wrong target. perhaps aimed at the wrong target. perhaps aimed at the wrong target. Here's the distinction that we make. The Here's the distinction that we make. The Here's the distinction that we make. The brittleness that you're picturing is a brittleness that you're picturing is a brittleness that you're picturing is a specific type of failure mode. It's specific type of failure mode. It's specific type of failure mode. It's unmanaged divergence inside a single unmanaged divergence inside a single unmanaged divergence inside a single artifact. artifact. artifact. Thousand-line files, everything can Thousand-line files, everything can Thousand-line files, everything can touch everything else, no boundaries. touch everything else, no boundaries. touch everything else, no boundaries. And And that's not not that's not necessarily that's not not that's not necessarily that's not not that's not necessarily brittle because it's AI-generated. It's brittle because it's AI-generated. It's brittle because it's AI-generated. It's brittle because there's no structure brittle because there's no structure brittle because there's no structure separating things. separating things. separating things. In our In our vision, we've we're In our In our vision, we've we're In our In our vision, we've we're thinking about per-user divergences. And thinking about per-user divergences. And thinking about per-user divergences. And don't write that is the opposite of don't write that is the opposite of don't write that is the opposite of that. You've got a stem plus that. You've got a stem plus that. You've got a stem plus divergences. The divergences are divergences. The divergences are divergences. The divergences are bounded, isolated, and individually bounded, isolated, and individually bounded, isolated, and individually reversible. reversible. reversible. A bad variant can silently corrupt the A bad variant can silently corrupt the A bad variant can silently corrupt the stem or reach another user. stem or reach another user. stem or reach another user. Which means that the blast radius Which means that the blast radius Which means that the blast radius of a changes in the system is one of a changes in the system is one of a changes in the system is one context. And context. And context. And any single divergence can roll back life any single divergence can roll back life any single divergence can roll back life with no deploy. So, the answer to the with no deploy. So, the answer to the with no deploy. So, the answer to the previous objection isn't trust us, AI is previous objection isn't trust us, AI is previous objection isn't trust us, AI is good at coordination or coding. good at coordination or coding. good at coordination or coding. The The The honest answer is you're brittle because honest answer is you're brittle because honest answer is you're brittle because there's a tangled artifact with no there's a tangled artifact with no there's a tangled artifact with no boundaries. boundaries. boundaries. In our case, we don't ship you a In our case, we don't ship you a In our case, we don't ship you a thousand single artifacts. We ship one thousand single artifacts. We ship one thousand single artifacts. We ship one stem and bounded divergences. Each stem and bounded divergences. Each stem and bounded divergences. Each isolated, each reversible. isolated, each reversible. isolated, each reversible. The thing that you're afraid of is the The thing that you're afraid of is the The thing that you're afraid of is the thing that this architecture exists to thing that this architecture exists to thing that this architecture exists to prevent. prevent. prevent. And as a developer, you can also set And as a developer, you can also set And as a developer, you can also set controls and boundaries. What can and controls and boundaries. What can and controls and boundaries. What can and cannot be adapted. To give a small cannot be adapted. To give a small cannot be adapted. To give a small example, um we can have a scenario where example, um we can have a scenario where example, um we can have a scenario where we've got a form and the form can be we've got a form and the form can be we've got a form and the form can be adapted in order to adapted in order to adapted in order to improve conversion rate, for example. improve conversion rate, for example. improve conversion rate, for example. However, as a developer, you can always However, as a developer, you can always However, as a developer, you can always indicate that specific fields can never indicate that specific fields can never indicate that specific fields can never be dropped or be dropped or be dropped or parts of your app, like off or payments, parts of your app, like off or payments, parts of your app, like off or payments, should always be out of um should always be out of um should always be out of um uh uh uh uh uh uh should always be off limits for any sort should always be off limits for any sort should always be off limits for any sort of adaptation. of adaptation. of adaptation. So, give another example to make So, give another example to make So, give another example to make adaptive software a bit more concrete. adaptive software a bit more concrete. adaptive software a bit more concrete. Uh think of, for example, um a CRM. Uh think of, for example, um a CRM. Uh think of, for example, um a CRM. In this case, uh we've got an investor In this case, uh we've got an investor In this case, uh we've got an investor who's using a CRM. Whereas, the CRM was who's using a CRM. Whereas, the CRM was who's using a CRM. Whereas, the CRM was mostly built with a salesperson in mind. mostly built with a salesperson in mind. mostly built with a salesperson in mind. And as an investor, And as an investor, And as an investor, you might use it slightly differently. you might use it slightly differently. you might use it slightly differently. So, this investor, she So, this investor, she So, this investor, she often logs founder intros, and she's often logs founder intros, and she's often logs founder intros, and she's always logging who intro'd her to which always logging who intro'd her to which always logging who intro'd her to which deal. deal. deal. So, the system observes that and creates So, the system observes that and creates So, the system observes that and creates a a um create creates a um create creates a um create creates a intro path. intro path. intro path. The system can also observe that she's The system can also observe that she's The system can also observe that she's always skipping specific fields. She always skipping specific fields. She always skipping specific fields. She never fills them out. So, over time, the never fills them out. So, over time, the never fills them out. So, over time, the system learns system learns system learns and and doesn't surface those fields, but doesn't surface those fields, but doesn't surface those fields, but instead it surfaces fields that she that instead it surfaces fields that she that instead it surfaces fields that she that she cares about more. Another example can be the fact that she Another example can be the fact that she is always um always checking specific is always um always checking specific is always um always checking specific types of deals or founders, and it types of deals or founders, and it types of deals or founders, and it doesn't exactly follow doesn't exactly follow doesn't exactly follow the prioritization that the system sets the prioritization that the system sets the prioritization that the system sets by default. So, again, can can we learn by default. So, again, can can we learn by default. So, again, can can we learn and make that smarter so that the and make that smarter so that the and make that smarter so that the information that the user cares about is information that the user cares about is information that the user cares about is surfaced first. surfaced first. surfaced first. Not only can a system observe, but there Not only can a system observe, but there Not only can a system observe, but there is also that the user can proactively is also that the user can proactively is also that the user can proactively request changes, and as long as these request changes, and as long as these request changes, and as long as these changes are within changes are within changes are within the boundaries that the developer sets, the boundaries that the developer sets, the boundaries that the developer sets, and as long as they are within the and as long as they are within the and as long as they are within the spirit of the software, within the spirit of the software, within the spirit of the software, within the purpose of what the software was purpose of what the software was purpose of what the software was originally made for, originally made for, originally made for, it can be implemented without having to it can be implemented without having to it can be implemented without having to go back to the developer. go back to the developer. go back to the developer. So, you can imagine for So, you can imagine for So, you can imagine for a for a horizontal SaaS like a CRM, a for a horizontal SaaS like a CRM, a for a horizontal SaaS like a CRM, you can address you can address you can address a much wider number of customer personas a much wider number of customer personas a much wider number of customer personas without increasing your R&D spend. without increasing your R&D spend. without increasing your R&D spend. Now, of course, there is there are many Now, of course, there is there are many Now, of course, there is there are many hard parts when it comes to bringing hard parts when it comes to bringing hard parts when it comes to bringing this to life. this to life. this to life. And I'm going to discuss a couple of And I'm going to discuss a couple of And I'm going to discuss a couple of hard challenges and problem and problems hard challenges and problem and problems hard challenges and problem and problems that we're uh that we're working on as that we're uh that we're working on as that we're uh that we're working on as we're making adaptive software real at we're making adaptive software real at we're making adaptive software real at scale. scale. scale. We haven't solved all of them, but we We haven't solved all of them, but we We haven't solved all of them, but we have a we have a point of view on each have a we have a point of view on each have a we have a point of view on each of them, and it's that it's exactly what of them, and it's that it's exactly what of them, and it's that it's exactly what we're working on day in, day out. we're working on day in, day out. we're working on day in, day out. So, what changes when there's no single So, what changes when there's no single So, what changes when there's no single artifact? artifact? artifact? Firstly, Firstly, Firstly, the source of truth. the source of truth. the source of truth. So, what when there's no single So, what when there's no single So, what when there's no single artifact, you can wonder what is the artifact, you can wonder what is the artifact, you can wonder what is the software? software? software? In our case, we consider the software to In our case, we consider the software to In our case, we consider the software to be the stem plus the plus all the mutual be the stem plus the plus all the mutual be the stem plus the plus all the mutual immutable divergences. immutable divergences. immutable divergences. But of course, that creates a lineage But of course, that creates a lineage But of course, that creates a lineage problem. problem. problem. What is this user running and why? What is this user running and why? What is this user running and why? That now becomes more of a graph query That now becomes more of a graph query That now becomes more of a graph query versus a version number. versus a version number. versus a version number. A bug report describes a program that A bug report describes a program that A bug report describes a program that exists exists exists for that specific user. for that specific user. for that specific user. How do you debug or How do you debug or How do you debug or inspect that? The answer is every inspect that? The answer is every inspect that? The answer is every divergence is immutable, inspectable, divergence is immutable, inspectable, divergence is immutable, inspectable, attributable. attributable. attributable. And And we also need to trace any version back we also need to trace any version back we also need to trace any version back to to to the signal that to this specific the signal that to this specific the signal that to this specific recommendation and this exact recommendation and this exact recommendation and this exact adaptation. adaptation. adaptation. That's one of the parts that we're That's one of the parts that we're That's one of the parts that we're working on. working on. working on. Second is correctness. Like how do you Second is correctness. Like how do you Second is correctness. Like how do you test that's the um how do you test that test that's the um how do you test that test that's the um how do you test that the code change that you've just the code change that you've just the code change that you've just implemented implemented implemented works, that UI is correct, works, that UI is correct, works, that UI is correct, and yeah, testing at and yeah, testing at and yeah, testing at a much larger scale of users means that a much larger scale of users means that a much larger scale of users means that you need to reason about the stem and you need to reason about the stem and you need to reason about the stem and also every possible divergence of it. And then there's desirability because And then there's desirability because perhaps you made a code change, it's perhaps you made a code change, it's perhaps you made a code change, it's perfectly correct and working, perfectly correct and working, perfectly correct and working, but you also need to know that whether but you also need to know that whether but you also need to know that whether whether this was desirable. whether this was desirable. whether this was desirable. Was it actually a good change? Was it actually a good change? Was it actually a good change? Because anyone can Because anyone can Because anyone can make a code change now, but the hard make a code change now, but the hard make a code change now, but the hard part is knowing whether you actually part is knowing whether you actually part is knowing whether you actually found found found an improvement, an uplift. And yeah, that that is something that is And yeah, that that is something that is extremely important to to keep track of extremely important to to keep track of extremely important to to keep track of and to measure and to consider what the and to measure and to consider what the and to measure and to consider what the goals for the company are. And this is goals for the company are. And this is goals for the company are. And this is not going to be the same for every not going to be the same for every not going to be the same for every single piece of software. In some cases, single piece of software. In some cases, single piece of software. In some cases, it might be retention or less churn it might be retention or less churn it might be retention or less churn or lowering the number of support or lowering the number of support or lowering the number of support tickets. tickets. tickets. So, it's extremely important to keep So, it's extremely important to keep So, it's extremely important to keep track of track of track of what are what are the goals that we're what are what are the goals that we're what are what are the goals that we're chasing with adaptive software and chasing with adaptive software and chasing with adaptive software and do the adaptations do the adaptations do the adaptations reach to reach to reach to improve improve improve the metrics that improve improve improve the metrics that improve improve improve the metrics that matter. matter. matter. Next is autonomy versus control. And Next is autonomy versus control. And Next is autonomy versus control. And the conservative answer here would be the conservative answer here would be the conservative answer here would be start with just recommendations and start with just recommendations and start with just recommendations and don't make any autonomous changes. And don't make any autonomous changes. And don't make any autonomous changes. And that's not a wrong strategy, that's not a wrong strategy, that's not a wrong strategy, but in our case, it's not really our but in our case, it's not really our but in our case, it's not really our vision. The vision is a system that vision. The vision is a system that vision. The vision is a system that understands the user well enough to act understands the user well enough to act understands the user well enough to act without asking the developer for without asking the developer for without asking the developer for permission every time first. permission every time first. permission every time first. So, for us, the challenge isn't building So, for us, the challenge isn't building So, for us, the challenge isn't building more control. more control. more control. It's winning enough trust that you don't It's winning enough trust that you don't It's winning enough trust that you don't have to. And it's a hard problem, but have to. And it's a hard problem, but have to. And it's a hard problem, but also also also also very interesting one. And you know, also very interesting one. And you know, also very interesting one. And you know, how do you make a system how do you make a system how do you make a system good enough, good enough, good enough, legible enough, reliable enough that legible enough, reliable enough that legible enough, reliable enough that humans in the loop choose to humans in the loop choose to humans in the loop choose to step back. step back. step back. And And that's certainly what we are building that's certainly what we are building that's certainly what we are building towards. And lastly, coordination. Everyone on And lastly, coordination. Everyone on their own version, how do you their own version, how do you their own version, how do you push new updates? How do how do changes push new updates? How do how do changes push new updates? How do how do changes propagate propagate propagate to like a million different versions? to like a million different versions? to like a million different versions? And this is one of the And this is one of the And this is one of the challenges that we've been thinking challenges that we've been thinking challenges that we've been thinking hardest about. hardest about. hardest about. Yeah, and the answer that we keep coming Yeah, and the answer that we keep coming Yeah, and the answer that we keep coming back to is back to is back to is don't merge code, merge intent, merge don't merge code, merge intent, merge don't merge code, merge intent, merge outcome. Which means not everyone has to outcome. Which means not everyone has to outcome. Which means not everyone has to run the same commit or this exact same run the same commit or this exact same run the same commit or this exact same piece of code, but everyone converges on piece of code, but everyone converges on piece of code, but everyone converges on the same goal through their own path. the same goal through their own path. the same goal through their own path. And if it wasn't already clear, the And if it wasn't already clear, the And if it wasn't already clear, the challenges that I just challenges that I just challenges that I just addressed are the hard challenges. addressed are the hard challenges. addressed are the hard challenges. Generation has become easy, and I would Generation has become easy, and I would Generation has become easy, and I would say that's actually the easy 80%. say that's actually the easy 80%. say that's actually the easy 80%. Calling a model to Calling a model to Calling a model to write some code is something that write some code is something that write some code is something that everyone can do. everyone can do. everyone can do. The other part, observability, The other part, observability, The other part, observability, validation, coordination, that is the validation, coordination, that is the validation, coordination, that is the entire business. Anyone can call an LM, but the substrate Anyone can call an LM, but the substrate is stem plus diver plus divergences, is stem plus diver plus divergences, is stem plus diver plus divergences, provenance, validation, provenance, validation, provenance, validation, that is really the hard part, and that is really the hard part, and that is really the hard part, and something that we're working on every something that we're working on every something that we're working on every single day. To close off with, when I started out in To close off with, when I started out in fintech, a bank with no branches sounded fintech, a bank with no branches sounded fintech, a bank with no branches sounded reckless. A decade later, A decade later, the the branch is the weird part. the the branch is the weird part. the the branch is the weird part. My co-founder, Noam, used to fight with My co-founder, Noam, used to fight with My co-founder, Noam, used to fight with engineers about why you need builds and engineers about why you need builds and engineers about why you need builds and CI. We've seen this We've seen these shifts We've seen this We've seen these shifts before, and adaptive software is the before, and adaptive software is the before, and adaptive software is the same type of shift. same type of shift. same type of shift. It doesn't feel obvious It doesn't feel obvious It doesn't feel obvious until it does. until it does. until it does. To go back to the start of the talk, the To go back to the start of the talk, the To go back to the start of the talk, the pipeline didn't fail because it didn't pipeline didn't fail because it didn't pipeline didn't fail because it didn't work anymore, but work anymore, but work anymore, but the constraint it was built for went the constraint it was built for went the constraint it was built for went away. away. away. The assumption underlying of that The assumption underlying of that The assumption underlying of that software is expensive, so we need to software is expensive, so we need to software is expensive, so we need to freeze it and ship it once. freeze it and ship it once. freeze it and ship it once. That assumption stopped being true. And when making software get as cheap as And when making software get as cheap as running it, running it, running it, the line between distribution and the line between distribution and the line between distribution and development isn't a line anymore. We spent 20 years getting good at We spent 20 years getting good at shipping one version for everyone. shipping one version for everyone. shipping one version for everyone. The next 20 years are about shipping the The next 20 years are about shipping the The next 20 years are about shipping the right version to anyone right version to anyone right version to anyone with the isolation and provenance that with the isolation and provenance that with the isolation and provenance that makes it safe instead of terrifying. makes it safe instead of terrifying. makes it safe instead of terrifying. I'm Iris, this is different and that's I'm Iris, this is different and that's I'm Iris, this is different and that's what we're building. Thank you for what we're building. Thank you for what we're building. Thank you for watching.