Composable Architecture Liberation with Brian Ballard, Part 1
Transcript
Justin Burrows: Hi.I'm Justin Burrows. This is Commerce Chats at High Velocity. Welcome to Commerce Chats. Thank you for joining me, Brian.
Brian Ballard: Yeah, no problem anytime.
Justin Burrows: I just wanted to talk a little bit because I think it's interesting that we are alike in a certain way and that our path to what we did went from kind of more of a consulting role where we would be SI’s, System Integrators. That is people who come in and help a company perform a discreet project. Our part was to help them either re platform their e-commerce website or perhaps create a new one, a Greenfield one, which those are fun too, because you're kind of making it from whole cloth, so to speak. But we have both gone from… you went to engaging with a particular client, Serena and Lily. Is it what you call them like home furnishings? They have very beautiful pillows. What is Serena and Lily?
Brian Ballard: I mean, furnishings is a great way to put it. I'm not on the marketing side. I don't know the marketing spiel. But if you've ever been to one they have about 20 design stores all over the US and they're gorgeous, right?
Justin Burrows: It's they have a very specific design aesthetics.
Brian Ballard: They're very kind of California coastal. We're headquartered in Sausalito, Sausalito, CA and I don't know if there's really any other competitor that kind of fills that niche.
Justin Burrows: It's really, I think a gorgeous aesthetic.
Brian Ballard: So it's everything from furniture, sofas, chairs, which can all be custom upholstered, a couple 100 fabrics to choose from, right down to pillows and pillow covers.
And, and we really furnish any room in the house, you know, bedrooms, kitchens, bathrooms…really anything you would ever want to stick in a home, we offer that but within that specific design aesthetic.
Justin Burrows: So it's interesting because I went from the SI world to the systems integrator world to product. I'm working on a for a company that's making a product for e-commerce. You went to actually working directly with a company.
Brian Ballard: I think one of the things that was challenging for me as in the SI side is that your relationships were kind of boxed in. It's very, very bounded. And some of those relationships can last a long time. I've had within my former capacity as a consultant or a subcontractor or you know, whatever that capacity was as an architect in, in the SI world. Some of those relationships have lasted years and I've worked on projects, walked away, been re-engaged later on because either I was requested or because it seemed like the best fit and I already knew that technology.
So there's there's an investment there that's already been made and as long as a particular client is willing to continue that relationship, then you keep coming back to the same clients over and over and over again. But and now I'm getting a call. Sorry, let me that's OK, turn this off.
Justin Burrows: We can edit anything or I might just leave it in for flavor.
Brian Ballard: So I think there is a mindset that goes into being an SI versus actually working within a company that at least for me there's a sense of continuity that exists working within a company that doesn't exist as an SI. And, and yes, of course, even as an SI, you want to leave things better than, than they were when you started. But in the case of Serena and Lily, the technology challenges that I'm working with today are, are so much fun to work with. I've kind of embraced the idea of really taking ownership of a platform and, and seeing it through over the long term. So I enjoy working with the people. Obviously people, the people you work with are a big part of that too. I probably would not have considered joining if I didn't really enjoy working with the people that I work with here. I love that the role that I play as a cross functional role. So I get to advocate not just for the back office technology teams, but also for the front of the house to the people that work in the design stores, the people that work in marketing. And I enjoy the the cross functional aspect. I'm not being thrown in to a project as, yeah, with a very narrow scope of build this thing, right? I get to advocate for the people that are using these tools every single day to get their job done, which I enjoy, I much prefer that. And I enjoy the process of building relationships, long term relationships with the people that I work with day in and day out.
Justin Burrows: I find that's interesting because when you come into a project as a consultant, you're like, this is your scope. This is what you're trying to do and you're what you're being paid to do.
Brian Ballard: This is, this is what the client is, you know, because it's, it's TNF, right? So you're, you're, you're being paid to do a job. And if you don't, you don't really feel the freedom to go beyond that that scope because it's there's a direct money for time exchange happening.
Justin Burrows: Yes, and I think that's kind of like that task-oriented element is it's ironic because in our lives as consultants, we see how different companies operate. You go in and you see that it's a little bit like Chuck Cop said, like each family is unhappy in its own way. Whereas like they're all kind of like families. This one has one magic worker who does all the magic, which is so dangerous in that it can cause such weird, weird things. But you get a kind of a feel for how like these different companies operate. And also sometimes there's the thing where you're brought in as a remedial, like you're coming in to fix something or something is caught on fire. You know, which you already are coming into kind of like an emotionally pitched thing. And I think what's interesting about technology is I find it inherently dramatic. I find it inherently emotional in that if you're going to succeed, you kind of have to understand how everybody's feeling. It's not just a straight technical thing. Maybe that's because I'm more on kind of like the requirements side and kind of the business analysis, but I know as an architect, I find where your magic is, is that you can kind of see both worlds, you're Yin and Yang together when you come back. Well, and I like to be very hands on as well.
Brian Ballard: Because I'm now with an organization, I, I get to, well, I've earned the right, I guess too got to decide how I want to contribute. I'm definitely not a hands off architect. I like to be in the weeds actually building the thing because, I mean, how can you ever, how can you ever, you know, design a system without actually understanding how, how it works, right? I mean, it's pretty obvious, but, things are changing all the time. It is, you know, the usual rigmarole of technology. You've got to be in there in the weeds seeing how things work and putting the pieces together. And then you can kind of step back and go, OK, well, now I understand how this works. Here's how to best apply it to the problem that we're trying to solve. And so that's another one of the many reasons why I like being within an organization 'cause I get to learn new stuff. Nobody wants to pay you to learn new things as a subcontractor, a consultant with an SI, nobody wants to pay for that right. To a certain extent, I mean, to, to some degree you, you know, if you're especially if you're doing a RE platform to a brand new tech stack and, and it's, and it's very fresh and shiny and kind of unproven. You do need to rely on people who have experience ramping up on, on new technology very quickly. And there's a skill there…and being able to look at something and really quickly understand how it's meant to be applied and how it's supposed to work well.
Justin Burrows: And what a lovely segue, because that skill you're talking about of how to ramp things up. That's not necessarily if you have companies with in house staff, they're really good at running the business. They're really good at the maintenance part of it. But so time was back in the old days when Stegosaurs were walking around, you had a single, you had like an e-commerce app like that.
You had it, you downloaded it was on, on premises like you, you had it and like that was it. And then we went from that to kind of like a hosted thing where it would be up on the cloud, but it's still a piece of software that you're like logging into or whatever. And then we started to just like kind of decompose that platform and so now you have little pieces here and there and everything has kind of like a separation of concerns and it's becoming more and more kind of granular in that that architecture culturally was really difficult for a lot of people when they moved into what they ended up calling like headless or composable. I know that you, you, you had the benefit of having the skills, but I mean, like, what are the, because if you do that wrong, not only is it more expensive, it's slower and you don't know why it's slow. You know, and there's no vendor that, you know, everyone says you need somebody to kick, you know, which I don't particularly like having been kicked, being the person who was kicked. But how, how do you approach that to make sure that, like you said, like it's unproven, it's new technology. I mean, it's not unproven, but it's unproven to you, to your particular client.
Brian Ballard: I've been in, in this modality for the last 2 1/2 years now. So the transition is fresh in my mind, right? So I really had to throw 25 years’ worth of learning, not entirely out the window, but in order to really shift my thinking to this new way of doing things, I, I kind of had to throw all that away, right? Because it really is very different. I mean, at the end of the day, it's still just software, right?
But, but there really is a, a kind of a perspective you have. So I actually took a few notes here that it's not about customization anymore. It's about building contracts across your separations, across these concerns, across systemic boundaries, right? It's not about, you know, I'm going to think about the years and years I spent customizing monoliths to get them to do what you want.
And, you know, decompiling the software and injecting new implementations in an effort to make it do what you want to do. Because that's the thing that you bought, right? So you need to make it work, right. So the thinking is, if I'm going to get something to work, it needs to work inside of this piece of software. And the composable style of architecture really liberates you from that, right? I've now licensed to use these 3 or 4 services, right? And we need to make them all work together. Well, I have choices, right? I have options, right? I need to add coupons toto our e-commerce platform. Well, which of these services that we've licensed to use actually provide that kind of functionality? And let's take a look at it, look at our options. And maybe we just built something new or something that we actually did initially, but that's composable gives you the freedom to kind of plug in your own thing and then decide later how you want to approach things. As opposed to monoliths where you're like, well we want coupons in our ecommerce offering, What is our preferred, what do our monoliths provide? And does it do what we need it to do? And if it doesn't, well, then we, we extend it to make it do what we want to do. We don't build something new. You're really locked in, right, because you're extending existing functionality in a platform that you're now paying hundreds of hundreds of thousands of dollars every year for licensing and hope you know, and beyond that, you have millions for hosting. And so now you're stuck. Right now you're really, really stuck, whereas I just love how composable kind of liberates you from from that vendor lock, right? And that's obvious. That's one of the hallmarks of composable versus anything that came before it is, is the ability to completely unlock yourself from a vendor.
You really are API first. You're thinking about what are the API contracts across systemic boundaries that I need to model in order to build our storefront, right? Irrespective of, agnostic of any of the vendors that we've selected to provide, to plug into the platform to drive that functionality cause any of that could be replaced at any time. And in fact, that's one of the things, your organization is thinking about is how do we build a storefront that lets you kind of plug in and sorry, I'm kind of rambling
Justin Burrows: I'm taking it in. I this is like, yes, go, go, go, go, go, go, Gimme, Gimme!
Brian Ballard: But it let me step back here that it's the contracts that that drive composable commerce. It's not the vendor selection anymore. The vendors serve. It's not about the vendors anymore.
An, an order is an order is an order, right? An order is just a container that contains a basket of items and prices and promotions and shipping options. And there's nothing about a particular vendor that makes an order in order, right? It's just a model object that contains the things that I need to fulfill as a retailer, right? And I can model that anywhere. I don't need to, I don't even need to use an e-commerce platform, right? I can model that in a JSon object that gets persisted to a cloud, you know, storage platform, right? You know, so it doesn't need to be really anything, right.
So you're really liberated from the thinking of I need an e-commerce partner, right? Or I need an e-commerce platform. Well, you really don't need one. You could build this yourself. You just need to be able to define what does an order look like? So let's take Serena and Lily, for example. You know, furniture is a notoriously difficult.
Justin Burrows: I've worked with a few furniture companies actually, and yes, like these are they're big, they're hard to visualize.
Brian Ballard: And it's just that near infinite customizability, right, You're not selling shirts. They come in small, medium and large and then and a dozen colors, right. You're, you are offering a sofa that comes in 200 fabrics with a wide variety of, of bolt-on options, you know, legs that are in one of a half dozen different wood finishes. So there's a huge element of customizability around the SKUs that exist in your catalogue and you can't model every single one of those. As you know, you're literally going to have thousands of SKUs in your catalogue for one product and most commerce offerings don't really know how to deal with that. But if you can model what a customized shoe looks like as a model object, well, now it's just a Json payload that you're moving around the world and you just need to be able to develop a pricing model to implement a middleware that models your pricing, the company's pricing model in just a little hunk of middleware that, that lives somewhere in the cloud that can price that thing. But as long as you've defined a contract that enables you to represent what that thing looks like, then you can throw it into an order, you can pass it to a pricing engine.
You can return it to your front end as an entity within a cart that, that the front end can consume and render in a way that's meaningful to the customer. You're not tied to what kind of catalog functionality does our e-commerce vendor support, right? And can we make it work with this? You don't have to think like that anymore, right? You could actually sign on with a commerce vendor that has that provides some very simple data modeling capability and just where you just store the customization options as metadata on the product and then let middleware do the work, right? You're not requiring the commerce vendor to do all that for you, to provide all that capability for you, because you start thinking, well, jeez, that seems like a lot of work right now.
I have to write some middleware to make this all come together.
Even in the Neolithic days of, of monoliths, you were still doing a tremendous amount of, of custom engineering to make that supposedly out-of-the-box platform to get it to do what you needed to do. You're still putting a lot of time and effort in the into, you know, shaping it to, to make it do what you want. So you're not really I think we all kind of came to the realization that why are we beating our heads against a wall trying to, you know, we're, we're still doing a lot of custom development anyways. So let's just take these commerce platforms, break them down into, into their essential elements and then make them cheaper and make them simpler and do the same amount of engineering to the that you used to do back in the old, you know, back in the the Neolithic days to bolt those things together.
And yeah, anyways, I'm kind of going on
Justin Burrows: but no, no, no, you are actually just spontaneously describing the challenges of going into headless. And I definitely have had this experience. I remember when I first started engaging with kind of headless type or composable platforms, cuz you could still build headless on a monolithic back end. Yeah. So it really is composable. But I was like, I was doing a task like I'm more like kind of the, you know, I would help to train cuz I would train people like that would be part of it. It's like this is the monolith. This is how you do the monolith. And the expectations for training are much lower now. But to be fair, there are a lot of tasks out there that are just much easier. It's like, wow, you know, like making this change and having it display on the site. It's kind of like the technology got out of its own way.
Brian Ballard: In some respects, yeah. And yeah, in some respects, composable makes front of the house's lives a little more difficult in some ways, because now they're doing their work across potentially three or four different tool sets, right? So I need to decorate my catalog, right? Well, now I need to go to this tool, I need to write some content. Well, now I go to a CMS and now I need to bolt on some sale pricing. Well, now I need to go to my ERP system and plug in some pricing. And now we we have a data pipeline that it's pushing that pricing to your e-commerce platform. And now I need to wait for that to happen. And so there's a lot more moving parts and for folks that are having to manage all this data they're probably touching three or four or five different, different user experiences to make to make that all come together. So that's,I think folks on the store merchandising and store management side of the house, their lives have become a little bit more difficult in some ways because of the lack of transparency. How is this all kind of gluing together?
Justin Burrows: And I definitely think something that we both have in common is one of the reasons that I got into this particular role is I was on the side of the merchandisers. I was coming in and I was helping to merchandise a site, which is brutal. And it was overwhelming. And you've got somebody, who's generally like in their 20s, this is their entry into kind of the e-commerce space. I find a lot of people like if I'm doing a project for a fashion house or like couture a brand that is, you know, like well regarded. I was talking with my friend Michaela Drapes with Made Media, they do ticketing for presenters like Royal Albert Hall. And, and you know, some of those names, you might have heard Sydney Opera House, that type of thing, but the people at the New York City Ballet, she was like the people who work customer service there, they are not there because they love customer service, which but they do, but they're there because they love ballet. They love it. But a lot of these merchandisers, especially in fashion, are people who are like, they're not in E com, they're in fashion working in Ecom. And so you're having people who are passionate about the product and what they're doing.
And a lot of times they end up, and I think I'm using this correctly when I talk about tech debt, is that tech debt is, as I understand it, and you correct me if I'm wrong, tech debt is when you gloss over some things just to get something working that there are other things that you needed to fix, but that you have a work around for. Would you say that that's correct? Or do I have that misinterpreted - maybe that that could introduce tech debt?
Brian Ballard: I guess one of the things I like about composable is that you can within reason, kind of kick the can on some architectural decisions down the road knowing that. And let's say you're adding promotions to your storefront and you don't really know yet the full extent to which the business wants to use promotions. So do you spend a lot of time and engineering effort or product research looking for the be all end all of promotions engines? Or do you kind of roll your own? You know, you could roll your own, you know, like micro micro promotion engine, right, just to just to get yourself over the hump. I mean, the nice thing about composable is you can define a promotion, You can define what does a promotion mean in the context of your business, right? Again, irrespective of any vendor selection or implementation, underlying implementation details or architecture, right? What does a promotion mean and what do you need to get out of it?
Well, I can implement that a wide variety of ways, but the notion of what a promotion is going to remain fairly constant. And you might expand on that over time as the needs of the business around promotions become more and more complex. But I think there's a conversation here around right sizing of projects and architectures, right designs. Don't caution against over engineering of something where you're not really sure what the degree of opt in is going to be around a certain feature set within your platform. Like don't just kind of bolt something in, make it work, make it resilient within the context of how you've implemented it and designed it. But you know that because you have a composable architecture, you could a year from now decide that oh yeah, these promotions that we built are really, you know, turning the company. They're driving massive growth in our business. We need to go even further with this, right? And now you go all in on promotions right now you reach out to any number of the enterprise grade promotion providers out there, marketing campaign promotion coupon providers out there. And but the contract stays the same, right? So the front end doesn't change, right? The promotion's still the promotion. Maybe you need to do a little bit of the data migration behind the scenes, or maybe you've coded your promotions in such a way that you could actually reach out to both systems. So you still have your micro promotion engine still running behind the scenes. Maybe you get clever with your coupon coding in such a way the coupons are coming from the new system. But again, the contract hasn't changed and you can now kind of starve out the old, the old integration, the old service and gracefully migrate to a new service.
I don't know how we got down this path, but they're all related.
Justin Burrows: Are we talking about tech debt?
Brian Ballard: It's not tech debt. It's, I mean, it technically is tech debt, but it's tech debt that you are actually baking into your platform. Real tech debt is where you just feel like I just don't know what I want to do with this. And you just kind of bolt it in there, unknowingly implementing it in a way that really isn't serving the business well.
Justin Burrows: But my argument is that the people who pay the price for that are the merchandisers, typically the people on the front end. Yes, because they end up being full time data masseuses. It's like you don't have this integration built. You don't know what you're gonna do. Don't worry, we'll just put it in a CSV. You know what? We, I'll put in a macro. I don't know how many times you'd see somewhere somebody's got a macro and an Excel to append a single letter to the beginning of a code because they're ERP only has like, like literally there was one where it's like their ERP only allowed for 6 digits and they had way more than 6 digits’ worth of these records.
Brian Ballard: You're right and in fact, that use case that I was just happilygoing through around promotions and being able to switch platforms without affecting the overall program. I mean that is tech debt from a merchandiser standpoint because now they're having to interact with two different platforms, the legacy platform that you built to just get you going. And now the new fancy, all the bells and whistles platform. And now they're schizophrenic having to work. All right, you know, we update the campaign that that we launched last year. Well, does it live in the new system or the old system?
And now there's, there's a lot of extra you're paying for that with all these little inefficiencies in the merchandising, marketing.
Justin Burrows: A lot of times it's worth it I just think it's important that you include merchandisers in the conversation in that you actually have because the more you do those things, those little physical, which are, which are necessary, it's part of the job. But if it goes too far, that person's personal agency goes away, and you're basically making a person work for the technology and not the technology working for the person. It's unavoidable. We all hold technology up and we all work for technology. But you know, I think it's something that we should always keep in mind.
But you mentioned something that I think is a real challenge in the composable space is that like a lot of vendors come to you and they like they have this bucket of good. I just happen to have a bucket just right there. They have this bucket of goods, right? And in it it's like, well, wegot a catalog, we've got an order, we've got promotions, we've got all these things. We've got a front end, you know, accelerator. But The thing is, is that the way they price it is like, OK, here's your bucket, right? And it's like I just wanted this, you guys are composable. I think it's interesting because fundamentally that the model is, it's not a composable model, right? And you end up with these things that it's like, I mean, we've seen it over and over again. Like even like with the monoliths, like going back to. Do you remember ATG? Do you remember that platform?
Brian Ballard: Yeah, yeah, yeah.
Justin Burrows: But you know how that platform and you even see it with Salesforce Commerce Cloud, you would see little changes. It was always a tell in the UI. There was always a little bit. There was a different color. There was a different platform, like where they bolted it in. You had, what is it, the admin control console ACC? You had the BCC, then you had the promotions engine, which then kind of became ENDECA, which got rebranded. And I think if you go into Salesforce, right, even within a monolith, you're still working within 3 or 4 different tools. Because and like, you know, like that platform's evolved over time and, you know, but you go into Salesforce Conference Cloud, it says Demandware. How long ago was that? So anyway, I see the same type of thing kind of happening right, in a lot of the composable providers. It's like, well, we got it, we got to sell more, we got to do more. So they acquire all these things and then they say, oh, no, these things are all agnostic. But yet also, but you can't you end up being getting committal because it's like there's this incentive and trying to sell that I think must be really, really challenging from that perspective, right.
Brian Ballard: Well, it's hard to sell…I can imagine, you know, on the on the sales side, you know, being a technology vendor within you know, selling yourself as composable if you are selling e-commerce, it's hard to sell yourself as an e-commerce vendor without providing campaigns and promotions because that really is something that could live on its own. But coming from the world of monoliths, it is kind of hard to wrap your head around.
What does that actually look like to provide a cart and a catalog and pricing capabilities, but without promotions, right. How, how do you separate those two? And it does require a little bit of re education to imagine what an e-commerce platform that provides cart endpoints that, drive support for carts, but without promotions, right? What would that actually look like? You know, and can you even call yourself a commerce vendor if you don't provide support for promotions?
Well, I think you can. But I think we're still trying to shift away from the mindset. You know, we're still, we're still evolving in this, in this space. I think people are still thinking in terms of platforms that do everything and maybe in terms of like content. I think I think content management is now finally left, you know, left the e-commerce stable, right. It's no longer expected that that CMS will be part of an E Comm platform. But we still think in terms of all right, promotions and coupons and all this stuff still need to be part of the e-commerce offering. But they don't really have to be… like catalog management doesn't need to live there. But then you get to a point where like, well, what is what is an e-commerce vendor? Well, this is my dividing. Like what do what do they actually provide?
Justin Burrows: That is I think the kind of existential question, like the what's the word? Like the very theory of the knowledge itself. If anything is an e-commerce platform, what is it? Because it's become just there's a word, a really good word.
Brian Ballard: Do we need to abandon that that whole term completely? I think it might be worth abandoning, you know, e-commerce, the word, the phrase e-commerce platform. I think we could get rid of that cause and then break it down to well, you have catalog, you have pricing, you have merchandising, you have campaigns, you have, you have promotions and coupons, you have search, you have, you know, category, you know, you have, you know, I guess category hierarchy is part of merchandising.
Yeah, Anyway, sorry, but right, Sorry.
Justin Burrows: No, no, you're accessing. No, no, you're looking up to your, unless you're mirrored, you're looking up to your, to your, the left side of your brain. So you're accessing.
Brian Ballard: It's all happening over here. We're trying to build a coherent statement.
Justin Burrows: What you're doing is what we all kind of have to do because actually you have to get inside your brain for these things
Brian Ballard: And to actually re restructure, rewiring my, my head.
Justin Burrows: I've been really fascinated by this paper called it's like originally in German or something like that, but it's like, what if anything is, is a rabbit? And there are hares and then they're what we would call bunnies. And they're all of these in between. And there's also a similar problem with what exactly is a zebra, because zebras, there's actually a few different zebras and there's and in horses. And then there's that one horse that is spelled like really weird, like Prezowski's horse or something. And they're all like interrelated, But some of those zebras like are cousins. They're not actually brother and sister, you know. So what is it anyway? The whole thing is that there is kind of like A, and now I'm getting like woo woo. But I think there is a kind of a, a basic taxonomy refactor that, that we have to, that we have to integrate, that we have to internalize when we approach these things because there's certain challenges like I'm thinking about. But so like when I was talking with Michaela, we were talking about you think that your catalogue is complex. I have an event is a product.
OK that event has 50,000 bespoke seats with another 50,000 like potential add-ons for you know like you can pay 5…She's always like she's talking in pounds cuz she's fancy and she lives in London now. So it's like you could pay £5 or you could pay £6000 to go to the same event for this little period You know, in time. And when you were talking about like all the permutations that come naturally in kind of like a configurable SKU, you are talking about abstracting that catalog that there is no real single record for that, that only exists as an abstraction of other options.
And I think that that leads to a lot of discomfort from the old way of thinking about a catalog where it's like the Sears catalog or something like that, where you just have a long list with products and with SKUs and like trying to get people to engage with that kind of openness of, you know, especially in terms of fundamentally I think it comes to like auditability and money. Like I think financials cast a very, which necessarily financial systems, ERPs, stuff like that fundamentally have to be very conservative, very stable.
Brian Ballard: But that's not always consistency, right? I mean, having to leave the world of acid transactions, right and moving into the into a way of thinking around, you know, eventual consistency as a way of life for, you know, data consistency.
The data is eventually going to get there, right? Because now you have data flying between disparate systems all over the place. You’ve got data crossing areas, different disparate concerns within your architecture