Cart Art with Brian Ballard, Part 2

Transcript

Justin Burrows: Hi.I'm Justin Burrows. This is Commerce Chats at High Velocity. Welcome to Commerce Chats. Thank you for joining me, Brian.

Brian Ballard: So you know what is a cart? Well, you know, it's just a basket of items, right?

So you've got items and then you've got items that so each item describes, I mean, really it's just a data model, right?

So your cart is just a model and you've got items within that model.

And I should be using a text editor or something that would probably be a little easier.

And then you might have other descriptive information about what what's happening here.

But you know, This is why I think everyone in the composable space thinks contracts first, right?

You've got promotions, you might have shipping groups

You basically have a data model, but this is driven by the business needs, not by the technology

And then the question is what do I do with this model?

So thinking about going back to the original question of what is an e-commerce platform.

Well, you know, back in the day it was, well, where does my cart live?

So that's, you know, ultimately it's you're trying to you buy an e-commerce platform because you're trying to sell stuff

Well, how do I sell stuff?

Well, I sell stuff through a shopping cart, right?

So I need a place to put the shopping cart.

Well, it doesn't need to be - does your cart even need to live in an e-commerce platform?

You've got a SKU, you have a quantity at a basic level, you got a quantity.

And in the case of in the case of furniture, you might have a configuration that describes that product. Obviously there's some pricing and that that lives in here.

You've got maybe some discounts that apply to this and those are tied back to the promotions that are in your cart, right? I can model this without having any understanding of what my commerce vendors are going to be right? And I can actually build this without actually contracting with. I know my partners in the commerce space are not going to be happy with this, but does this have to live in a quote piece of commerce software that I've licensed to use?

No, I mean, I could, I could persist this this model object anywhere, right?

I could put this in the cloud and give it an ID and, you know, store it in S3, right?

You know, a lot of us use AWS, right?

So store it in S3 and then fire off an event, right?

So, or maybe we turn this into an order, right?

I know this is getting kind of small here, but it's just, yeah, like, so back to the question of what is an e-commerce platform? Well, guess what, you're building it, right?

So going back to your bucket of bolts mentality, well, AWSS 3 is in your tool chest, right? It's in your bucket of bolts.

Is S3 an e-commerce platform? No, it's just, it's just a tool that is used.

It's just object storage, right?

It's file storage or it's cloud storage, but this could be object storage or maybe all right, maybe you get more.

All right, so maybe S3 can't handle the volume, right?

So now your commerce, now your storefront grows and now you're pushing through, you know, 100,000 orders a day and S3 can't keep up because of the latencies.

Or maybe you're paying too much for S3 and now you want to put this now you want to store these objects in, I don't know, elastic cash or something like that, right?

You need to push these, these card objects closer to where the client is, right?

And you know, with S3, you're pegged to a, to a region, right?

So if the customer's coming in from, from France right now, they're, you know, now they're having to, you know, call an API that may be in the cloud or maybe it's on the edge somewhere.

But it's still the persistence of your cart is still pegged to S3, which is living in a region probably, you know, in the US or not where you need it, right?

So now you can start, you know, it's the thing about tech debt, right? So maybe the fact that I'm stuffing it in my carts in S3 is tech debt. And that's fine.

It's tech debt that I'm willing to live with right now because I want to get this going and give my business a chance to grow.

And I don't want to spend time imagining…I don't want to build for 100,000 orders a day.

I want to build for 500 orders a day, right.

And so, you know, why spend, you know, countless hundreds of thousands of dollars now building for a future that may never come to fruition because I spent so much time building the damn thing that we're never going to get there.

So what is an e-commerce platform? Well, it's just a bucket of bolts, right?

I might decide that maybe the cart lives eventually some sort of a catalog platform or, you know, like a cart engine that provides support for catalog because, you know, you have to think about where does the catalog live too, right?

I've got a SKU here, right. So I'm driving PDPs and CLPs and what's driving that, right.

Well, we could talk about that,

Justin Burrows: product detail page, category listing page, which is the grid of all the things going down.

Brian Ballard: Yeah, right.

But when I'm trying to build a design around selling, I'm not, I'm not worried about that.

When I'm trying to solve this particular problem, right now, what I'm trying to solve for is I've got a basket of items and I want to be able to convert that basket of items to an order. So now we have a contract, right? We have a contract that's a cart. And now I want to define an API…I want to define a verb called convert cart to order

So now we need to bolt on things like shipping addresses and customer information, 'cause now we need to gather that we've got payment information, payments, you know, there could be multiple, right?

If we're splitting payments across multiple services, you might have gift cards, etcetera, etcetera, right?

We've got, you know, shipping information, we've got billing, etcetera, right?

It all just gets added to the contract, right? And now we want to convert this to an order.

And, you know, maybe, maybe this is just an event, right? We've already got a payload.

So let's just send an event to an event bus, you know, like SQS or something like that, right?

So now I have not picked an e-commerce vendor in the classic sense yet, so we're just maybe we're storing, right?

But the key components of this architecture are I need, what's the word I'm thinking of in technology terms, I need a storage, I need persistent storage, right?

So I need, what's the word durability, right?

I need to make sure that it, that this payload lives somewhere and I need durability in the events, right?

And I need the events that are getting fired off.

We need we need an order here and we need a we need an event pipeline that right.

I haven't even picked an e-commerce vendor yet.

Justin Burrows: But what do you mean by durability? Does that just like it's persistence? Is it up time? What does that mean? I'm liberal arts.

Brian Ballard: It's both, right? So I need a place where I could put something where it's going to be available forever and ever. And that if for some reason, let's say, we've got an OMS system back here where these orders are eventually going to maybe we're going, this is all this is probably asynchronous So this is probably flowing through some other like there's an event here that's getting fired off. So there's a, you know, maybe it's SQF, we're staying in the AWS world is it's probably SQS, right? Something like that. But again, we haven't, this is not vendor locked. Our order pipeline is not locked into a particular e-commerce vendor.

This is asynchronous, but anyways, so it's durable because we need to be able to…let's say this fails, right? So we need to re-queue the event and right.

This this needs to be durable storage. We need it needs to be available, right. So our essential technical requirements here are availability, right?

We need to be able to persist it, it needs to be durable persistence, it needs to be durable, right? I guess my point is our requirements around this are not durability needs to be high, but latency can be low, for example, right?

So I don't need this to be available. Maybe we need to turn these orders around very quickly and lighten latency is important.

Justin Burrows: But in you mean latency is high. High latency is OK. Is that what you meant?

Brian Ballard: This particular here needs to be low, right. I mean, once we move to 100,000 orders a day. Well, now, all right, you know, what are, what are the characteristics of this, of this durable storage?

Maybe they no longer meet our requirements that we might need to move to Elasticache or we might need to move to some other model. S3 might not be a good place to store your carts.

This is wheresigning on with a commerce vendor might be helpful because now you might need to transition to a vendor that provides some edge functionality

Maybe they have a really fast cart implementation, you know where, but we're probably still going through an API gateway, right.

Justin Burrows: When you say Edge that I believe that refers to like you're basically taking data and saving it out much closer to the person who's going to be using them. Is that right?

Brian Ballard: Exactly right. One of the things that made monoliths so difficult is that you're running several instances of your monoliths and it's sort of the classic load balancing model, right? So anytime you need to to scale up, you spin up new instances of that thing. And that thing is huge, right?

So the, the operating cost of that thing is, is massive, right?

You can't get by with this tiny little runtime that you know, that it's, it, it's a massive platform.

It's probably, you know, like some massive Java stack that picks up gobs of memory because it's 100,000 active sessions running inside of that thing.

It's running thousands of threads and, and so it's huge, right? And that thing lived somewhere.

And if you've got a client that's running…if you're an international retailer and let's let's go out to, I don't know, Bulgaria or something, right?

And you've got somebody on a mobile device and let's say the long pole in this tent is the connection between the mobile device and whatever API, whatever endpoint or whatever origin it's talking to

Well, if the origin is in let's say your API gateway that application, that mobile app, is talking to us out in AWS, you know US West one, right?

Well the latency on this is going to be crazy, right? Justin Burrows: Did you say connection or convection? Did I mishear?

Brian Ballard: That was connection, connection, connection.

Justin Burrows: I was like, oh, that's a new one. It's like an oven.

Brian Ballard: OK, yes, connection on that. So let's so let's say you're, the classic headless, headless on top of a monolith, right?

So you've got an API gateway here.

Or maybe it's maybe the API living in your monolith, you know, which is right, It's a good monolith that's running, let's say in an EC2 container, you know, in in US West too

In order for this customer in Bulgaria to make the half dozen API calls needed to render the view that that this customer is trying to see, let's say it's their cart.

Well, now you've got massive latencies here that are causing it to take forever to render this view.

All right, so now you introduce some edge functionality. Maybe it's a caching layer, right?

So you got an edge cache, but there are a lot of caveats with this.

So your monolith still lives and you know US West too.

This is all the functionality, but there's a cache here that let's you know you've got to think about what's the turn

So if you're building a caching layer, it's adding this massive layer of complexity into your application.

But you're saving so the edge, your edge cache runs everywhere, right?

So if you're using AWS as a cloud provider they have presence, you know, cloud front. Is there a cloud front? I forget what they're they're it probably has Cloudflare. Maybe it's Cloudflare, you know, whatever it is, right, But you've got an edge cache and the round trip here is much, much lower because they're this this edge network has a presence in Poland

So they've, they've got a node in Poland and you've now got an asynchronous pipeline between your edge and your monolith that is keeping the data here relatively fresh So now you have object cache.

This is where we start to talk about eventual consistency

So now you have object caches, an object cache that, you know, maybe holds your product information and pricing, right?

And instead of having a blocking connection…obviously you don't want a blocking call here because now you're back to square one.

Justin Burrows: But what's a blocking call?

Brian Ballard: Just so a blocking call is if a web app or you know what, let's say a mobile app is making an API call on behalf of, of the view. So let's say you're viewing a bag, a shopping bag, right?

You've got to make an API call somewhere in a headless model, you got to make one, at least one or more API calls somewhere to get the data needed to render that shopping bag view in your app, right?

So what do we need to do to pull that together?

Well, we certainly need to get a cart.

We might need to make multiple calls to get decoration for each of the items in that cart, right?

If you need to put images and other media elements, maybe there's decorative content describing each of those products, right?

So you may need to make multiple API calls.

I'm not sure I would advocate for that because the more work that you're forcing your app here to do, the more risk of network latencies that are going to affect the user experience here, right?

So I kind of like to build this sort of large object, this kind of monolithic view of the cart with all of the decorative data.

But it's a choice, right?

So you might have product decoration here that describes our product content here, but if you're smart, you would build an API gateway that says, hey, if the client hasn't asked for product content, let's not orchestrate that, right?

Let's just only build the view that, you know, build the model that that the client requested.

So you know, you're probably using GraphQL on your API gateway, right?

So now in your edge cache, you probably have an API gateway here right in front of it. I'm sorry, kind of filling this set on the fly here. So I've got people on the other end who are eye rolling.

It's like, that's not the way I would and that's fine. You know, it's, it's just a choice, right? It's getting connections to work.

18:02 Justin Burrows: Yeah, on the fly when you're trying to this is brilliant.

Brian Ballard: It's like, go ahead. Getting connections to work on the API gateway that's smart enough to grab. So, you know, let's see if an API called get cart back here. Hopefully it's smart enough to only orchestrate what's needed right.

Maybe I want to go directly to the back end for the cart, but to get the decoration, because the decoration doesn't change very much.

Maybe I want to go to the edge for that, right?

So now I'm orchestrating to get the cart object

The cart object may only have a list of SKUs and quantities and configurations…just raw data, probably pricing and promotion in there too, but it's less information. And while I'm fetching the cart and maybe it's a much smaller payload. I am concerned about latency, but now I'm doing an awful lot less to get this.

And concurrently, I can go get all the product decoration here and it's less sensitive to cache latencies, right?

So I can cache this data on the edge.

Let's say they have a node here in Poland that has that data and there's a TTL on or time to live right on that payload that says, well, you know, this doesn't change more than once a day.

Or maybe the churn rate on your catalogue is well, you know, the merchandising team makes changes throughout the day and then they publish once a day at the end of the day, right.

Or maybe you have, you know, you have a CMS that's driving.

I'm kind of going all over the place here, maybe CMS here that that your merchandising team is using to manage all of your decorative content, right? So here's where all your product content lives and your merchandising, your merchandisers are up here, you know, doing their job merchandising.

And now you have a publishing process that says, hey, I published something whoops, now go bust the caches, right? So now this data is just cached forever, right?

Your edge cache caches your product data, your product decoration forever.

And if I'm a client and customer in Bulgaria, I only have to call an API in Poland to go get its decoration, product content.

As you start to grow, then then you start thinking about all right. And well, now we need to serve customers in Bulgaria. How do we not give them a user experience that is awful?

So when we publish now we can say, let's go bust the cache and publish, bust the cache for the products that were affected

And now we reach out to, we call whatever your edge provider is and say, every piece of content in the edge cache has a key, right?

And that key is the product ID or the SKU ID of that product.

Hey, edge provider go bust the cache for all these things.

I think it's asking a lot for a commerce vendor to provide all of this functionality, right?

That's why I say start with the contracts first, because this is what's going to live forever.

This notion of what a cart is that that basket of items will hopefully will never change.

And I can shift from a, a cart persistence or you know, a cart architecture that that is just storing payloads in S3 to, let's say we decide to use a first class e-commerce vendor at some point to persist our carts.

Well, we can through middleware living in here, right?

It's not just there's a payload, but then there's middleware that is orchestrating across because we haven't talked about middleware at all.

There's middleware that lives and this is where we're doing the work. All of our work is here and it's abstracting, you know, it's protecting us. Oh gosh, sorry, I'm drawing this all wrong. But you know what I'm saying, right?

There's there's middleware in here that is saying, all right, I know how to get from a cart to I know where to, I know what needs to be done behind the scenes to make this cart. Yeah, to persist this part, right.

And day one, it could be in, in a cloud storage bucket somewhere. But day 800, you know, maybe it's living with a, with an e-commerce vendor, but this has not changed, right?

This is the beauty of composable commerce with, you know, if I have a monolith back here, I can't really do this. And I suppose this is where composable kind of took on. Yeah, this is how we got to composable in the 1st place is, well, I need to re-platform away from our monolith.

How do I do that? Do I do it monolithically or can I do it in increments?

Can I say, well, you know what this monolith is providing?

Sorry, I'm going, I'm literally go, this is just a stream of consciousness, consciousness here.

But, you know, our monolith is providing cart promotions, coupons, campaigns, CMS, you know, it's doing everything right.

And now let's say, all right, yeah, we're tired of having everything under 1.

You know, we're spending a $2,000,000 a year in licensing fees to maintain this monolith.

24:14 And by the way, the personalization that we said that we really, really loved and that we were really, really going to use and pay a lot for. We don't use it all.

Yeah, well, and that's all changing with, you know, actually the AI is starting to maybe make that finally a kind of a reality fresh.

So, yeah, it's that's a whole ‘nother topic.

But so now we want to find there's something that you don't use that you thought you were going to use. That's how you're going to use.

You're paying for, Yeah, you're paying through the nose for it because they, you know, this vendor sees it as a as a differentiator and, and gosh darn it, you know, our customers are going to pay for that differentiation, but you know, you're not taking advantage of it at all.

And I mean, who takes full advantage of all the bells and whistles in their e-commerce platform day one, their monoliths day one? Nobody does, right? Nobody can afford to just be in the trenches building for a year and a half, you know, two years while while your company is not selling, right?

And who can afford to do that?

I don't want to.

Justin Burrows: I want to keep going with this.

But do you remember there was something? There, there were these behavioral rules that you could set and you could have like crazy scenarios. In fact, I think they were called scenarios that have scenarios where it's like you somebody visits a site three times and you could pop up a message, you could change the whole assortment. You could really almost do anything. Brought the site to a crawl. Like if you had like a lot of these and nobody, it was a great idea. Everybody loved the idea. And it was like, this is like, you know, it's like saying I'm going to take up archery one day. You know, it sounds really interesting. But the chances of you actually doing something like that, and certainly in a way that would actually move the needle in terms of like revenue, like never happened because it was onerous. It was cool, it was fun, shiny. But but yeah, you have all of these things. But again, who, who, who from day one takes advantage of all of those who even in day 365 or 720 takes advantage of those. Well, yeah, I mean, there's evolving needs.

Brian Ballard: Well, and one of the things I threw on my things to think about in your, you know, composable implementation that you don't fully need to worry about, but the more you do upfront, the better off you are. Is collecting data. Data is your friend. You don't necessarily need to do anything with it day one. You just need to collect it, right.

So for example, you mentioned writing a scenario that drives a rule that says when I hit the site a certain number of times, then show this message and you know, throw it into this slot on a page or whatever. Like you said, nobody does that, but I do care about the fact that I've had a customer login 3 times a certain period of time.

I don't necessarily know what I want to do with that data right now. And I don't want to add a rule to my platform that is blocking access to my content computationally, right?

So the problem with things like scenarios and those rules based implementations is that every time you hit the site, it's running every interaction with the platform is going through that rule set every time

What about this? Maybe this happened or maybe this happened and maybe none of those things happened, but I still need to evaluate all those rules every time. And the more rules you add to the platform, the slower it gets.

So you provided this great functionality, but the costs are tremendous of actually taking advantage of it.

Whereas maybe we do it all behind the scenes, right?

So we just gather the data.

So we've got interactions, you know, obviously we're gathering selling data, but what about other events, right?

Most retailers do this now through things like Google Analytics that folks are doing this today, but it's, usually used to help drive like, you know, like metrics around conversion, KPI, stuff like that, right.

So you're driving a dashboard that's telling you how effective my site is, but I'm not able to use it to drive user experience, right.

So ultimately all this conversation around targets and scenarios, it's all about how do I customize, how do I drive a dynamic user experience that is based on the relationship that we've built with that customer over time. We want to treat a customer who's spent lots of time and spent lots of money on our storefront, you know, give them something a little more tailored to them.

If we know that they're I don't know, let's use I guess the furniture metaphor. If we know that they've bought products with a particular fabric or design aesthetic repeatedly over multiple years, don't present them with something that is the complete opposite of the design aesthetic that they're interested in, because it's insulting to the customer.It's like, well, don't you know me by now?

You know, but it's also going to lead to a lower conversion rates.

So thinking in terms of, well, data, right?

So, you know, we're starting to talk about this now at Serena and Lily is how do we use the learning when we convert an order? And what is the when we convert an order, it's not enough to know that it's not even just conversion, you know, converging conversion.

We want to start keeping track of things like, well, what fabrics did they upholster? Did they add or custom, you know, configure on their product? What styles. We have, you know, various product families. What styles do they seem interested in, right? So it's not just about conversion.

Do they seem interested in let's sit in. So it's not just conversion, but it's also impressions, right?

Again, we do this pretty well with, you know, various analytic engines, but they're not used.

We're not good at using that data to improve conversion rates.

I guess to, to provide what's 1 of the vendors that we use or what do they call it? Products, product recommendations. There's a new word for it now and it's now escaping me. We used to call it recommendations. But there's a different term anyways, but it's basically recommendations. So browse, search, right? What is the customer telling you? If I search for something that's very important information, the customer is literally typing things into search like they're taking the effort, right? What is the effort that the that the customer is putting into finding the product that they want? Product discovery. Product discovery is kind of the new. So it's not just the quote fancy new term now. These are all events that and this is where I think this is going to start making an impact is right. So now you're starting to see more AI engines that are kind of e-commerce oriented, right? The e-commerce AI’s that are meant to replace all these old targeting, targeting recommendations engines in a wreck, engines, right, that are very statically, yeah, they, they're driven by statically engineered rule sets Static rule sets, right. If, if customer does this, then show them this. If a customer does this, show them this, right?

And nobody has the time to build those rule sets, right? Whereas now, if you haven't, in theory, if you have an AI that's gathering all of this data, we should be able to spit out set of actually, you're really just matching, right? You're just matching product to customers, right. But you're not doing it through a static rule set. You're doing it through a matching engine, right.

So you've got e-commerce AI that these e-commerce AI’s are consuming customer-facing events, right? Like browse and obviously conversions and there's information inherent in those conversions, you know, what products were bought with what right conversions. All right let's say they converted a a living room. Let's say we sold a living room. Well, do we want to sell more living room furniture to a customer that's already bought?

You know that's already furnished the living room.

Well let's show them some dining room content using some of the styles the same style like that's what a an intelligent product discovery engine would provide.

So we've got event data, but then we also have data that describes our products. And that's all being fed into the, into this AI.

So we've got our catalog and it describes our products and you've got customers that are describing themselves. Ultimately that's what's happening with these events. We're sending events and it's just customers telling us who they are.

Customers describing themselves, it's catalogues that are describing our products and, and we've got an end AI that's trying to build a model of what does this customer look like what do these products look like and trying to trying to match these up to each other, and we also in the case of our design shop model, right our design shops are also telling us interesting information because they're using our designers are basically sending us data because they're they're holding the customer's hand and taking them through the sales funnel right themselves instead of the customer being on their own on the website to the design shops.

Those designers have a lot of knowledge in their heads about how to translate what a customer is telling us about what their design aesthetic is into a set of product lines.

So if a customer is telling us that their design aesthetic is they love blue and they love red, right, or they love blue and off white, right? We're not going to push them fabrics that are, you know, red and purple, right?

We're going to or maybe purple, but I'm not a designer, I'm not not a home designer so I wouldn't really know but you know what I'm saying, right?

So, AI is going to start playing a more, more and more important role here. So in the monolithic world, we would be sitting around waiting for our monolithic, our locked in monolith to support AI.

Whereas in in the composable world, we can just say, well, you know, we've been gathering data.

You know, thinking in terms of what are the, what are the important things we need to do from a composable standpoint? I mentioned defining contracts, vendor agnostic, business focused contracts, API or otherwise, API or, or data contracts. Gathering data, right? Data is important. Gathering data, we don't need to know what we're gonna do with it. Don't worry about what we're gonna do with it, about what to do with it right now, right? Just gather it. Yeah, sorry, I have a few other things here, but I think we mentioned don't over engineer right now.

So right sizing be intentional about your technical debt, about your tech debt, right. If you're right sizing a promotions and you know ACMS or promotions engine, don't go crazy upfront, but know that you might decide to change that at some point of the future. Don't feel like you're locked into that choice, but it will cost you.

So you know it is going to cost you to rewrite it at some point in the future, but the impact of that rewrite on your overall application is going to be minimized by the fact that you're composable and that you've defined vendor agnostic contracts up here.

And then if you want to add AI later, OK, you know whatever it's just yeah, you have it. What is your AI meant to drive? Well, it's the AI is meant to drive your category landing pages? Or maybe your search result, your search result grids? So now you know, so you've defined a contract that's called search results or grid. We've got a contract called a, a product list. That product list drives your grids. I've got product A, product B, product C, right. But that you've got a view that knows how to render a product list, right. If you've got a grid page and search results page, you've got a category landing page, you know, you've got a bunch of things, but the product list contract is, is what it is. And you can drive that. You can drive that product list, that search results page from a traditional search engine that isn't employing any AI. Maybe it's just using the metadata on your products to drive. Or maybe the order of the products is driven by your merchandisers, right?

Maybe they want to boost, very specifically boost certain products and they're not willing to give over control to an AI that they have limited control over.

Or maybe, you know, if you do want to plug in an AI behind the scenes, but you can do that with a composable architecture you need to do a little bit of work in your middleware to make that happen.

But the contracts have remained unchanged. It's literally the same front end application. And it's the same user experience. I could just plug in that new engine behind the scenes and still drive the same experience. I'm rambling on. I'm going all over the place.

Justin Burrows: No, no, no, no. I think you beautifully illustrated like what happens?

OK, so I remembered the word I was talking about earlier and it was decomposed. And what I find interesting is that vendors really have to, you know, and almost it's like less vendors now and service providers is really kind of a thing like their services where a vendor is like, I have this big, I got a bucket where services you are having to sing for your supper on every single functionality. And kind of the charge of composable is having to think about what are the irreducible items components of this system. And you hit on it with like things that we don't think of as products or rather we used to not think of as something that is a product, but cart search content, like all of these things or product, product list, like these are all, they're always going to be that way.

And like you say it like we think of in terms of product list instead of grid or whatever else. It's like whenever you need a list of products, this is the contract that gets you that. And it goes back to your, you know, your, you know, initial thesis of it's like, well, what is, what is, what does the, what does e-commerce platform even mean anything?

And I think it really it, it kind of melts your brain a little bit.

But I think one of the challenges we have is that there are a lot of boxes out there that say middleware. And that's like, what is the term small matter of programming, which I have to understand is ironic where it's like we'll set this,we'll just we'll figure it out. Like eventually sometimes that's that's not especially if you're not used to it. Like creating that middleware maybe is the way to have manageable middleware like is starting by thinking through like what you just did and getting the fundamental components and kind of thinking what you're wanting to provide? That's not a small question. I understand that you could say it depends.

Brian Ballard: Well, I mean, it definitely depends because really it's just a repository of, of the orchestrations that that, you know, generate that populate the contracts that you've defined, right. So it's, it is kind of a this big bucket of whatever for your business. I wouldn't even call it business logic. I think of middleware as just orchestrating access to the data that drives the view models that you're trying to present to the to the application

So let me go back to my view again.

So drew a middleware box in here.

Justin Burrows: You'll have to forgive me, but this is my BA background. Anytime somebody does a box, it's like, wait, that's a box. What is it? Where does that box live? And what although nothing lives, where does it live? Where does it live? What's in the box? There's code in the box, right? There's, I mean, at least this is the way I think about it. It's code, right?

Brian Ballard: So it's code that orchestrates across the service providers that are, you know, holding the, the, you know, holding data. Well, what are they holding? It could be data, it could be other middlewares, right? It could be other API or it could be APIs that, well, there's always APIs back here, right? Those APIs give you access to data to other business logics to infrastructure. It's basically the glue that brings all of these disparate… what did you get back here? So you've got, I mean, I think about all the different service providers we have. You might have, I just want to draw boxes. You might have catalog, you know, provider, whatever that looks like, right? So when I say catalog provider, it's a persistent representation of what your catalog looks like right now. There might be multiple catalogs you might have, you've got pricing back here. What are all the things that are needed to drive pricings? So you got price books, pricing, price books and pricing. You've got content back here, right? So there's what is needed to drive this view, right? And what are all the things I need to talk to that to make this happen? And your middleware is the code that brings all these things together.

Justin Burrows: I guess where I'm getting at is that I found that and I've been on a couple of headless projects where we go live and the latency is atrocious and it's hard to figure out what's going on. We had a platform that just didn't have any type of front end platform that didn't really have a solid caching strategy. Like they didn't even think about it. And a lot of people can get themselves in trouble when they're doing this., I guess what I'm asking is, is that I don't know what your experience is. You probably had it all figured out. Like immediately a lot of people put in middleware, they start with the orchestration. It's the first time that they've had any elaborate, you know, of that layer, that middleware layer, and they pants themselves in terms of performance. I mean, what is it? Why is that so hard or what is it that do you think that they might be missing or is it an it depends type of thing because it's an important answer.

Brian Ballard: Yeah, I think it's a failure to think about the cost, you know, the latency cost of your orchestrations up front, right. So I think that's why a lot of folks use programming languages like Node and JavaScript and in their middleware is because it provides, it makes it very easy to write, you know, through the use of promise based orchestrations to write orchestrations that perform relatively well, especially if you have to reach out to a lot of different systems. I'm a weenie when it comes to performance. I really care an awful lot about these things. I spend a tremendous amount of time thinking about application performance monitoring and looking at the waterfall of an orchestration and understanding, you know, you got to, so you got to think about, you know, obviously concurrency. How much, how much work can I do in parallel? I need to worry about rate limits against service providers. Might need to introduce some caching here, right.

In my current world, I don't worry too, too much about caching because there's a lot of cost, there's a lot of engineering cost associated with introducing caching into because now you need to worry about data consistency. So it just gets very complicated. It's a problem that I would like to have where it'd be nice to have to worry about caching. So I don't worry so much about that upfront. When you're writing your orchestrations, you want to measure latencies across the orchestration. So for example, if I'm writing, I'm running out of space here. If I'm writing an orchestration that needs to reach across to multiple systems, all right, let's say we're trying to paint a view of a cart, right? So I've got what are the entities that I need to to get? Well, I need information about products. I might need product content. So products could be, you know, a core data, right? What is the SKU of the product?

What are the core attributes that that are part of the this core data that describes your catalog that is part of that is sort of enterprise wide right? Then your fulfillment system doesn't care about the care and feeding content that is rendered to the customer, right? But it does care about the SKU. It cares about the price that we promised. It cares about the delivery dates that we promised, right? It cares about that stuff and it cares about some other, you know, it cares about the fulfillment center that we're that we're, we've, you know, instead we'll fulfill that product, etcetera, right? But we have product content, the core data, we have product content and those probably live in different systems, maybe not, you know, you never know. You've got inventory, right? Inventory, you've got pricing that could live somewhere completely different.

You've got promotions and what are the promotions and coupons that have been added to that cart so you're trying to render a cart and you've got an orchestration that is trying to pull all this together well, and it also needs to know what's in the cart. So there's the cart contents.

Let's say we're trying to add items to the cart right and there's a payload in and there's going to be a payload out and the payload out is the updated representation of the cart and the payload in is one of the things I want to add to cart. So what are the things I need to do right up front? I probably need to validate that I have do I have access to this cart.

Do the products exist, are they available for sale or are the products available for sale in the insufficient quantities? There's a lot of stuff I need to do up front and then I can actually mutate the cart, right? Change the cart to write and now I need to write

So just to do this stuff up front, I need to go to inventory to make sure that we have sufficient product available. I might not need to get the cart contents. Well, no, I, I do need to know. I need to know who owns that cart, right? Because you might be trying to get access to somebody else's cart as there's a lot I need to do. So in terms of thinking about how do I optimize this orchestration?

I don't think about cashing upfront, right? Let's, let's not solve a problem that may or may not exist upfront. So we know we need to get the cart.

That's a sunk cost because we don't know what products are on the cart yet we don't know because we don't know what the products are we don't know how to get the inventory. So we need to get the cart contents first - we're making a call there. So thinking about the waterfall, right?

The first thing we need to do is get the cart and its contents. That's a sunk cost, right? So the first part of your orchestration is doing that. Maybe we need to validate the payload to make sure it's correct. But that's compute bound. So when I say compute bound, that means we can do that in middleware without reaching out to an external system. Validate the payload - there's a cost there.

So you need to think about what is the latest? This is the timeline, right? You need to get the card to get a tent. And then the next thing we need to do once we have that, now we need to do a little bit of validation, right? So you could do this with a sequence diagram, right? But a sequence diagram I think doesn't really it's, it describes an implementation, but it doesn't describe how to optimize that implementation.

So validate access to that cart. For example, we have customer data too, right? We know the customer that's trying to access the cart, right? When I think about optimization, well, can we get that while we're getting the cart, also get the customer right, get the customer data right, because we're trying to validate that this customer has access to this card.

So when I think in terms of because a classic a lot of people that are coming from the world of monoliths are coming from the world of probably Java or maybe like PHP or Python, right? Either a Java or PHP based back end, maybe JavaScript, but probably not, right.

So those folks are thinking, oh, get the cart, get the customer, validate access, right.

But really you could do this in parallel and I think a lot of monoliths could apply or couldn't start applying principles around improving concurrency, right?

Because if it's a monolith, you're probably making a call to a database to get a cart, you're making another call to a database to get a customer. The SDKs that wrap though, that give you access to those monoliths are really designed to be called sequentially. You're getting the card that makes a database call, you're getting the customer that makes a database call. So it's like, it's like doing this, then do this, then you know, it's it's just time consuming, right? Because you also don't want to overload your database. And we'll talk about rate limits.

Thinking in terms of concurrency, like you can make both of these calls at the same time and this would reduce the overall latency.

Then once you've verified access to the cart, now you need to fetch all of this other data, right?

Well, think about what can I do now that I know the what the list of products are like, I can spin up again, thinking more in terms of node, spin up a promise to get all the inventories of those products like using bulk.

So if your commerce or the vendors that are providing the backing store for this information, they're probably providing a REST endpoint to give you access to that data.

Are there bulk ways of fetching data in bulk, right? or do I have to or do I have to get products, you know, one at a time, right? Hopefully not. Hopefully you can get products in bulk. I'd use a query, like a query API to get you a bunch of products at once. So you know what the product IDs are, you know what the SKUs are. Let me go get all ten of those products at the same time. Let me go get all the inventories for all those products at the same time. Minimize the number of API calls you're making because you probably have rate limits that you're also being subjected to, right? So you have to stay within those rate limits.

There's a lot more we can talk about here and, and I'm happy to chat about this if you want to, we can set up another session. But thinking about concurrency, thinking about designing the orchestration in a way that, you know, that applies, you know, like a waterfall based methodology, right.

What's the first thing I need to do? Get the cart and the and the customer to, you know, that's that's the first step. The next step is get the products, get the product contents, get the product inventory, right. I can do all these potentially in parallel, right? I don't need to do this sequentially anymore like I like we used to have to in, in the monolithic days, right? Get the products, get their inventories right. You're only your latency is, is the overall latency of the the sequestration is is based on it's the longest straw right in this in this parallel a little promise, right. You might be orchestrating 20 calls across three different systems to build a view of a cart. But the latency of that orchestration and that's why I suggest to introduce logging into your orchestrations that, that are designed to measure. So you want measurements here, right? So you want to measure, measure latencies here right at every step of the way. Each promise that you spin up to get the products, get the inventory should measure its own latency, right? So do you know how long it's taking?

And a good APM, like if you're using an APM like a new relic or data dog, it's going to be able to do that for you because you can run this, this orchestration through, through an instrumentor, like an APM instrumentation that will give you a nice view of that waterfall.

58:33 So you could see exactly what's happening, right? Look at the inventory so you know where, what the long pole is or the long straw is at every step of the way. So you can really tighten up that orchestration because like a get cart or add stuff to a cart should ideally only take a, you know, maybe a few 100 milliseconds or you know, if it's taking longer than say 800 milliseconds, then you can introduce some caching. Well, maybe I don't need the latest and greatest product data.I need the latest and greatest inventories, right. So you got to think about, well, what, where do I need? Where can I cache and where can't I cache. You generally don't want to cache inventories because you don't want to oversell, right? You probably don't want to cache pricing. Well, it depends on how quickly your pricing churns. You really don't want to cache promotions. So if you're trying to get the promotions on your cart, you need to have the latest and greatest because if the promotion rolls off, you need to know that and pull that promotion off the cart. So stuff like that.

You want to be thinking about where can I get away with not having the freshest data in the world in an effort to improve user experience.

Justin Burrows: Well, I think one of the things I see from here and thank you for being here. I know you have to you have to wrap up for now, but I think you see a lot of the patterns of the way of thinking that we're having to there was a linear monolithic way of doing things. And now we're kind of in a composable way, part of the composable thing. It enables parallel thinking and I think that, you know, we're seeing a lot of that of how do we as creatures of linear thinking adapt to a multi-faceted, abstracted world. I think not letting that creep in to how we develop our products and stuff, I think really, really is a challenge. It's good for our brains. It keeps us young. It is, it really is.

Brian Ballard: And it's, it's had a huge impact on our, our ability to, to sell. I mean, we, you know, thanks to the, the inherent scalability of composable and sort of a micro services architecture built on say AWS Lambda or any of the other small runtime rapid, rapid scaling type compute compute frameworks that exist out there in the cloud. You know, we were able to scale up during peak without, you know, beating a sweat, right? We weren't sweating through peak, through the holiday peak going like, oh, is it going to survive? I mean, we were to a certain extent, but we, but it wasn't because wethought that the, that the, that the software was going to fall over. We do have bottlenecks that we want to improve in our architecture, but it wasn't, it wasn't the composable stuff that we were worried about