architecture posts - idmontie's Portfolio

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = Object.assign({
    p: "p",
    img: "img",
    h2: "h2",
    ol: "ol",
    li: "li"
  }, props.components);
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "Every system has its limits. When designing and architecting systems, sometimes we as engineers like to assume things can scale infinitely. Maybe we can use larger machines, or maybe we can deploy more instances of a service. But even trying to scale systems isn’t magically infinite. It’s a smart move to understand the limitations of what we are building upfront so we can make better technology choices, and squeeze the performance out of our existing systems."
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2023-12-08-the-breaking-point/splash.png",
        alt: "The Breaking Point"
      })
    }), "\n", _jsx(_components.h2, {
      children: "The Myth of Scaling"
    }), "\n", _jsx(_components.p, {
      children: "Software engineering often has an optimistic mindset that scaling a service is easy: we just let the more instances of the service be deployed, and viola, perfect scaling! But the reality is a bit more complex. Instances don't always fire up instantly, services can fail unexpectedly, and not every component can scale indefinitely."
    }), "\n", _jsx(_components.p, {
      children: "That’s why it’s important to understand the limitations of our services. Knowing where the system might buckle under pressure allows us to devise strategies to prevent these breaking points from turning into disasters."
    }), "\n", _jsx(_components.p, {
      children: "For example, we may have a public API service with set auto-scaling rules. When the service is hit with many requests, the service spawns additional instances to handle the workload. This ends up working well in most cases, but in the real-world, the service ends up getting hit with huge bursts of requests from downstream users, rather than a steady increase of traffic. When this happens, we scale the service instances up, but now the database gets hit with too many writes and fails over, since the database was originally designed as a read heavy use-case. Of course this is a contrived example, but similar edge-cases commonly appear in complex architectures."
    }), "\n", _jsx(_components.p, {
      children: "Without understanding the full system and the constraints of each part, it can be difficult to successfully scale."
    }), "\n", _jsx(_components.h2, {
      children: "Understanding Server Limitations"
    }), "\n", _jsx(_components.p, {
      children: "A good starting point for finding your system’s breaking point is to see how it fares on a single machine. Profile the memory usage when under load, and use load testing to determine how many simultaneous requests the service can handle before performance degrades. To make sense of it all, set some benchmarks:"
    }), "\n", _jsxs(_components.ol, {
      children: ["\n", _jsx(_components.li, {
        children: "Determine what it means for performance to degrade. In some systems, you may have SLAs for some percentage of requests to complete within a given response time. Other systems may have more leniency."
      }), "\n", _jsx(_components.li, {
        children: "Use readily available tools to bombard your service with requests and observe the results. For web API services, there are plenty of open source tools that can sustain issuing millions of requests a second and reporting results."
      }), "\n", _jsx(_components.li, {
        children: "Run the tests multiple times and throw out any outliers in the results. From the rest, understand the peak memory usage, peak disk usage, and peak CPU usage of the service."
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "Once you have the information from these types of load tests, it becomes easy to identify indicators that a service will begin to degrade. For some services it is CPU usage that will increase more than the other resources. For some services, memory can get exhausted first."
    }), "\n", _jsx(_components.p, {
      children: "The load testing will also be a good indicator of how many requests a single instance of your service can handle before falling over."
    }), "\n", _jsx(_components.h2, {
      children: "Scaling Strategies"
    }), "\n", _jsx(_components.p, {
      children: "When dealing with monolithic services or hefty microservices, you may find from load testing that just vertically scaling the service by adding more hardware is more cost efficient than replicating the instance."
    }), "\n", _jsx(_components.p, {
      children: "Fore smaller service that use less memory and CPU footprint, it may make more sense to horizontally scale the service with many additional smaller servers being deployed."
    }), "\n", _jsx(_components.p, {
      children: "Cloud services like AWS, Azure, and GCP help enable either decision. With AWS, there are many different instance types to get the right size for your specific service deployment. With the right size picked out, auto-scaling becomes a more budget-friendly option, as you're only spinning up instances that meet your specific resource requirements."
    }), "\n", _jsx(_components.p, {
      children: "Cloud providers additionally have ways to detect when to auto-scale an instance based on resource usage, making load testing very informative to tune these values."
    }), "\n", _jsx(_components.h2, {
      children: "Conclusion"
    }), "\n", _jsx(_components.p, {
      children: "Scaling isn’t a one-size-fits-all solution. It’s about understanding the unique limitations and potential of your system. Through careful analysis, load testing, and a bit of strategic thinking, you can develop a scaling strategy that not only meets your performance needs but also keeps an eye on the cost of your system. Whether it’s beefing up your hardware or multiplying smaller instances, the key lies in making informed, tailored decisions for your system's specific demands and quirks."
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, props)
  })) : _createMdxContent(props);
}
return {
  default: MDXContent
};


<p>Every system has its limits. When designing and architecting systems, sometimes we as engineers like to assume things can scale infinitely. Maybe we can use larger machines, or maybe we can deploy more instances of a service. But even trying to scale systems isn’t magically infinite. It’s a smart move to understand the limitations of what we are building upfront so we can make better technology choices, and squeeze the performance out of our existing systems.</p>
<p><img alt="The Breaking Point" src="/media/2023-12-08-the-breaking-point/splash.png" style="max-height:500px;margin:auto;text-align:center"/></p>
<h2>The Myth of Scaling</h2>
<p>Software engineering often has an optimistic mindset that scaling a service is easy: we just let the more instances of the service be deployed, and viola, perfect scaling! But the reality is a bit more complex. Instances don&#x27;t always fire up instantly, services can fail unexpectedly, and not every component can scale indefinitely.</p>
<p>That’s why it’s important to understand the limitations of our services. Knowing where the system might buckle under pressure allows us to devise strategies to prevent these breaking points from turning into disasters.</p>
<p>For example, we may have a public API service with set auto-scaling rules. When the service is hit with many requests, the service spawns additional instances to handle the workload. This ends up working well in most cases, but in the real-world, the service ends up getting hit with huge bursts of requests from downstream users, rather than a steady increase of traffic. When this happens, we scale the service instances up, but now the database gets hit with too many writes and fails over, since the database was originally designed as a read heavy use-case. Of course this is a contrived example, but similar edge-cases commonly appear in complex architectures.</p>
<p>Without understanding the full system and the constraints of each part, it can be difficult to successfully scale.</p>
<h2>Understanding Server Limitations</h2>
<p>A good starting point for finding your system’s breaking point is to see how it fares on a single machine. Profile the memory usage when under load, and use load testing to determine how many simultaneous requests the service can handle before performance degrades. To make sense of it all, set some benchmarks:</p>
<ol>
<li>Determine what it means for performance to degrade. In some systems, you may have SLAs for some percentage of requests to complete within a given response time. Other systems may have more leniency.</li>
<li>Use readily available tools to bombard your service with requests and observe the results. For web API services, there are plenty of open source tools that can sustain issuing millions of requests a second and reporting results.</li>
<li>Run the tests multiple times and throw out any outliers in the results. From the rest, understand the peak memory usage, peak disk usage, and peak CPU usage of the service.</li>
</ol>
<p>Once you have the information from these types of load tests, it becomes easy to identify indicators that a service will begin to degrade. For some services it is CPU usage that will increase more than the other resources. For some services, memory can get exhausted first.</p>
<p>The load testing will also be a good indicator of how many requests a single instance of your service can handle before falling over.</p>
<h2>Scaling Strategies</h2>
<p>When dealing with monolithic services or hefty microservices, you may find from load testing that just vertically scaling the service by adding more hardware is more cost efficient than replicating the instance.</p>
<p>Fore smaller service that use less memory and CPU footprint, it may make more sense to horizontally scale the service with many additional smaller servers being deployed.</p>
<p>Cloud services like AWS, Azure, and GCP help enable either decision. With AWS, there are many different instance types to get the right size for your specific service deployment. With the right size picked out, auto-scaling becomes a more budget-friendly option, as you&#x27;re only spinning up instances that meet your specific resource requirements.</p>
<p>Cloud providers additionally have ways to detect when to auto-scale an instance based on resource usage, making load testing very informative to tune these values.</p>
<h2>Conclusion</h2>
<p>Scaling isn’t a one-size-fits-all solution. It’s about understanding the unique limitations and potential of your system. Through careful analysis, load testing, and a bit of strategic thinking, you can develop a scaling strategy that not only meets your performance needs but also keeps an eye on the cost of your system. Whether it’s beefing up your hardware or multiplying smaller instances, the key lies in making informed, tailored decisions for your system&#x27;s specific demands and quirks.</p>


Every system has its limits. When designing and architecting systems, sometimes we as engineers like to assume things can scale infinitely. Maybe we can use larger machines, or maybe we can deploy more instances of a service. But even trying to scale systems isn’t magically infinite. It’s a smart move to understand the limitations of what we are building upfront so we can make better technology choices, and squeeze the performance out of our existing systems.

![The Breaking Point](/media/2023-12-08-the-breaking-point/splash.png)

## The Myth of Scaling

Software engineering often has an optimistic mindset that scaling a service is easy: we just let the more instances of the service be deployed, and viola, perfect scaling! But the reality is a bit more complex. Instances don't always fire up instantly, services can fail unexpectedly, and not every component can scale indefinitely.

That’s why it’s important to understand the limitations of our services. Knowing where the system might buckle under pressure allows us to devise strategies to prevent these breaking points from turning into disasters.

For example, we may have a public API service with set auto-scaling rules. When the service is hit with many requests, the service spawns additional instances to handle the workload. This ends up working well in most cases, but in the real-world, the service ends up getting hit with huge bursts of requests from downstream users, rather than a steady increase of traffic. When this happens, we scale the service instances up, but now the database gets hit with too many writes and fails over, since the database was originally designed as a read heavy use-case. Of course this is a contrived example, but similar edge-cases commonly appear in complex architectures.

Without understanding the full system and the constraints of each part, it can be difficult to successfully scale.

## Understanding Server Limitations

A good starting point for finding your system’s breaking point is to see how it fares on a single machine. Profile the memory usage when under load, and use load testing to determine how many simultaneous requests the service can handle before performance degrades. To make sense of it all, set some benchmarks:

1. Determine what it means for performance to degrade. In some systems, you may have SLAs for some percentage of requests to complete within a given response time. Other systems may have more leniency.
2. Use readily available tools to bombard your service with requests and observe the results. For web API services, there are plenty of open source tools that can sustain issuing millions of requests a second and reporting results.
3. Run the tests multiple times and throw out any outliers in the results. From the rest, understand the peak memory usage, peak disk usage, and peak CPU usage of the service.

Once you have the information from these types of load tests, it becomes easy to identify indicators that a service will begin to degrade. For some services it is CPU usage that will increase more than the other resources. For some services, memory can get exhausted first.

The load testing will also be a good indicator of how many requests a single instance of your service can handle before falling over.

## Scaling Strategies

When dealing with monolithic services or hefty microservices, you may find from load testing that just vertically scaling the service by adding more hardware is more cost efficient than replicating the instance.

Fore smaller service that use less memory and CPU footprint, it may make more sense to horizontally scale the service with many additional smaller servers being deployed.

Cloud services like AWS, Azure, and GCP help enable either decision. With AWS, there are many different instance types to get the right size for your specific service deployment. With the right size picked out, auto-scaling becomes a more budget-friendly option, as you're only spinning up instances that meet your specific resource requirements.

Cloud providers additionally have ways to detect when to auto-scale an instance based on resource usage, making load testing very informative to tune these values.

## Conclusion

Scaling isn’t a one-size-fits-all solution. It’s about understanding the unique limitations and potential of your system. Through careful analysis, load testing, and a bit of strategic thinking, you can develop a scaling strategy that not only meets your performance needs but also keeps an eye on the cost of your system. Whether it’s beefing up your hardware or multiplying smaller instances, the key lies in making informed, tailored decisions for your system's specific demands and quirks.


The Breaking Point - Understanding the performance of your systems

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = Object.assign({
    p: "p",
    a: "a",
    blockquote: "blockquote",
    ul: "ul",
    li: "li",
    code: "code",
    strong: "strong"
  }, props.components), {Mermaid} = _components;
  if (!Mermaid) _missingMdxReference("Mermaid", true);
  return _jsxs(_Fragment, {
    children: [_jsxs(_components.p, {
      children: ["Back when ", _jsx(_components.a, {
        href: "https://www.meteor.com/",
        children: "Meteor"
      }), " was in vogue, they popularized the concept of “Data on the wire”. This was the idea that the UI would be updated with changes from the dataset it is subscribed to."]
    }), "\n", _jsxs(_components.blockquote, {
      children: ["\n", _jsx(_components.p, {
        children: "Data on the wire refers mostly to the way that Meteor forms a websocket connection to the server on page load and then transfers the data needed over that connection."
      }), "\n", _jsx(_components.p, {
        children: "The websocket is a live connection and as the data changes, the updates are automatically pushed over the wire and the client updates in real time. (Similar to how many people can edit the same document simultaneously in Google Docs and see each others changes in real time)"
      }), "\n"]
    }), "\n", _jsxs(_components.ul, {
      children: ["\n", _jsx(_components.li, {
        children: _jsx(_components.a, {
          href: "https://stackoverflow.com/questions/48512242/in-meteor-what-does-data-on-the-wire-mean",
          children: "coagmano on Stackoverflow"
        })
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "This idea was pretty interesting: a UI could always be considered up to date even if another client created a new row in the database. This is because Meteor would listen to the changelog of the MongoDB it was connected to, and then if a client was subscribed to the dataset that change belonged it, Meteor would push that change over the connected websocket."
    }), "\n", _jsx(_components.p, {
      children: "Data on the wire enabled clients to stay up-to-date with any changes and create an amazing experience for users who would see changes instantly. For developers, this was also a huge win for prototyping applications."
    }), "\n", _jsx(_components.p, {
      children: "But the problem with data on the wire is that the client can subscribe to a dataset with additional MongoDB filters, but the server now needs to keep track of what clients are subscribed to what subset of data."
    }), "\n", _jsxs(_components.p, {
      children: ["If we start with a dataset ", _jsx(_components.code, {
        children: "A"
      }), ", and a user creates a new item ", _jsx(_components.code, {
        children: "i"
      }), ", then we know that if a user is subscribed to dataset ", _jsx(_components.code, {
        children: "A"
      }), ", we can just send the changes for ", _jsx(_components.code, {
        children: "i"
      }), " via a websocket."]
    }), "\n", _jsx(Mermaid, {
      chart: "sequenceDiagram\n    Client->>Server: Subscribe to dataset A\n    MongoDB-->>Server: New item i was inserted into A\n    Server-->>Client: Push new item i via websocket"
    }), "\n", _jsxs(_components.p, {
      children: ["But if we want to start scaling our servers up, then we must broadcast any change to dataset ", _jsx(_components.code, {
        children: "A"
      }), " to every service, and then each service must check if any websocket is subscribed to ", _jsx(_components.code, {
        children: "A"
      }), ":"]
    }), "\n", _jsx(Mermaid, {
      chart: "graph LR\n  MongoDB --> Server1\n  Client --> Server2\n  MongoDB --> Server2\n  MongoDB --> Server3 "
    }), "\n", _jsx(_components.p, {
      children: "(Technically, each server would either read from the MongoDB change stream, or have a service push an event to them that is listening to the change stream)"
    }), "\n", _jsxs(_components.p, {
      children: ["Now, each Service needs to additionally check if the item ", _jsx(_components.code, {
        children: "i"
      }), " meets any additional filter criteria that each subscriber may have for dataset ", _jsx(_components.code, {
        children: "A"
      }), "."]
    }), "\n", _jsx(_components.p, {
      children: "As datasets grow and filters become more computationally expensive, this can cause a lot of processing and memory burden for each Server."
    }), "\n", _jsxs(_components.p, {
      children: ["This is such a problem that it is even called out in the Meteor documentation: ", _jsx(_components.strong, {
        children: "Scaling Updates"
      }), "."]
    }), "\n", _jsxs(_components.blockquote, {
      children: ["\n", _jsx(_components.p, {
        children: "As previously mentioned, Meteor uses MongoDB’s Oplog to identify which changes to apply to which publication. Each change to the database is processed by every Meteor server, so frequent changes can result in high CPU usage across the board. At the same time, your database will come under higher load as all your servers keep fetching data from the oplog."
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "I’ve just thought about this recently since I had a conversation about keeping UI’s up-to-date with realtime data based on subscribing to that dataset via a websocket and it sounded eerily familiar to problems I have seen in the past."
    }), "\n", _jsx(_components.p, {
      children: "In fact, similarly, I’ve seen trouble with GraphQL Subscriptions having similar issues, since each server node now needs to identify when to push data back to a subscriber:"
    }), "\n", _jsxs(_components.blockquote, {
      children: ["\n", _jsx(_components.p, {
        children: "GraphQL Subscriptions (GQLS) are a mechanism which allow clients to subscribe to changes in a piece of data from the server, and get notified whenever that data changes."
      }), "\n"]
    }), "\n", _jsxs(_components.p, {
      children: [_jsx(_components.a, {
        href: "https://relay.dev/docs/v13.0.0/guided-tour/updating-data/graphql-subscriptions/",
        children: "The relay spec"
      }), " makes no mention of how a server might do this though. There is a handy tutorial by LogRocket on implementing a NodeJS server to handle subscriptions though: ", _jsx(_components.a, {
        href: "https://blog.logrocket.com/graphql-subscriptions-nodejs-express/",
        children: "GraphQL Subscriptions with NodeJS and Express"
      }), "."]
    }), "\n", _jsx(_components.p, {
      children: "But if you look at the diagrams, they seem awfully familiar…"
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, props)
  })) : _createMdxContent(props);
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it.");
}


<p>Back when <a href="https://www.meteor.com/">Meteor</a> was in vogue, they popularized the concept of “Data on the wire”. This was the idea that the UI would be updated with changes from the dataset it is subscribed to.</p>
<blockquote class="border-l-4 border-gray-300 pl-4">
<p>Data on the wire refers mostly to the way that Meteor forms a websocket connection to the server on page load and then transfers the data needed over that connection.</p>
<p>The websocket is a live connection and as the data changes, the updates are automatically pushed over the wire and the client updates in real time. (Similar to how many people can edit the same document simultaneously in Google Docs and see each others changes in real time)</p>
</blockquote>
<ul>
<li><a href="https://stackoverflow.com/questions/48512242/in-meteor-what-does-data-on-the-wire-mean">coagmano on Stackoverflow</a></li>
</ul>
<p>This idea was pretty interesting: a UI could always be considered up to date even if another client created a new row in the database. This is because Meteor would listen to the changelog of the MongoDB it was connected to, and then if a client was subscribed to the dataset that change belonged it, Meteor would push that change over the connected websocket.</p>
<p>Data on the wire enabled clients to stay up-to-date with any changes and create an amazing experience for users who would see changes instantly. For developers, this was also a huge win for prototyping applications.</p>
<p>But the problem with data on the wire is that the client can subscribe to a dataset with additional MongoDB filters, but the server now needs to keep track of what clients are subscribed to what subset of data.</p>
<p>If we start with a dataset <code>A</code>, and a user creates a new item <code>i</code>, then we know that if a user is subscribed to dataset <code>A</code>, we can just send the changes for <code>i</code> via a websocket.</p>
<div class="py-8 [&amp;_svg]:m-auto"><div class="mermaid" data-mermaid-src="sequenceDiagram
    Client-&gt;&gt;Server: Subscribe to dataset A
    MongoDB--&gt;&gt;Server: New item i was inserted into A
    Server--&gt;&gt;Client: Push new item i via websocket">sequenceDiagram
    Client-&gt;&gt;Server: Subscribe to dataset A
    MongoDB--&gt;&gt;Server: New item i was inserted into A
    Server--&gt;&gt;Client: Push new item i via websocket</div></div>
<p>But if we want to start scaling our servers up, then we must broadcast any change to dataset <code>A</code> to every service, and then each service must check if any websocket is subscribed to <code>A</code>:</p>
<div class="py-8 [&amp;_svg]:m-auto"><div class="mermaid" data-mermaid-src="graph LR
  MongoDB --&gt; Server1
  Client --&gt; Server2
  MongoDB --&gt; Server2
  MongoDB --&gt; Server3 ">graph LR
  MongoDB --&gt; Server1
  Client --&gt; Server2
  MongoDB --&gt; Server2
  MongoDB --&gt; Server3 </div></div>
<p>(Technically, each server would either read from the MongoDB change stream, or have a service push an event to them that is listening to the change stream)</p>
<p>Now, each Service needs to additionally check if the item <code>i</code> meets any additional filter criteria that each subscriber may have for dataset <code>A</code>.</p>
<p>As datasets grow and filters become more computationally expensive, this can cause a lot of processing and memory burden for each Server.</p>
<p>This is such a problem that it is even called out in the Meteor documentation: <strong>Scaling Updates</strong>.</p>
<blockquote class="border-l-4 border-gray-300 pl-4">
<p>As previously mentioned, Meteor uses MongoDB’s Oplog to identify which changes to apply to which publication. Each change to the database is processed by every Meteor server, so frequent changes can result in high CPU usage across the board. At the same time, your database will come under higher load as all your servers keep fetching data from the oplog.</p>
</blockquote>
<p>I’ve just thought about this recently since I had a conversation about keeping UI’s up-to-date with realtime data based on subscribing to that dataset via a websocket and it sounded eerily familiar to problems I have seen in the past.</p>
<p>In fact, similarly, I’ve seen trouble with GraphQL Subscriptions having similar issues, since each server node now needs to identify when to push data back to a subscriber:</p>
<blockquote class="border-l-4 border-gray-300 pl-4">
<p>GraphQL Subscriptions (GQLS) are a mechanism which allow clients to subscribe to changes in a piece of data from the server, and get notified whenever that data changes.</p>
</blockquote>
<p><a href="https://relay.dev/docs/v13.0.0/guided-tour/updating-data/graphql-subscriptions/">The relay spec</a> makes no mention of how a server might do this though. There is a handy tutorial by LogRocket on implementing a NodeJS server to handle subscriptions though: <a href="https://blog.logrocket.com/graphql-subscriptions-nodejs-express/">GraphQL Subscriptions with NodeJS and Express</a>.</p>
<p>But if you look at the diagrams, they seem awfully familiar…</p>


Back when [Meteor](https://www.meteor.com/) was in vogue, they popularized the concept of “Data on the wire”. This was the idea that the UI would be updated with changes from the dataset it is subscribed to.

> Data on the wire refers mostly to the way that Meteor forms a websocket connection to the server on page load and then transfers the data needed over that connection.
>
> The websocket is a live connection and as the data changes, the updates are automatically pushed over the wire and the client updates in real time. (Similar to how many people can edit the same document simultaneously in Google Docs and see each others changes in real time)
>

- [coagmano on Stackoverflow](https://stackoverflow.com/questions/48512242/in-meteor-what-does-data-on-the-wire-mean)

This idea was pretty interesting: a UI could always be considered up to date even if another client created a new row in the database. This is because Meteor would listen to the changelog of the MongoDB it was connected to, and then if a client was subscribed to the dataset that change belonged it, Meteor would push that change over the connected websocket.

Data on the wire enabled clients to stay up-to-date with any changes and create an amazing experience for users who would see changes instantly. For developers, this was also a huge win for prototyping applications.

But the problem with data on the wire is that the client can subscribe to a dataset with additional MongoDB filters, but the server now needs to keep track of what clients are subscribed to what subset of data.

If we start with a dataset `A`, and a user creates a new item `i`, then we know that if a user is subscribed to dataset `A`, we can just send the changes for `i` via a websocket.

```mermaid
sequenceDiagram
    Client->>Server: Subscribe to dataset A
    MongoDB-->>Server: New item i was inserted into A
    Server-->>Client: Push new item i via websocket
```

But if we want to start scaling our servers up, then we must broadcast any change to dataset `A` to every service, and then each service must check if any websocket is subscribed to `A`:

```mermaid
graph LR
  MongoDB --> Server1
  Client --> Server2
  MongoDB --> Server2
  MongoDB --> Server3 
```

(Technically, each server would either read from the MongoDB change stream, or have a service push an event to them that is listening to the change stream)

Now, each Service needs to additionally check if the item `i` meets any additional filter criteria that each subscriber may have for dataset `A`.

As datasets grow and filters become more computationally expensive, this can cause a lot of processing and memory burden for each Server.

This is such a problem that it is even called out in the Meteor documentation: **Scaling Updates**.

> As previously mentioned, Meteor uses MongoDB’s Oplog to identify which changes to apply to which publication. Each change to the database is processed by every Meteor server, so frequent changes can result in high CPU usage across the board. At the same time, your database will come under higher load as all your servers keep fetching data from the oplog.
>

I’ve just thought about this recently since I had a conversation about keeping UI’s up-to-date with realtime data based on subscribing to that dataset via a websocket and it sounded eerily familiar to problems I have seen in the past.

In fact, similarly, I’ve seen trouble with GraphQL Subscriptions having similar issues, since each server node now needs to identify when to push data back to a subscriber:

> GraphQL Subscriptions (GQLS) are a mechanism which allow clients to subscribe to changes in a piece of data from the server, and get notified whenever that data changes.
>

[The relay spec](https://relay.dev/docs/v13.0.0/guided-tour/updating-data/graphql-subscriptions/) makes no mention of how a server might do this though. There is a handy tutorial by LogRocket on implementing a NodeJS server to handle subscriptions though: [GraphQL Subscriptions with NodeJS and Express](https://blog.logrocket.com/graphql-subscriptions-nodejs-express/).

But if you look at the diagrams, they seem awfully familiar…


Data on the Wire

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = Object.assign({
    p: "p",
    ul: "ul",
    li: "li",
    h2: "h2"
  }, props.components), {Mermaid} = _components;
  if (!Mermaid) _missingMdxReference("Mermaid", true);
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "The Backend for the Frontend (BEFFE) is typically stateless and acts as a proxy for other services, including authentication, authorization, and core services. The recent divorce of browser code being rendered by backend services was created by SPAs - Single Page Applications. In simpler architectures, a SPA and service could be as simple as:"
    }), "\n", _jsx(Mermaid, {
      chart: "graph LR\n  SPA --> Backend"
    }), "\n", _jsx(_components.p, {
      children: "SPAs also became popular because the codebase could be built into static assets that could be services."
    }), "\n", _jsx(_components.p, {
      children: "However, as complexity arises with the above architecture and the SPA starts to rely on more services, a thin proxy is typically introduced, like NGINX:"
    }), "\n", _jsx(Mermaid, {
      chart: "graph LR\n  SPA --> NGINX\n  NGINX --> Service1\n  NGINX --> Service2\n  NGINX --> Service3"
    }), "\n", _jsx(_components.p, {
      children: "While proxies like NGINX can continue to be useful, using it as a proxy ends up putting a lot of routing and additional API handling logic on the client. The client now ends up also having to support, understand, and embed how to interact with the data rom all the downstream services. To simplify and create consistent contracts with the UI, we can create a proxy service: a Backend for the Frontend:"
    }), "\n", _jsx(Mermaid, {
      chart: "graph LR\n  SPA --> BEFFE\n  BEFFE --> Service1\n  BEFFE --> Service2\n  BEFFE --> Service3"
    }), "\n", _jsx(_components.p, {
      children: "The Backend for the Frontend ends up serving two purposes:"
    }), "\n", _jsxs(_components.ul, {
      children: ["\n", _jsx(_components.li, {
        children: "It acts as a proxy for all UI requests"
      }), "\n", _jsx(_components.li, {
        children: "It manages and massages the APIs of downstream services to be a consistent API for the UI."
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "When building a UI, only a single service that acts like an API Gateway is required to understand and encode within the application."
    }), "\n", _jsx(_components.p, {
      children: "The Backend for the Frontend model also allows engineers working on services to separate logic for the UI with the logic from an internal service. An internal service can rely on the Backend for the Frontend to handle things like Authentication, Authorization, Caching, and Rate-limits. These mechanisms may also be implemented as independent services, but the Backend for the Frontend can make multiple API calls per any API request from the UI."
    }), "\n", _jsx(_components.h2, {
      children: "Moving away from SPAs"
    }), "\n", _jsx(_components.p, {
      children: "A newer development in the UI space is server side render the application. If we are already introducing additional complexity by having a Backend for the Frontend, then why not combine the UI code with the backend service and generate hydrated and cacheable pages via the server instead of using a SPA?"
    }), "\n", _jsx(Mermaid, {
      chart: "graph LR\n  UIServer[\"UI Server\"] --> Service1\n  UIServer --> Service2\n UIServer --> Service3"
    }), "\n", _jsx(_components.p, {
      children: "With this design, we increase the complexity - the backend not has to be able to render the UI code. It must now render, hydrate, make requests to downstream APIs, and send those server-side rendered pages to the browser."
    }), "\n", _jsx(_components.p, {
      children: "But this additional complexity enabled us to:"
    }), "\n", _jsxs(_components.ul, {
      children: ["\n", _jsx(_components.li, {
        children: "create truly authenticated and authorized routes – the code for those routes isn’t even streamed to the browser, unlike a SPA."
      }), "\n", _jsx(_components.li, {
        children: "near instant page loads – reducing the bundle size by just sending the JavaScript needed to run the page makes this possible. And since API calls are happening within the same network, this saves the user from additional loading spinners after the initial bundle has been loaded."
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "The general trend of having a Backend for the Frontend comes with additional complexity. It stems from the want to enhance the experience of the user, and probably from a subconscious desire to return to a simpler time:"
    }), "\n", _jsx(Mermaid, {
      chart: "graph TD\n  Server"
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, props)
  })) : _createMdxContent(props);
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it.");
}


<p>The Backend for the Frontend (BEFFE) is typically stateless and acts as a proxy for other services, including authentication, authorization, and core services. The recent divorce of browser code being rendered by backend services was created by SPAs - Single Page Applications. In simpler architectures, a SPA and service could be as simple as:</p>
<div class="py-8 [&amp;_svg]:m-auto"><div class="mermaid" data-mermaid-src="graph LR
  SPA --&gt; Backend">graph LR
  SPA --&gt; Backend</div></div>
<p>SPAs also became popular because the codebase could be built into static assets that could be services.</p>
<p>However, as complexity arises with the above architecture and the SPA starts to rely on more services, a thin proxy is typically introduced, like NGINX:</p>
<div class="py-8 [&amp;_svg]:m-auto"><div class="mermaid" data-mermaid-src="graph LR
  SPA --&gt; NGINX
  NGINX --&gt; Service1
  NGINX --&gt; Service2
  NGINX --&gt; Service3">graph LR
  SPA --&gt; NGINX
  NGINX --&gt; Service1
  NGINX --&gt; Service2
  NGINX --&gt; Service3</div></div>
<p>While proxies like NGINX can continue to be useful, using it as a proxy ends up putting a lot of routing and additional API handling logic on the client. The client now ends up also having to support, understand, and embed how to interact with the data rom all the downstream services. To simplify and create consistent contracts with the UI, we can create a proxy service: a Backend for the Frontend:</p>
<div class="py-8 [&amp;_svg]:m-auto"><div class="mermaid" data-mermaid-src="graph LR
  SPA --&gt; BEFFE
  BEFFE --&gt; Service1
  BEFFE --&gt; Service2
  BEFFE --&gt; Service3">graph LR
  SPA --&gt; BEFFE
  BEFFE --&gt; Service1
  BEFFE --&gt; Service2
  BEFFE --&gt; Service3</div></div>
<p>The Backend for the Frontend ends up serving two purposes:</p>
<ul>
<li>It acts as a proxy for all UI requests</li>
<li>It manages and massages the APIs of downstream services to be a consistent API for the UI.</li>
</ul>
<p>When building a UI, only a single service that acts like an API Gateway is required to understand and encode within the application.</p>
<p>The Backend for the Frontend model also allows engineers working on services to separate logic for the UI with the logic from an internal service. An internal service can rely on the Backend for the Frontend to handle things like Authentication, Authorization, Caching, and Rate-limits. These mechanisms may also be implemented as independent services, but the Backend for the Frontend can make multiple API calls per any API request from the UI.</p>
<h2>Moving away from SPAs</h2>
<p>A newer development in the UI space is server side render the application. If we are already introducing additional complexity by having a Backend for the Frontend, then why not combine the UI code with the backend service and generate hydrated and cacheable pages via the server instead of using a SPA?</p>
<div class="py-8 [&amp;_svg]:m-auto"><div class="mermaid" data-mermaid-src="graph LR
  UIServer[&quot;UI Server&quot;] --&gt; Service1
  UIServer --&gt; Service2
 UIServer --&gt; Service3">graph LR
  UIServer[&quot;UI Server&quot;] --&gt; Service1
  UIServer --&gt; Service2
 UIServer --&gt; Service3</div></div>
<p>With this design, we increase the complexity - the backend not has to be able to render the UI code. It must now render, hydrate, make requests to downstream APIs, and send those server-side rendered pages to the browser.</p>
<p>But this additional complexity enabled us to:</p>
<ul>
<li>create truly authenticated and authorized routes – the code for those routes isn’t even streamed to the browser, unlike a SPA.</li>
<li>near instant page loads – reducing the bundle size by just sending the JavaScript needed to run the page makes this possible. And since API calls are happening within the same network, this saves the user from additional loading spinners after the initial bundle has been loaded.</li>
</ul>
<p>The general trend of having a Backend for the Frontend comes with additional complexity. It stems from the want to enhance the experience of the user, and probably from a subconscious desire to return to a simpler time:</p>
<div class="py-8 [&amp;_svg]:m-auto"><div class="mermaid" data-mermaid-src="graph TD
  Server">graph TD
  Server</div></div>


The Backend for the Frontend (BEFFE) is typically stateless and acts as a proxy for other services, including authentication, authorization, and core services. The recent divorce of browser code being rendered by backend services was created by SPAs - Single Page Applications. In simpler architectures, a SPA and service could be as simple as:

```mermaid
graph LR
  SPA --> Backend
```

SPAs also became popular because the codebase could be built into static assets that could be services.

However, as complexity arises with the above architecture and the SPA starts to rely on more services, a thin proxy is typically introduced, like NGINX:

```mermaid
graph LR
  SPA --> NGINX
  NGINX --> Service1
  NGINX --> Service2
  NGINX --> Service3
```

While proxies like NGINX can continue to be useful, using it as a proxy ends up putting a lot of routing and additional API handling logic on the client. The client now ends up also having to support, understand, and embed how to interact with the data rom all the downstream services. To simplify and create consistent contracts with the UI, we can create a proxy service: a Backend for the Frontend:

```mermaid
graph LR
  SPA --> BEFFE
  BEFFE --> Service1
  BEFFE --> Service2
  BEFFE --> Service3
```

The Backend for the Frontend ends up serving two purposes:

- It acts as a proxy for all UI requests
- It manages and massages the APIs of downstream services to be a consistent API for the UI.

When building a UI, only a single service that acts like an API Gateway is required to understand and encode within the application.

The Backend for the Frontend model also allows engineers working on services to separate logic for the UI with the logic from an internal service. An internal service can rely on the Backend for the Frontend to handle things like Authentication, Authorization, Caching, and Rate-limits. These mechanisms may also be implemented as independent services, but the Backend for the Frontend can make multiple API calls per any API request from the UI.

## Moving away from SPAs

A newer development in the UI space is server side render the application. If we are already introducing additional complexity by having a Backend for the Frontend, then why not combine the UI code with the backend service and generate hydrated and cacheable pages via the server instead of using a SPA?

```mermaid
graph LR
  UIServer["UI Server"] --> Service1
  UIServer --> Service2
 UIServer --> Service3
```

With this design, we increase the complexity - the backend not has to be able to render the UI code. It must now render, hydrate, make requests to downstream APIs, and send those server-side rendered pages to the browser.

But this additional complexity enabled us to:

- create truly authenticated and authorized routes – the code for those routes isn’t even streamed to the browser, unlike a SPA.
- near instant page loads – reducing the bundle size by just sending the JavaScript needed to run the page makes this possible. And since API calls are happening within the same network, this saves the user from additional loading spinners after the initial bundle has been loaded.

The general trend of having a Backend for the Frontend comes with additional complexity. It stems from the want to enhance the experience of the user, and probably from a subconscious desire to return to a simpler time:

```mermaid
graph TD
  Server
```


The Breaking Point - Understanding the performance of your systems

Data on the Wire

Backend for Frontend