7 articles tagged “ai” — software, engineering, and related writing by idmontie.

AI blog posts - idmontie's Portfolio

"use strict";
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = {
    a: "a",
    code: "code",
    img: "img",
    p: "p",
    pre: "pre",
    ...props.components
  };
  return _jsxs(_Fragment, {
    children: [_jsxs(_components.p, {
      children: ["I recently published a utility ", _jsx(_components.a, {
        href: "https://github.com/idmontie/human-agent-review-tool",
        children: "Human Agent Review Tool"
      }), " that helps keep track of the code I've actually reviewed while working with coding agents."]
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2026-09-16-review-tool/repo.png",
        alt: "Review Tool Repo"
      })
    }), "\n", _jsx(_components.p, {
      children: "AI coding agents have made it possible to iterate on ideas and features extremely quickly. This is extremely helpful when building proofs of concept, experimenting with ideas, and creating internal scripts and utilities."
    }), "\n", _jsx(_components.p, {
      children: "But that workflow of working with a coding agent to produce code introduces a simple problem: which files have I actually reviewed?"
    }), "\n", _jsx(_components.p, {
      children: "When using Cursor, each agent workflow typically let's you review files. But this interface is typically all or nothings – once you commit and move onto the next workflow, that diff is gone."
    }), "\n", _jsx(_components.p, {
      children: "In a traditional workflow, a pull request provides a nice interface and check-in point for code review. When working locally with an agent, I don't necessarily create a PR for every single feature, nor do I want to for work that might be throwaway code."
    }), "\n", _jsx(_components.p, {
      children: "After several rounds of prompting, testing, and iterating, it can become suprisingly difficult to remember what code changes I have personally looked at."
    }), "\n", _jsx(_components.p, {
      children: "The Human Agent Review Tool gives a lightweight way to track that."
    }), "\n", _jsx(_components.p, {
      children: "After globally installing via"
    }), "\n", _jsx(_components.pre, {
      children: _jsx(_components.code, {
        className: "language-bash",
        children: "pnpm i -g review-tool\n"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["you can run ", _jsx(_components.code, {
        children: "review-tool"
      }), " from a Git repository and it will open a local dashboard containing all the files in the project. Each file can be marked as Reviewed, Unreviewed, or Needs Changes."]
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2026-09-16-review-tool/review-tool-dashboard.png",
        alt: "Review Tool Dashboard"
      })
    }), "\n", _jsx(_components.p, {
      children: "I can also attach notes to the files that need additional work, giving me a simple queue for the next iteration with the coding agent."
    }), "\n", _jsx(_components.p, {
      children: "The tool tracks the review state separately from the source code, and if a previously reviewed file changes, that review state becomes stale so that it can be reviewed again. It also supports ignoring generated files or other files that aren't useful to inspect manually."
    }), "\n", _jsx(_components.p, {
      children: "The goal isn't to replace pull requests, or any traditional code review. Rather it is meant to help with the local development workflow that happens before a traditional code review: allowing a human and agent to rapidly build together on a local codebase."
    }), "\n", _jsx(_components.p, {
      children: "As coding agents become capable of producing larger features and code changes, I think tools that help developers maintain oversight will become increasingly important."
    }), "\n", _jsxs(_components.p, {
      children: ["The project is available on Github: ", _jsx(_components.a, {
        href: "https://github.com/idmontie/human-agent-review-tool",
        children: "https://github.com/idmontie/human-agent-review-tool"
      }), "."]
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, {
    ...props,
    children: _jsx(_createMdxContent, {
      ...props
    })
  }) : _createMdxContent(props);
}
return {
  default: MDXContent
};


I recently published a utility <a href="https://github.com/idmontie/human-agent-review-tool">Human Agent Review Tool</a> that helps keep track of the code I&#x27;ve actually reviewed while working with coding agents.
<img alt="Review Tool Repo" loading="lazy" width="1" height="1" decoding="async" data-nimg="1" class="mx-auto block !h-auto !w-auto max-h-[500px] max-w-full" style="color:transparent;width:auto;height:auto;max-width:100%;max-height:500px;margin:auto" src="/media/2026-09-16-review-tool/repo.png"/>
AI coding agents have made it possible to iterate on ideas and features extremely quickly. This is extremely helpful when building proofs of concept, experimenting with ideas, and creating internal scripts and utilities.
But that workflow of working with a coding agent to produce code introduces a simple problem: which files have I actually reviewed?
When using Cursor, each agent workflow typically let&#x27;s you review files. But this interface is typically all or nothings – once you commit and move onto the next workflow, that diff is gone.
In a traditional workflow, a pull request provides a nice interface and check-in point for code review. When working locally with an agent, I don&#x27;t necessarily create a PR for every single feature, nor do I want to for work that might be throwaway code.
After several rounds of prompting, testing, and iterating, it can become suprisingly difficult to remember what code changes I have personally looked at.
The Human Agent Review Tool gives a lightweight way to track that.
After globally installing via
<div class="overflow-auto rounded-2xl bg-[#0e005d6d] p-4 font-mono text-sm dark:bg-[#0e005d6d] dark:text-gray-100"><pre><code class="language-bash">pnpm i -g review-tool
</code></pre></div>
you can run <code>review-tool</code> from a Git repository and it will open a local dashboard containing all the files in the project. Each file can be marked as Reviewed, Unreviewed, or Needs Changes.
<img alt="Review Tool Dashboard" loading="lazy" width="1" height="1" decoding="async" data-nimg="1" class="mx-auto block !h-auto !w-auto max-h-[500px] max-w-full" style="color:transparent;width:auto;height:auto;max-width:100%;max-height:500px;margin:auto" src="/media/2026-09-16-review-tool/review-tool-dashboard.png"/>
I can also attach notes to the files that need additional work, giving me a simple queue for the next iteration with the coding agent.
The tool tracks the review state separately from the source code, and if a previously reviewed file changes, that review state becomes stale so that it can be reviewed again. It also supports ignoring generated files or other files that aren&#x27;t useful to inspect manually.
The goal isn&#x27;t to replace pull requests, or any traditional code review. Rather it is meant to help with the local development workflow that happens before a traditional code review: allowing a human and agent to rapidly build together on a local codebase.
As coding agents become capable of producing larger features and code changes, I think tools that help developers maintain oversight will become increasingly important.
The project is available on Github: <a href="https://github.com/idmontie/human-agent-review-tool">https://github.com/idmontie/human-agent-review-tool</a>.


I recently published a utility [Human Agent Review Tool](https://github.com/idmontie/human-agent-review-tool) that helps keep track of the code I've actually reviewed while working with coding agents.

![Review Tool Repo](/media/2026-09-16-review-tool/repo.png)

AI coding agents have made it possible to iterate on ideas and features extremely quickly. This is extremely helpful when building proofs of concept, experimenting with ideas, and creating internal scripts and utilities.

But that workflow of working with a coding agent to produce code introduces a simple problem: which files have I actually reviewed?

When using Cursor, each agent workflow typically let's you review files. But this interface is typically all or nothings – once you commit and move onto the next workflow, that diff is gone.

In a traditional workflow, a pull request provides a nice interface and check-in point for code review. When working locally with an agent, I don't necessarily create a PR for every single feature, nor do I want to for work that might be throwaway code.

After several rounds of prompting, testing, and iterating, it can become suprisingly difficult to remember what code changes I have personally looked at.

The Human Agent Review Tool gives a lightweight way to track that.

After globally installing via

```bash
pnpm i -g review-tool
```

you can run `review-tool` from a Git repository and it will open a local dashboard containing all the files in the project. Each file can be marked as Reviewed, Unreviewed, or Needs Changes.

![Review Tool Dashboard](/media/2026-09-16-review-tool/review-tool-dashboard.png)

I can also attach notes to the files that need additional work, giving me a simple queue for the next iteration with the coding agent.

The tool tracks the review state separately from the source code, and if a previously reviewed file changes, that review state becomes stale so that it can be reviewed again. It also supports ignoring generated files or other files that aren't useful to inspect manually.

The goal isn't to replace pull requests, or any traditional code review. Rather it is meant to help with the local development workflow that happens before a traditional code review: allowing a human and agent to rapidly build together on a local codebase.

As coding agents become capable of producing larger features and code changes, I think tools that help developers maintain oversight will become increasingly important.

The project is available on Github: https://github.com/idmontie/human-agent-review-tool.


Keeping Humans in the Loop for Agent-Written Code

"use strict";
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = {
    a: "a",
    blockquote: "blockquote",
    img: "img",
    p: "p",
    ...props.components
  }, {Separator} = _components;
  if (!Separator) _missingMdxReference("Separator", true);
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "What happens to SEO when the crawler itself becomes one of your primary consumers?"
    }), "\n", _jsx(_components.p, {
      children: "We all intuitively know how SEO is supposed to work. Google and other search engines have been working for decades to make sure that search returns high quality content, rather than spam that stuffs the page with keywords. We would write meaningful, quality content, and Google with crawl the page, index the data, and humans would search and visit the website."
    }), "\n", _jsxs(_components.p, {
      children: ["We built the modern web around the assumption that crawlers are lightweight readers of web pages. Increasingly, they are behaving more like automated users. With ", _jsx(_components.a, {
        href: "https://www.imperva.com/blog/bad-bot-report-2026-bots-agentic-age",
        children: "53% percent of traffic"
      }), " is bot traffic on the internet, how we write web applications, serve them, and the end goal of SEO may be changing."]
    }), "\n", _jsx(Separator, {}), "\n", _jsxs(_components.p, {
      children: ["When we launched the new version of Couchsurfing, we expected all sorts of issues. We anticipated a large spike in traffic when we launched, even though we weren’t planning on any immediate marketing push. But what we didn’t expect was to continue to see large spikes in traffic after we launched. These spikes wouldn’t just get a web page via a cURL GET request, but they would request all the different locales, all the image assets on the page, and all the other assets on the page. Some of the traffic we looked at would even attempt to create URLs that didn’t exist in our web application, hammering us with requests that we would 404. These weren’t from Googlebot. These were tons of other crawlers that were scraping our site for information - ", _jsx(_components.a, {
        href: "https://claude.com/",
        children: "Claudebot"
      }), ", ", _jsx(_components.a, {
        href: "https://meta.com/",
        children: "meta"
      }), ", other nefarious bots that would not return correct User Agent strings."]
    }), "\n", _jsx(_components.p, {
      children: "Why were AI crawlers hitting the application so differently from Google bot?"
    }), "\n", _jsx(Separator, {}), "\n", _jsx(_components.p, {
      children: "To understand why this was so surprising, we have to understand who the site was originally designed for."
    }), "\n", _jsx(_components.p, {
      children: "When we started thinking about the next version of Couchsurfing, we decided that we would want to focus on “SSR” - Server Side Rendering” - as much of the page as possible. That is essentially how the existing application worked and we wanted to make sure the majority of the page wouldn’t have cascading loaders like Single Page Applications often have. We also knew wanted to move the application into React. The previous team had already tried implementing a few React components into the application, but it was slow progress."
    }), "\n", _jsx(_components.p, {
      children: "With SSR and React as the baseline of what we wanted for the web application, we settled on NextJS as the framework. From there, every product kick-off would start with a few key questions: is this a public facing feature or page? And how much of it should be server-side rendered?"
    }), "\n", _jsx(_components.p, {
      children: "During the inception of this new project, there was quite a lot of discussion by Vercel, Next.js and web engineering at large about how much SSR was really required."
    }), "\n", _jsxs(_components.p, {
      children: [_jsx(_components.a, {
        href: "https://nextjs.org/learn/seo/webcrawlers",
        children: "NextJS"
      }), " and ", _jsx(_components.a, {
        href: "https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics",
        children: "Google Search Central"
      }), " have typically described the indexing process like this:"]
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2026-09-08-are-crawlers-our-primary-consumers/crawlqueue.png",
        alt: "Crawling Indexing Process"
      })
    }), "\n", _jsx(_components.p, {
      children: "Where the crawler will have items added to the render queue to actually execute the JS on the page. When we kicked off the project, we assumed that with the number of pages we were going to have to re-index with Google, that this step would take awhile to happen for our site. And when it did happen, we wouldn’t expect all our pages to be rendered in a reasonable timeframe. We estimated that our new multi-sitemap would have millions of pages, rather than the single 500,000 entries we had in the existing sitemap. We would have to worry about the crawl budget, so we settled on SSR’ing as much of the page as possible so that indexing could happen without the rendering step."
    }), "\n", _jsx(_components.p, {
      children: "But Googlebot wasn’t the problem."
    }), "\n", _jsx(Separator, {}), "\n", _jsx(_components.p, {
      children: "AI Crawlers hammer the application."
    }), "\n", _jsx(_components.p, {
      children: "Crawling traffic. Thousands, hundreds of thousands of requests a day. We had anticipated Googlebot - the traditional bot that crawls pages to index them for Google search results. And that is the bot we had optimized SEO for. But we saw way more traffic than we expected from other crawl bots. We saw Meta’s bot crawling every single one of our user pages, location pages. We saw claude bot hit every single page in every single language we had localizes We saw weird scrapers that didn’t properly represent themselves try to hit thousand of user pages that required users to be logged in to see."
    }), "\n", _jsx(_components.p, {
      children: "We were seeing lots of burst traffic. Claudebot for example would suddenly hit us with thousands of requests all at once. Our WAF had 429 rules to tell it to back-off, and we even had additional rate-limits on certain APIs that return 429 as well. This helped when the requests would come from the same IP address, but we would often see these large bursts come from clusters of IP addresses."
    }), "\n", _jsx(_components.p, {
      children: "Pages would start responding slow due the insane influx of traffic, which wasn’t just to the page it was crawling, but to all the assets on the page, including images, JS files, and additional APIs that the page would normally request once hydrated on the client. This is the opposite of what we had expected - we expected crawling bots to simple do a GET to the page they were requesting and crawl the contents of that page. But crawling traffic for these LLM bots would actually evaluate the page like an end-user - some explicitly using tools like Puppeteer or Playwright according to the request headers."
    }), "\n", _jsx(_components.p, {
      children: "The crawler was no longer simply requesting the server-side rendered version of the page. Instead the crawlers were acting like full users - hitting the page, requesting all the assets and image, and then following links on that page. Worse, some acted like malicious attacked - trying to enumerate pages that didn’t even exist to see if there was content on them."
    }), "\n", _jsx(Separator, {}), "\n", _jsxs(_components.p, {
      children: ["Before AI, we would have considered these attacks. After AI, this is the new normal. As one Redditor describes, ", _jsx(_components.a, {
        href: "https://www.reddit.com/r/webdev/comments/1vy80na/the_web_is_all_bots_now/",
        children: "The Web Is All Bots Now"
      }), ", which means as engineers we have had to pivot from the old way of thinking about who our pages are serving to this new normal that most of the requests are going to bots."]
    }), "\n", _jsxs(_components.p, {
      children: ["Even Konstantin Ryabitsev, Director of the Linux Foundation, has come out with a blog post ", _jsx(_components.a, {
        href: "https://people.kernel.org/monsieuricon/creepy-crawlies",
        children: "Creepy Crawlies"
      }), " that details just how much CPU bots are using hitting the git.kernel.org site, using the HTML version of the site instead of more efficient mechanisms:"]
    }), "\n", _jsxs(_components.blockquote, {
      children: ["\n", _jsx(_components.p, {
        children: "TL;DR: we spend more CPU cycles rendering commits for scrapers than we spend on all other kinds of legitimate access, including git clones. At any one time, across 5 geo-distributed nodes, there are 14 CPU cores doing nothing but rendering git commits as html."
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "At the end of the day, we have to decide from an engineering stand point what we do about this issue. We can rate-limit requests, we can block certain user-agents, and we can set-up robots.txt files with rules. But not all bots are playing by the rules. Some bots aren’t identifying themselves via unique user-agent strings. Some aren’t even looking at robots.txt."
    }), "\n", _jsx(_components.p, {
      children: "And by blocking some agents, our sites may not be cited by agents, leading to less traffic – which is already decreasing since users are asking agents instead of doing text searches now."
    }), "\n", _jsx(_components.p, {
      children: "The future is still foggy around SEO and AEO, but the current situation is clear: the majority of traffic on the internet is bots, and that is going to shape how we build and serve apps."
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, {
    ...props,
    children: _jsx(_createMdxContent, {
      ...props
    })
  }) : _createMdxContent(props);
}
return {
  default: MDXContent
};
function _missingMdxReference(id, component) {
  throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it.");
}


What happens to SEO when the crawler itself becomes one of your primary consumers?
We all intuitively know how SEO is supposed to work. Google and other search engines have been working for decades to make sure that search returns high quality content, rather than spam that stuffs the page with keywords. We would write meaningful, quality content, and Google with crawl the page, index the data, and humans would search and visit the website.
We built the modern web around the assumption that crawlers are lightweight readers of web pages. Increasingly, they are behaving more like automated users. With <a href="https://www.imperva.com/blog/bad-bot-report-2026-bots-agentic-age">53% percent of traffic</a> is bot traffic on the internet, how we write web applications, serve them, and the end goal of SEO may be changing.
<div class="flex w-full items-center justify-center gap-2 py-4"><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div></div>
When we launched the new version of Couchsurfing, we expected all sorts of issues. We anticipated a large spike in traffic when we launched, even though we weren’t planning on any immediate marketing push. But what we didn’t expect was to continue to see large spikes in traffic after we launched. These spikes wouldn’t just get a web page via a cURL GET request, but they would request all the different locales, all the image assets on the page, and all the other assets on the page. Some of the traffic we looked at would even attempt to create URLs that didn’t exist in our web application, hammering us with requests that we would 404. These weren’t from Googlebot. These were tons of other crawlers that were scraping our site for information - <a href="https://claude.com/">Claudebot</a>, <a href="https://meta.com/">meta</a>, other nefarious bots that would not return correct User Agent strings.
Why were AI crawlers hitting the application so differently from Google bot?
<div class="flex w-full items-center justify-center gap-2 py-4"><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div></div>
To understand why this was so surprising, we have to understand who the site was originally designed for.
When we started thinking about the next version of Couchsurfing, we decided that we would want to focus on “SSR” - Server Side Rendering” - as much of the page as possible. That is essentially how the existing application worked and we wanted to make sure the majority of the page wouldn’t have cascading loaders like Single Page Applications often have. We also knew wanted to move the application into React. The previous team had already tried implementing a few React components into the application, but it was slow progress.
With SSR and React as the baseline of what we wanted for the web application, we settled on NextJS as the framework. From there, every product kick-off would start with a few key questions: is this a public facing feature or page? And how much of it should be server-side rendered?
During the inception of this new project, there was quite a lot of discussion by Vercel, Next.js and web engineering at large about how much SSR was really required.
<a href="https://nextjs.org/learn/seo/webcrawlers">NextJS</a> and <a href="https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics">Google Search Central</a> have typically described the indexing process like this:
<img alt="Crawling Indexing Process" loading="lazy" width="1" height="1" decoding="async" data-nimg="1" class="mx-auto block !h-auto !w-auto max-h-[500px] max-w-full" style="color:transparent;width:auto;height:auto;max-width:100%;max-height:500px;margin:auto" src="/media/2026-09-08-are-crawlers-our-primary-consumers/crawlqueue.png"/>
Where the crawler will have items added to the render queue to actually execute the JS on the page. When we kicked off the project, we assumed that with the number of pages we were going to have to re-index with Google, that this step would take awhile to happen for our site. And when it did happen, we wouldn’t expect all our pages to be rendered in a reasonable timeframe. We estimated that our new multi-sitemap would have millions of pages, rather than the single 500,000 entries we had in the existing sitemap. We would have to worry about the crawl budget, so we settled on SSR’ing as much of the page as possible so that indexing could happen without the rendering step.
But Googlebot wasn’t the problem.
<div class="flex w-full items-center justify-center gap-2 py-4"><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div></div>
AI Crawlers hammer the application.
Crawling traffic. Thousands, hundreds of thousands of requests a day. We had anticipated Googlebot - the traditional bot that crawls pages to index them for Google search results. And that is the bot we had optimized SEO for. But we saw way more traffic than we expected from other crawl bots. We saw Meta’s bot crawling every single one of our user pages, location pages. We saw claude bot hit every single page in every single language we had localizes We saw weird scrapers that didn’t properly represent themselves try to hit thousand of user pages that required users to be logged in to see.
We were seeing lots of burst traffic. Claudebot for example would suddenly hit us with thousands of requests all at once. Our WAF had 429 rules to tell it to back-off, and we even had additional rate-limits on certain APIs that return 429 as well. This helped when the requests would come from the same IP address, but we would often see these large bursts come from clusters of IP addresses.
Pages would start responding slow due the insane influx of traffic, which wasn’t just to the page it was crawling, but to all the assets on the page, including images, JS files, and additional APIs that the page would normally request once hydrated on the client. This is the opposite of what we had expected - we expected crawling bots to simple do a GET to the page they were requesting and crawl the contents of that page. But crawling traffic for these LLM bots would actually evaluate the page like an end-user - some explicitly using tools like Puppeteer or Playwright according to the request headers.
The crawler was no longer simply requesting the server-side rendered version of the page. Instead the crawlers were acting like full users - hitting the page, requesting all the assets and image, and then following links on that page. Worse, some acted like malicious attacked - trying to enumerate pages that didn’t even exist to see if there was content on them.
<div class="flex w-full items-center justify-center gap-2 py-4"><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div><div class="!m-0 h-1 w-1 rounded-full bg-gray-300"></div></div>
Before AI, we would have considered these attacks. After AI, this is the new normal. As one Redditor describes, <a href="https://www.reddit.com/r/webdev/comments/1vy80na/the_web_is_all_bots_now/">The Web Is All Bots Now</a>, which means as engineers we have had to pivot from the old way of thinking about who our pages are serving to this new normal that most of the requests are going to bots.
Even Konstantin Ryabitsev, Director of the Linux Foundation, has come out with a blog post <a href="https://people.kernel.org/monsieuricon/creepy-crawlies">Creepy Crawlies</a> that details just how much CPU bots are using hitting the git.kernel.org site, using the HTML version of the site instead of more efficient mechanisms:
<blockquote class="border-l-4 border-gray-300 pl-4">
TL;DR: we spend more CPU cycles rendering commits for scrapers than we spend on all other kinds of legitimate access, including git clones. At any one time, across 5 geo-distributed nodes, there are 14 CPU cores doing nothing but rendering git commits as html.
</blockquote>
At the end of the day, we have to decide from an engineering stand point what we do about this issue. We can rate-limit requests, we can block certain user-agents, and we can set-up robots.txt files with rules. But not all bots are playing by the rules. Some bots aren’t identifying themselves via unique user-agent strings. Some aren’t even looking at robots.txt.
And by blocking some agents, our sites may not be cited by agents, leading to less traffic – which is already decreasing since users are asking agents instead of doing text searches now.
The future is still foggy around SEO and AEO, but the current situation is clear: the majority of traffic on the internet is bots, and that is going to shape how we build and serve apps.

What happens to SEO when the crawler itself becomes one of your primary consumers?

We all intuitively know how SEO is supposed to work. Google and other search engines have been working for decades to make sure that search returns high quality content, rather than spam that stuffs the page with keywords. We would write meaningful, quality content, and Google with crawl the page, index the data, and humans would search and visit the website.

We built the modern web around the assumption that crawlers are lightweight readers of web pages. Increasingly, they are behaving more like automated users. With [53% percent of traffic](https://www.imperva.com/blog/bad-bot-report-2026-bots-agentic-age) is bot traffic on the internet, how we write web applications, serve them, and the end goal of SEO may be changing.

<Separator />

When we launched the new version of Couchsurfing, we expected all sorts of issues. We anticipated a large spike in traffic when we launched, even though we weren’t planning on any immediate marketing push. But what we didn’t expect was to continue to see large spikes in traffic after we launched. These spikes wouldn’t just get a web page via a cURL GET request, but they would request all the different locales, all the image assets on the page, and all the other assets on the page. Some of the traffic we looked at would even attempt to create URLs that didn’t exist in our web application, hammering us with requests that we would 404. These weren’t from Googlebot. These were tons of other crawlers that were scraping our site for information - [Claudebot](https://claude.com/), [meta](https://meta.com/), other nefarious bots that would not return correct User Agent strings.

Why were AI crawlers hitting the application so differently from Google bot?

<Separator />

To understand why this was so surprising, we have to understand who the site was originally designed for. 

When we started thinking about the next version of Couchsurfing, we decided that we would want to focus on “SSR” - Server Side Rendering” - as much of the page as possible. That is essentially how the existing application worked and we wanted to make sure the majority of the page wouldn’t have cascading loaders like Single Page Applications often have. We also knew wanted to move the application into React. The previous team had already tried implementing a few React components into the application, but it was slow progress.

With SSR and React as the baseline of what we wanted for the web application, we settled on NextJS as the framework. From there, every product kick-off would start with a few key questions: is this a public facing feature or page? And how much of it should be server-side rendered?

During the inception of this new project, there was quite a lot of discussion by Vercel, Next.js and web engineering at large about how much SSR was really required. 

[NextJS](https://nextjs.org/learn/seo/webcrawlers) and [Google Search Central](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics) have typically described the indexing process like this:

![Crawling Indexing Process](/media/2026-09-08-are-crawlers-our-primary-consumers/crawlqueue.png)

Where the crawler will have items added to the render queue to actually execute the JS on the page. When we kicked off the project, we assumed that with the number of pages we were going to have to re-index with Google, that this step would take awhile to happen for our site. And when it did happen, we wouldn’t expect all our pages to be rendered in a reasonable timeframe. We estimated that our new multi-sitemap would have millions of pages, rather than the single 500,000 entries we had in the existing sitemap. We would have to worry about the crawl budget, so we settled on SSR’ing as much of the page as possible so that indexing could happen without the rendering step.

But Googlebot wasn’t the problem.

<Separator />

AI Crawlers hammer the application.

Crawling traffic. Thousands, hundreds of thousands of requests a day. We had anticipated Googlebot - the traditional bot that crawls pages to index them for Google search results. And that is the bot we had optimized SEO for. But we saw way more traffic than we expected from other crawl bots. We saw Meta’s bot crawling every single one of our user pages, location pages. We saw claude bot hit every single page in every single language we had localizes We saw weird scrapers that didn’t properly represent themselves try to hit thousand of user pages that required users to be logged in to see.

We were seeing lots of burst traffic. Claudebot for example would suddenly hit us with thousands of requests all at once. Our WAF had 429 rules to tell it to back-off, and we even had additional rate-limits on certain APIs that return 429 as well. This helped when the requests would come from the same IP address, but we would often see these large bursts come from clusters of IP addresses.

Pages would start responding slow due the insane influx of traffic, which wasn’t just to the page it was crawling, but to all the assets on the page, including images, JS files, and additional APIs that the page would normally request once hydrated on the client. This is the opposite of what we had expected - we expected crawling bots to simple do a GET to the page they were requesting and crawl the contents of that page. But crawling traffic for these LLM bots would actually evaluate the page like an end-user - some explicitly using tools like Puppeteer or Playwright according to the request headers.

The crawler was no longer simply requesting the server-side rendered version of the page. Instead the crawlers were acting like full users - hitting the page, requesting all the assets and image, and then following links on that page. Worse, some acted like malicious attacked - trying to enumerate pages that didn’t even exist to see if there was content on them.

<Separator />

Before AI, we would have considered these attacks. After AI, this is the new normal. As one Redditor describes, [The Web Is All Bots Now](https://www.reddit.com/r/webdev/comments/1vy80na/the_web_is_all_bots_now/), which means as engineers we have had to pivot from the old way of thinking about who our pages are serving to this new normal that most of the requests are going to bots.

Even Konstantin Ryabitsev, Director of the Linux Foundation, has come out with a blog post [Creepy Crawlies](https://people.kernel.org/monsieuricon/creepy-crawlies) that details just how much CPU bots are using hitting the git.kernel.org site, using the HTML version of the site instead of more efficient mechanisms:

> TL;DR: we spend more CPU cycles rendering commits for scrapers than we spend on all other kinds of legitimate access, including git clones. At any one time, across 5 geo-distributed nodes, there are 14 CPU cores doing nothing but rendering git commits as html.
> 

At the end of the day, we have to decide from an engineering stand point what we do about this issue. We can rate-limit requests, we can block certain user-agents, and we can set-up robots.txt files with rules. But not all bots are playing by the rules. Some bots aren’t identifying themselves via unique user-agent strings. Some aren’t even looking at robots.txt.

And by blocking some agents, our sites may not be cited by agents, leading to less traffic – which is already decreasing since users are asking agents instead of doing text searches now.

The future is still foggy around SEO and AEO, but the current situation is clear: the majority of traffic on the internet is bots, and that is going to shape how we build and serve apps.

Are Crawlers Our Primary Consumers?

"use strict";
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = {
    a: "a",
    img: "img",
    p: "p",
    ...props.components
  };
  return _jsxs(_Fragment, {
    children: [_jsxs(_components.p, {
      children: ["In 2008, Steve Ballmer once famously said “", _jsx(_components.a, {
        href: "https://www.youtube.com/watch?v=Vhh_GeBPOhs",
        children: "Developers, developers, developers"
      }), "” in a highly meme-able video. He was trying to emphasize the importance of software developers in business and that their importance was only going to continue to grow. Microsoft shifted to try to support software developers, noticing that if they supported developers, they would build better software for Windows, which in turn would make people want to use Windows."]
    }), "\n", _jsx(_components.p, {
      children: "The 2010s saw companies like “Twilio” have guiding questions like Jeff Lawson’s “Why Isn’t That and API?”. This guiding principle was based around the idea that if you build great APIs for software developers, they would want to use them, and in doing so, spend money on those services. For that entire decade, companies would start, build for engineers, and use the fly-wheel effect to become multi-billion dollar companies."
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2025-04-21-agents-agents-agents/why-isnt-that-an-api.png",
        alt: "Why isn't that an API?"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["Now, in the age of LLMs (Large Language Models) and AI, my assumption about the future is that we are going to find that “AI Agents” are the next Big Thing. We will probably see software engineers coding not to make APIs that other developers can use, but so that AI Agents can use. With ", _jsx(_components.a, {
        href: "https://openai.com/index/introducing-operator/",
        children: "OpenAI introducing Operator"
      }), ", their AI that can do tasks for you autonomously, the trajectory looks clear for companies that want to partner in this space: make products that will interface with AI Agents*."]
    }), "\n", _jsxs(_components.p, {
      children: ["And this isn’t going to only apply to software engineering. Even those in marketing are starting to think about how their content can be better consumed by the crawlers that feed into the training data for these LLMs. In “", _jsx(_components.a, {
        href: "https://benjamincongdon.me/blog/2025/01/25/AI-Slop-Suspicion-and-Writing-Back/",
        children: "AI Slop Suspicion and Writing Back"
      }), "”, the author hypothesizes about the future of SEO. Will we be writing not for other people to reach our content via search, but to see our content as part of an LLM’s answer to a question? If people are starting to transition to asking LLMs like ChatGPT a question first before they even Google, then wouldn’t getting your name and your thoughts ingested by the massive corpus of training data be beneficial in a sense, similar to a web-crawler indexing for search?"]
    }), "\n", _jsx(_components.p, {
      children: "These are questions I think we will see answered very soon, and some of the answers may not be ones we like. We may very well be starting to write content, apps, and APIs not for other people, but for machines."
    }), "\n", _jsx(_components.p, {
      children: "*Of course, not everyone is going to jump on this bandwagon. There are companies right now that make concerted efforts to make sure only humans use their site. They don’t have APIs, they stick their entire site behind a Cloudflare captcha check. These aren’t the companies I’m talking about as they weren’t trying to bridge into the larger software developer ecosystem."
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, {
    ...props,
    children: _jsx(_createMdxContent, {
      ...props
    })
  }) : _createMdxContent(props);
}
return {
  default: MDXContent
};


In 2008, Steve Ballmer once famously said “<a href="https://www.youtube.com/watch?v=Vhh_GeBPOhs">Developers, developers, developers</a>” in a highly meme-able video. He was trying to emphasize the importance of software developers in business and that their importance was only going to continue to grow. Microsoft shifted to try to support software developers, noticing that if they supported developers, they would build better software for Windows, which in turn would make people want to use Windows.
The 2010s saw companies like “Twilio” have guiding questions like Jeff Lawson’s “Why Isn’t That and API?”. This guiding principle was based around the idea that if you build great APIs for software developers, they would want to use them, and in doing so, spend money on those services. For that entire decade, companies would start, build for engineers, and use the fly-wheel effect to become multi-billion dollar companies.
<img alt="Why isn&#x27;t that an API?" loading="lazy" width="1" height="1" decoding="async" data-nimg="1" class="mx-auto block !h-auto !w-auto max-h-[500px] max-w-full" style="color:transparent;width:auto;height:auto;max-width:100%;max-height:500px;margin:auto" src="/media/2025-04-21-agents-agents-agents/why-isnt-that-an-api.png"/>
Now, in the age of LLMs (Large Language Models) and AI, my assumption about the future is that we are going to find that “AI Agents” are the next Big Thing. We will probably see software engineers coding not to make APIs that other developers can use, but so that AI Agents can use. With <a href="https://openai.com/index/introducing-operator/">OpenAI introducing Operator</a>, their AI that can do tasks for you autonomously, the trajectory looks clear for companies that want to partner in this space: make products that will interface with AI Agents*.
And this isn’t going to only apply to software engineering. Even those in marketing are starting to think about how their content can be better consumed by the crawlers that feed into the training data for these LLMs. In “<a href="https://benjamincongdon.me/blog/2025/01/25/AI-Slop-Suspicion-and-Writing-Back/">AI Slop Suspicion and Writing Back</a>”, the author hypothesizes about the future of SEO. Will we be writing not for other people to reach our content via search, but to see our content as part of an LLM’s answer to a question? If people are starting to transition to asking LLMs like ChatGPT a question first before they even Google, then wouldn’t getting your name and your thoughts ingested by the massive corpus of training data be beneficial in a sense, similar to a web-crawler indexing for search?
These are questions I think we will see answered very soon, and some of the answers may not be ones we like. We may very well be starting to write content, apps, and APIs not for other people, but for machines.
*Of course, not everyone is going to jump on this bandwagon. There are companies right now that make concerted efforts to make sure only humans use their site. They don’t have APIs, they stick their entire site behind a Cloudflare captcha check. These aren’t the companies I’m talking about as they weren’t trying to bridge into the larger software developer ecosystem.


In 2008, Steve Ballmer once famously said “[Developers, developers, developers](https://www.youtube.com/watch?v=Vhh_GeBPOhs)” in a highly meme-able video. He was trying to emphasize the importance of software developers in business and that their importance was only going to continue to grow. Microsoft shifted to try to support software developers, noticing that if they supported developers, they would build better software for Windows, which in turn would make people want to use Windows.

The 2010s saw companies like “Twilio” have guiding questions like Jeff Lawson’s “Why Isn’t That and API?”. This guiding principle was based around the idea that if you build great APIs for software developers, they would want to use them, and in doing so, spend money on those services. For that entire decade, companies would start, build for engineers, and use the fly-wheel effect to become multi-billion dollar companies.

![Why isn't that an API?](/media/2025-04-21-agents-agents-agents/why-isnt-that-an-api.png)

Now, in the age of LLMs (Large Language Models) and AI, my assumption about the future is that we are going to find that “AI Agents” are the next Big Thing. We will probably see software engineers coding not to make APIs that other developers can use, but so that AI Agents can use. With [OpenAI introducing Operator](https://openai.com/index/introducing-operator/), their AI that can do tasks for you autonomously, the trajectory looks clear for companies that want to partner in this space: make products that will interface with AI Agents*.

And this isn’t going to only apply to software engineering. Even those in marketing are starting to think about how their content can be better consumed by the crawlers that feed into the training data for these LLMs. In “[AI Slop Suspicion and Writing Back](https://benjamincongdon.me/blog/2025/01/25/AI-Slop-Suspicion-and-Writing-Back/)”, the author hypothesizes about the future of SEO. Will we be writing not for other people to reach our content via search, but to see our content as part of an LLM’s answer to a question? If people are starting to transition to asking LLMs like ChatGPT a question first before they even Google, then wouldn’t getting your name and your thoughts ingested by the massive corpus of training data be beneficial in a sense, similar to a web-crawler indexing for search?

These are questions I think we will see answered very soon, and some of the answers may not be ones we like. We may very well be starting to write content, apps, and APIs not for other people, but for machines.

*Of course, not everyone is going to jump on this bandwagon. There are companies right now that make concerted efforts to make sure only humans use their site. They don’t have APIs, they stick their entire site behind a Cloudflare captcha check. These aren’t the companies I’m talking about as they weren’t trying to bridge into the larger software developer ecosystem.


Agents, Agents, Agents

"use strict";
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = {
    a: "a",
    blockquote: "blockquote",
    h2: "h2",
    img: "img",
    li: "li",
    p: "p",
    ul: "ul",
    ...props.components
  };
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "I’ve been experimenting with ChatGPT, just like everyone else. But why has it become so popular? It truly is a revolutionary piece of technology. Is it The Next Big Thing? Will it really replace all of us?"
    }), "\n", _jsx(_components.p, {
      children: "My day to day is architecture and programming, and I’ve heard all sorts of things on Twitter:"
    }), "\n", _jsxs(_components.ul, {
      children: ["\n", _jsx(_components.li, {
        children: "That ChatGPT can build entire iOS applications"
      }), "\n", _jsx(_components.li, {
        children: "ChatGPT can write whole files of code"
      }), "\n", _jsx(_components.li, {
        children: "ChatGPT can find bugs in code when writing tests"
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "When I started by own company Clarity Hub, we had a focus on machine learning to help augment human intelligence. My thought at the time was that we could leverage machine learning to augment and supplement human intelligence. The focus wouldn't be to replace any person's job, but to make it streamlined and easier to accomplish that job. Our journey started with us serving customer success agents with suggested replies and contextual information, but we eventually pivoted to a Dovetail-like application to help product teams gather, annotate, and contextualize customer interviews."
    }), "\n", _jsx(_components.p, {
      children: "We found that augmenting activity with machine learning was not only easier to do from a technology point of view, but empowered users, rather than make them feel like their job was being replaced."
    }), "\n", _jsx(_components.p, {
      children: "Even with the advent of ChatGPT, I still see this being the short term future of it's use."
    }), "\n", _jsx(_components.h2, {
      children: "False Starts"
    }), "\n", _jsx(_components.p, {
      children: "I was curious how much ChatGPT could really do given all of its hype"
    }), "\n", _jsx(_components.p, {
      children: "I asked it to build an Asteroids game using PhaserJS. Don’t worry, this isn’t another article on “Look, I got ChatGPT to build a game!”"
    }), "\n", _jsx(_components.p, {
      children: "I originally just asked ChatGPT to build an Asteroids game using PhaserJS. A very vague prompt, yes, but a great starting point to figure out the limitations of ChatGPT."
    }), "\n", _jsx(_components.p, {
      children: "The response was basically, “No I can’t”:"
    }), "\n", _jsxs(_components.blockquote, {
      children: ["\n", _jsx(_components.p, {
        children: "As an AI language model, I'm not able to write and execute code in real-time. However, I can provide you with a detailed framework to get started with creating an Asteroids game in PhaserJS."
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "This was followed by instructions on how I could do it myself."
    }), "\n", _jsx(_components.p, {
      children: "The next steps of course are to be more specific, asking ChatGPT to build the application to build specific functionality:"
    }), "\n", _jsxs(_components.blockquote, {
      children: ["\n", _jsx(_components.p, {
        children: "Add the code to let the player shoot asteroids. If the player gets hit by 3 asteroids, its game over"
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "Most interactions with ChatGPT seem to go like this:"
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2023-07-23-chatgpt-coding-loop/Flowchart_Template_1.jpg",
        alt: "Flowchart workflow"
      })
    }), "\n", _jsx(_components.p, {
      children: "This flow eventually leads to either:"
    }), "\n", _jsxs(_components.ul, {
      children: ["\n", _jsx(_components.li, {
        children: "Not getting what you want."
      }), "\n", _jsx(_components.li, {
        children: "Getting what you want across many different responses, and then you must manually coalesce all of the different responses together to form the result you want."
      }), "\n"]
    }), "\n", _jsx(_components.p, {
      children: "Once I like the set of responses that ChatGPT has given me, I’ll ask ChatGPT to summarize for me so that I can improve the prompt further:"
    }), "\n", _jsxs(_components.blockquote, {
      children: ["\n", _jsx(_components.p, {
        children: "Take everything we just talked about and give me a prompt to ask you in the future that encapsulates all of the requirements."
      }), "\n"]
    }), "\n", _jsx(_components.h2, {
      children: "A More Natural Flow"
    }), "\n", _jsxs(_components.p, {
      children: ["I’ve talked with ChatGPT on a variety of topics, but the majority of questions relate to coding. I actively use Github Copilot, which utilizes the ChatGPT engine in the backend. I’ve built ", _jsx(_components.a, {
        href: "https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat",
        children: "Sora by Capsule Cat"
      }), " to have ChatGPT generate entire coding files for me as well with project context. Even then, I still use the ChatGPT UI as well."]
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2023-07-23-chatgpt-coding-loop/Flowchart_Template_2.jpg",
        alt: "Flowchart line"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["I find that the above flow leaves me less fatigued, and more productive - at least in the context of side-projects that I’ve used ChatGPT. Research shows that when we read code and attempt logic puzzles, we use a completely different part of our brain than when we use language [", _jsx(_components.a, {
        href: "https://hub.jhu.edu/2020/12/17/brain-activity-while-reading-code/",
        children: "link"
      }), "]. In the above chart, you can imagine that when we write prompts, we utilize some logical reasoning, but we lean heavily on language as we talk with ChatGPT. Then when we get a response, a completely different part of the brain kicks in to read and write code."]
    }), "\n", _jsx(_components.h2, {
      children: "The Hard Part About Coding"
    }), "\n", _jsx(_components.p, {
      children: "Eventually this leads to “the hard part about coding” which anecdotally everyone will tell you that surprise, it isn’t the coding part. What is nice about ChatGPT though is that even though the hard part isn’t writing code, it does take time, thought, and energy."
    }), "\n", _jsx(_components.p, {
      children: "If the hard part of coding isn’t coding, then what is it? The non-exhaustive list is that it’s teamwork, communication, debugging, and requirements gathering. Oversimplifying, it’s the writing prompts part of the chart."
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, {
    ...props,
    children: _jsx(_createMdxContent, {
      ...props
    })
  }) : _createMdxContent(props);
}
return {
  default: MDXContent
};


I’ve been experimenting with ChatGPT, just like everyone else. But why has it become so popular? It truly is a revolutionary piece of technology. Is it The Next Big Thing? Will it really replace all of us?
My day to day is architecture and programming, and I’ve heard all sorts of things on Twitter:
<ul>
<li>That ChatGPT can build entire iOS applications</li>
<li>ChatGPT can write whole files of code</li>
<li>ChatGPT can find bugs in code when writing tests</li>
</ul>
When I started by own company Clarity Hub, we had a focus on machine learning to help augment human intelligence. My thought at the time was that we could leverage machine learning to augment and supplement human intelligence. The focus wouldn&#x27;t be to replace any person&#x27;s job, but to make it streamlined and easier to accomplish that job. Our journey started with us serving customer success agents with suggested replies and contextual information, but we eventually pivoted to a Dovetail-like application to help product teams gather, annotate, and contextualize customer interviews.
We found that augmenting activity with machine learning was not only easier to do from a technology point of view, but empowered users, rather than make them feel like their job was being replaced.
Even with the advent of ChatGPT, I still see this being the short term future of it&#x27;s use.
<h2>False Starts</h2>
I was curious how much ChatGPT could really do given all of its hype
I asked it to build an Asteroids game using PhaserJS. Don’t worry, this isn’t another article on “Look, I got ChatGPT to build a game!”
I originally just asked ChatGPT to build an Asteroids game using PhaserJS. A very vague prompt, yes, but a great starting point to figure out the limitations of ChatGPT.
The response was basically, “No I can’t”:
<blockquote class="border-l-4 border-gray-300 pl-4">
As an AI language model, I&#x27;m not able to write and execute code in real-time. However, I can provide you with a detailed framework to get started with creating an Asteroids game in PhaserJS.
</blockquote>
This was followed by instructions on how I could do it myself.
The next steps of course are to be more specific, asking ChatGPT to build the application to build specific functionality:
<blockquote class="border-l-4 border-gray-300 pl-4">
Add the code to let the player shoot asteroids. If the player gets hit by 3 asteroids, its game over
</blockquote>
Most interactions with ChatGPT seem to go like this:
<img alt="Flowchart workflow" loading="lazy" width="1" height="1" decoding="async" data-nimg="1" class="mx-auto block !h-auto !w-auto max-h-[500px] max-w-full" style="color:transparent;width:auto;height:auto;max-width:100%;max-height:500px;margin:auto" src="/media/2023-07-23-chatgpt-coding-loop/Flowchart_Template_1.jpg"/>
This flow eventually leads to either:
<ul>
<li>Not getting what you want.</li>
<li>Getting what you want across many different responses, and then you must manually coalesce all of the different responses together to form the result you want.</li>
</ul>
Once I like the set of responses that ChatGPT has given me, I’ll ask ChatGPT to summarize for me so that I can improve the prompt further:
<blockquote class="border-l-4 border-gray-300 pl-4">
Take everything we just talked about and give me a prompt to ask you in the future that encapsulates all of the requirements.
</blockquote>
<h2>A More Natural Flow</h2>
I’ve talked with ChatGPT on a variety of topics, but the majority of questions relate to coding. I actively use Github Copilot, which utilizes the ChatGPT engine in the backend. I’ve built <a href="https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat">Sora by Capsule Cat</a> to have ChatGPT generate entire coding files for me as well with project context. Even then, I still use the ChatGPT UI as well.
<img alt="Flowchart line" loading="lazy" width="1" height="1" decoding="async" data-nimg="1" class="mx-auto block !h-auto !w-auto max-h-[500px] max-w-full" style="color:transparent;width:auto;height:auto;max-width:100%;max-height:500px;margin:auto" src="/media/2023-07-23-chatgpt-coding-loop/Flowchart_Template_2.jpg"/>
I find that the above flow leaves me less fatigued, and more productive - at least in the context of side-projects that I’ve used ChatGPT. Research shows that when we read code and attempt logic puzzles, we use a completely different part of our brain than when we use language [<a href="https://hub.jhu.edu/2020/12/17/brain-activity-while-reading-code/">link</a>]. In the above chart, you can imagine that when we write prompts, we utilize some logical reasoning, but we lean heavily on language as we talk with ChatGPT. Then when we get a response, a completely different part of the brain kicks in to read and write code.
<h2>The Hard Part About Coding</h2>
Eventually this leads to “the hard part about coding” which anecdotally everyone will tell you that surprise, it isn’t the coding part. What is nice about ChatGPT though is that even though the hard part isn’t writing code, it does take time, thought, and energy.
If the hard part of coding isn’t coding, then what is it? The non-exhaustive list is that it’s teamwork, communication, debugging, and requirements gathering. Oversimplifying, it’s the writing prompts part of the chart.


I’ve been experimenting with ChatGPT, just like everyone else. But why has it become so popular? It truly is a revolutionary piece of technology. Is it The Next Big Thing? Will it really replace all of us?

My day to day is architecture and programming, and I’ve heard all sorts of things on Twitter:

- That ChatGPT can build entire iOS applications
- ChatGPT can write whole files of code
- ChatGPT can find bugs in code when writing tests

When I started by own company Clarity Hub, we had a focus on machine learning to help augment human intelligence. My thought at the time was that we could leverage machine learning to augment and supplement human intelligence. The focus wouldn't be to replace any person's job, but to make it streamlined and easier to accomplish that job. Our journey started with us serving customer success agents with suggested replies and contextual information, but we eventually pivoted to a Dovetail-like application to help product teams gather, annotate, and contextualize customer interviews.

We found that augmenting activity with machine learning was not only easier to do from a technology point of view, but empowered users, rather than make them feel like their job was being replaced.

Even with the advent of ChatGPT, I still see this being the short term future of it's use.

## False Starts

I was curious how much ChatGPT could really do given all of its hype  

I asked it to build an Asteroids game using PhaserJS. Don’t worry, this isn’t another article on “Look, I got ChatGPT to build a game!”

I originally just asked ChatGPT to build an Asteroids game using PhaserJS. A very vague prompt, yes, but a great starting point to figure out the limitations of ChatGPT.

The response was basically, “No I can’t”:

> As an AI language model, I'm not able to write and execute code in real-time. However, I can provide you with a detailed framework to get started with creating an Asteroids game in PhaserJS.

This was followed by instructions on how I could do it myself.

The next steps of course are to be more specific, asking ChatGPT to build the application to build specific functionality:

> Add the code to let the player shoot asteroids. If the player gets hit by 3 asteroids, its game over

Most interactions with ChatGPT seem to go like this:

![Flowchart workflow](/media/2023-07-23-chatgpt-coding-loop/Flowchart_Template_1.jpg)

This flow eventually leads to either:

- Not getting what you want.
- Getting what you want across many different responses, and then you must manually coalesce all of the different responses together to form the result you want.

Once I like the set of responses that ChatGPT has given me, I’ll ask ChatGPT to summarize for me so that I can improve the prompt further:

> Take everything we just talked about and give me a prompt to ask you in the future that encapsulates all of the requirements.

## A More Natural Flow

I’ve talked with ChatGPT on a variety of topics, but the majority of questions relate to coding. I actively use Github Copilot, which utilizes the ChatGPT engine in the backend. I’ve built [Sora by Capsule Cat](https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat) to have ChatGPT generate entire coding files for me as well with project context. Even then, I still use the ChatGPT UI as well.

![Flowchart line](/media/2023-07-23-chatgpt-coding-loop/Flowchart_Template_2.jpg)

I find that the above flow leaves me less fatigued, and more productive - at least in the context of side-projects that I’ve used ChatGPT. Research shows that when we read code and attempt logic puzzles, we use a completely different part of our brain than when we use language [[link](https://hub.jhu.edu/2020/12/17/brain-activity-while-reading-code/)]. In the above chart, you can imagine that when we write prompts, we utilize some logical reasoning, but we lean heavily on language as we talk with ChatGPT. Then when we get a response, a completely different part of the brain kicks in to read and write code.

## The Hard Part About Coding

Eventually this leads to “the hard part about coding” which anecdotally everyone will tell you that surprise, it isn’t the coding part. What is nice about ChatGPT though is that even though the hard part isn’t writing code, it does take time, thought, and energy.

If the hard part of coding isn’t coding, then what is it? The non-exhaustive list is that it’s teamwork, communication, debugging, and requirements gathering. Oversimplifying, it’s the writing prompts part of the chart.


ChatGPT, Coding, and Language

"use strict";
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = {
    mermaid: "mermaid",
    p: "p",
    ...props.components
  };
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "We are starting to see a rise of novel use-cases for AI in products and games using LLMs. Rather than the simple chatbot like experiences we have seen in the past using AI, we are starting to see feedback systems being added to these experiences, providing additional context to the LLM than just the past conversation."
    }), "\n", _jsx(_components.p, {
      children: "A typical game loop for this type of system would look like:"
    }), "\n", _jsx(_components.mermaid, {
      chart: "graph LR\n  Rules --> LLM\n  InputStates[\"Input States\"] --> LLM\n  LLM --> OutputState\n  OutputState[\"Output State\"] --> GameEngine\n  GameEngine[\"Game Engine\"] --> InputStates"
    }), "\n", _jsx(_components.p, {
      children: "Rules can be describes as written-word description, with an additional set of rules telling the LLM to reply using JSON output of a given schema. In this area, I have had success giving LLMs descriptions of output schemas in Typescript and asking for a JSON response that adheres to the type. Other methods of getting a consistent schema are more than likely possible here, as well as additional output methods."
    }), "\n", _jsx(_components.p, {
      children: "When the asynchronous task of creating and output state is complete, the Game Engine in this case can read, parse, and apply that new state to the world. Any additional interaction would then lead to the next set of input states that can be given to the LLM as a JSON blob."
    }), "\n", _jsx(_components.p, {
      children: "For a more concrete example, we can imagine a game where we want our player to interact with a set of agents. The input states would be the state of each agent, the user’s interaction, and maybe some global environment data. The rules may be how each agent should behave, the rules of the game, and additional context. The LLM would take these inputs, and the output is instructed to be the next state of each agent. When the LLM returns this data, the Game Engine read it and applies it to the game’s representation of each agent, showing the player the impact of their actions."
    }), "\n", _jsx(_components.p, {
      children: "I’m looking forward to more novel use-cases for LLMs!"
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, {
    ...props,
    children: _jsx(_createMdxContent, {
      ...props
    })
  }) : _createMdxContent(props);
}
return {
  default: MDXContent
};


We are starting to see a rise of novel use-cases for AI in products and games using LLMs. Rather than the simple chatbot like experiences we have seen in the past using AI, we are starting to see feedback systems being added to these experiences, providing additional context to the LLM than just the past conversation.
A typical game loop for this type of system would look like:
<mermaid chart="graph LR
 Rules --&gt; LLM
 InputStates[&quot;Input States&quot;] --&gt; LLM
 LLM --&gt; OutputState
 OutputState[&quot;Output State&quot;] --&gt; GameEngine
 GameEngine[&quot;Game Engine&quot;] --&gt; InputStates"></mermaid>
Rules can be describes as written-word description, with an additional set of rules telling the LLM to reply using JSON output of a given schema. In this area, I have had success giving LLMs descriptions of output schemas in Typescript and asking for a JSON response that adheres to the type. Other methods of getting a consistent schema are more than likely possible here, as well as additional output methods.
When the asynchronous task of creating and output state is complete, the Game Engine in this case can read, parse, and apply that new state to the world. Any additional interaction would then lead to the next set of input states that can be given to the LLM as a JSON blob.
For a more concrete example, we can imagine a game where we want our player to interact with a set of agents. The input states would be the state of each agent, the user’s interaction, and maybe some global environment data. The rules may be how each agent should behave, the rules of the game, and additional context. The LLM would take these inputs, and the output is instructed to be the next state of each agent. When the LLM returns this data, the Game Engine read it and applies it to the game’s representation of each agent, showing the player the impact of their actions.
I’m looking forward to more novel use-cases for LLMs!


We are starting to see a rise of novel use-cases for AI in products and games using LLMs. Rather than the simple chatbot like experiences we have seen in the past using AI, we are starting to see feedback systems being added to these experiences, providing additional context to the LLM than just the past conversation.

A typical game loop for this type of system would look like:

```mermaid
graph LR
  Rules --> LLM
  InputStates["Input States"] --> LLM
  LLM --> OutputState
  OutputState["Output State"] --> GameEngine
  GameEngine["Game Engine"] --> InputStates
```

Rules can be describes as written-word description, with an additional set of rules telling the LLM to reply using JSON output of a given schema. In this area, I have had success giving LLMs descriptions of output schemas in Typescript and asking for a JSON response that adheres to the type. Other methods of getting a consistent schema are more than likely possible here, as well as additional output methods.

When the asynchronous task of creating and output state is complete, the Game Engine in this case can read, parse, and apply that new state to the world. Any additional interaction would then lead to the next set of input states that can be given to the LLM as a JSON blob.

For a more concrete example, we can imagine a game where we want our player to interact with a set of agents. The input states would be the state of each agent, the user’s interaction, and maybe some global environment data. The rules may be how each agent should behave, the rules of the game, and additional context. The LLM would take these inputs, and the output is instructed to be the next state of each agent. When the LLM returns this data, the Game Engine read it and applies it to the game’s representation of each agent, showing the player the impact of their actions.

I’m looking forward to more novel use-cases for LLMs!


AI Feedback Systems

"use strict";
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = {
    a: "a",
    code: "code",
    h2: "h2",
    img: "img",
    p: "p",
    pre: "pre",
    strong: "strong",
    ...props.components
  };
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "Github Copilot and other AI tools are hitting the scene. I decided to create my own Visual Studio Code extension, which is designed to use OpenAI’s APIs to bring some additional ChatGPT functionality into the code editor. The goal with Sora was to enable a developer to thoughtfully write a comment about the code they would like the AI to write, and then commit to it – rather than the real-time typeahead that Github Copilot provides."
    }), "\n", _jsxs(_components.p, {
      children: ["With that goal in mind, Sora provides two ways to activate OpenAI: by typing ", _jsx(_components.code, {
        children: "@OpenAI"
      }), " (formally ", _jsx(_components.code, {
        children: "@ChatGPT"
      }), ") or clicking “Send to OpenAI” when hovering over a comment. The other improvement is that Sora will read any relative link references to files in your project. A great way to use this is to have OpenAI write code in the style that already exists in your project, for example:"]
    }), "\n", _jsx(_components.pre, {
      children: _jsx(_components.code, {
        className: "language-tsx",
        children: "/**\n * Write tests for [my-file](./my-file.ts] using\n * [other-test](../something/other-file.test.ts) as an example\n */\n"
      })
    }), "\n", _jsx(_components.p, {
      children: "Using this extension lets users write specifications as comments, and have ChatGPT write the entire file for you."
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2023-06-06-sora/sora-preview.gif",
        alt: "Sora Preview"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["This extension leverages the OpenAI API to send any referenced files and a starting prompt to the ", _jsx(_components.code, {
        children: "gpt-3.5-turbo"
      }), " chat completion endpoint. The prompt mainly sets the context for creating working code using a given language and reference files."]
    }), "\n", _jsx(_components.p, {
      children: "Once the response comes back, the extension parses it and appends it to the original file."
    }), "\n", _jsx(_components.h2, {
      children: _jsx(_components.strong, {
        children: "Installation and Usage"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["You can install the extension by going to ", _jsx(_components.a, {
        href: "https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat",
        children: "the VSCode Marketplace"
      }), ", or searching for “Sora” in Visual Studio Code extensions."]
    }), "\n", _jsxs(_components.p, {
      children: ["Once installed, you will need to enter your own OpenAI API key. You can get your key by following ", _jsx(_components.a, {
        href: "https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key",
        children: "these instructions"
      }), ". Then just enter ", _jsx(_components.code, {
        children: "Sora: Set API Key"
      }), " into the Visual Studio Code command prompt."]
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2023-06-06-sora/sora-set-api-key.png",
        alt: "Sora Set API Key"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["You can review the code ", _jsx(_components.a, {
        href: "https://github.com/CapsuleCat/sora-by-capsule-cat",
        children: "on Github"
      }), "."]
    }), "\n", _jsx(_components.h2, {
      children: _jsx(_components.strong, {
        children: "Conclusion"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["It was fun building my first extension. Please feel free to reach out on our ", _jsx(_components.a, {
        href: "https://github.com/CapsuleCat/sora-by-capsule-cat",
        children: "Github"
      }), " for feedback or questions."]
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, {
    ...props,
    children: _jsx(_createMdxContent, {
      ...props
    })
  }) : _createMdxContent(props);
}
return {
  default: MDXContent
};


Github Copilot and other AI tools are hitting the scene. I decided to create my own Visual Studio Code extension, which is designed to use OpenAI’s APIs to bring some additional ChatGPT functionality into the code editor. The goal with Sora was to enable a developer to thoughtfully write a comment about the code they would like the AI to write, and then commit to it – rather than the real-time typeahead that Github Copilot provides.
With that goal in mind, Sora provides two ways to activate OpenAI: by typing <code>@OpenAI</code> (formally <code>@ChatGPT</code>) or clicking “Send to OpenAI” when hovering over a comment. The other improvement is that Sora will read any relative link references to files in your project. A great way to use this is to have OpenAI write code in the style that already exists in your project, for example:
<div class="overflow-auto rounded-2xl bg-[#0e005d6d] p-4 font-mono text-sm dark:bg-[#0e005d6d] dark:text-gray-100"><pre><code class="language-tsx">/**
 * Write tests for [my-file](./my-file.ts] using
 * [other-test](../something/other-file.test.ts) as an example
 */
</code></pre></div>
Using this extension lets users write specifications as comments, and have ChatGPT write the entire file for you.
<img alt="Sora Preview" loading="lazy" width="1" height="1" decoding="async" data-nimg="1" class="mx-auto block !h-auto !w-auto max-h-[500px] max-w-full" style="color:transparent;width:auto;height:auto;max-width:100%;max-height:500px;margin:auto" src="/media/2023-06-06-sora/sora-preview.gif"/>
This extension leverages the OpenAI API to send any referenced files and a starting prompt to the <code>gpt-3.5-turbo</code> chat completion endpoint. The prompt mainly sets the context for creating working code using a given language and reference files.
Once the response comes back, the extension parses it and appends it to the original file.
<h2>Installation and Usage</h2>
You can install the extension by going to <a href="https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat">the VSCode Marketplace</a>, or searching for “Sora” in Visual Studio Code extensions.
Once installed, you will need to enter your own OpenAI API key. You can get your key by following <a href="https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key">these instructions</a>. Then just enter <code>Sora: Set API Key</code> into the Visual Studio Code command prompt.
<img alt="Sora Set API Key" loading="lazy" width="1" height="1" decoding="async" data-nimg="1" class="mx-auto block !h-auto !w-auto max-h-[500px] max-w-full" style="color:transparent;width:auto;height:auto;max-width:100%;max-height:500px;margin:auto" src="/media/2023-06-06-sora/sora-set-api-key.png"/>
You can review the code <a href="https://github.com/CapsuleCat/sora-by-capsule-cat">on Github</a>.
<h2>Conclusion</h2>
It was fun building my first extension. Please feel free to reach out on our <a href="https://github.com/CapsuleCat/sora-by-capsule-cat">Github</a> for feedback or questions.

Github Copilot and other AI tools are hitting the scene. I decided to create my own Visual Studio Code extension, which is designed to use OpenAI’s APIs to bring some additional ChatGPT functionality into the code editor. The goal with Sora was to enable a developer to thoughtfully write a comment about the code they would like the AI to write, and then commit to it – rather than the real-time typeahead that Github Copilot provides.

With that goal in mind, Sora provides two ways to activate OpenAI: by typing `@OpenAI` (formally `@ChatGPT`) or clicking “Send to OpenAI” when hovering over a comment. The other improvement is that Sora will read any relative link references to files in your project. A great way to use this is to have OpenAI write code in the style that already exists in your project, for example:

```tsx
/**
 * Write tests for [my-file](./my-file.ts] using
 * [other-test](../something/other-file.test.ts) as an example
 */
```

Using this extension lets users write specifications as comments, and have ChatGPT write the entire file for you.

![Sora Preview](/media/2023-06-06-sora/sora-preview.gif)

This extension leverages the OpenAI API to send any referenced files and a starting prompt to the `gpt-3.5-turbo` chat completion endpoint. The prompt mainly sets the context for creating working code using a given language and reference files.

Once the response comes back, the extension parses it and appends it to the original file.

## **Installation and Usage**

You can install the extension by going to [the VSCode Marketplace](https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat), or searching for “Sora” in Visual Studio Code extensions.

Once installed, you will need to enter your own OpenAI API key. You can get your key by following [these instructions](https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key). Then just enter `Sora: Set API Key` into the Visual Studio Code command prompt.

![Sora Set API Key](/media/2023-06-06-sora/sora-set-api-key.png)

You can review the code [on Github](https://github.com/CapsuleCat/sora-by-capsule-cat).

## **Conclusion**

It was fun building my first extension. Please feel free to reach out on our [Github](https://github.com/CapsuleCat/sora-by-capsule-cat) for feedback or questions.


Sora - OpenAI Visual Studio Code Extension

"use strict";
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
 const _components = {
 a: "a",
 code: "code",
 em: "em",
 hr: "hr",
 p: "p",
 pre: "pre",
 ...props.components
 };
 return _jsxs(_Fragment, {
 children: [_jsxs(_components.p, {
 children: ["Looks like everyone is trying to discover ChatGPT’s limitations. In one article I read, the author asked whether ", _jsx(_components.a, {
 href: "https://medium.com/@theworldaccordingtocgpt/chatgpt-can-play-20-questions-7911405f7aff",
 children: "ChatGPT can play 20 questions"
 }), ". The human in this scenario tried to play 20 questions with ChatGPT, and found that they needed to guide the AI to ask questions. From my perspective, it looked more like a failure to engineer the initial prompt correctly."]
 }), "\n", _jsx(_components.p, {
 children: "I attempted to do 20 questions with ChatGPT using a different prompt:"
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: I would like to play a game. It's called twenty questions. You will ask me one question at a time and I will answer it. The goal is for you guess the thing I am thinking about."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Sure, I'd be happy to play \"Twenty Questions\" with you. Please go ahead and think of something, and I will start asking my first question."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: I am ready"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Great, here is my first question: Is the thing you are thinking of an object?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Yes"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object something that you can hold in your hand?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Yes"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object something that you would typically find in a kitchen?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: No"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object something that you would use for transportation?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: No"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object something that you would use for entertainment or leisure?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Yes"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object a musical instrument?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: No"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object a type of game or puzzle?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: No"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object a piece of technology such as a phone or computer?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: No"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object a book or magazine?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Yes"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object a novel or a collection of short stories?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Yes"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object a specific book that is widely known and read?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Yes"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object \"To Kill a Mockingbird\" by Harper Lee?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: No"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: I'm sorry, I'm running out of ideas. Can you give me a hint or tell me the answer?"
 })
 }), "\n", _jsx(_components.p, {
 children: "Seems like we hit a wall here with twenty questions, but ChatGPT is very close, so I decided to give a hint."
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: You are on the right track. It is by Walt Whitman"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Is the object \"Leaves of Grass\" by Walt Whitman?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Yes! Congratulations!"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Thank you! I'm glad I was able to guess the object you were thinking of. It was fun playing \"Twenty Questions\" with you. Let me know if you want to play again sometime."
 })
 }), "\n", _jsx(_components.hr, {}), "\n", _jsxs(_components.p, {
 children: ["I’ve been keeping an eye on how other people are using ChatGPT, and ", _jsx(_components.a, {
 href: "https://en.wikipedia.org/wiki/Prompt_engineering",
 children: "prompt engineering"
 }), " is one of the more fascinating aspects around how people are communicating with the AI. Prompt engineering plays a key role in communicating with models like ChatGPT. Carefully crafting prompts can help maximize the quality of the reponse. However, it is important to keep in mind that, like any AI model, ChatGPT has its own set of limitations."]
 }), "\n", _jsxs(_components.p, {
 children: ["So I was curious when I read that “", _jsx(_components.a, {
 href: "https://www.reddit.com/r/ChatGPT/comments/101e454/chatgpt_cant_write_sentences_that_end_with_a/",
 children: "ChatGPT can’t write sentences that end in a specific letter"
 }), "”. Is this a real limitation of ChatGPT, or maybe this was just a poor prompt."]
 }), "\n", _jsx(_components.p, {
 children: "So let’s start with a very simple question to ChatGPT:"
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Can you write a sentence that ends with the letter \"a\"?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: The cat sat on the mat."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: That sentence does not end in the letter \"a\". Can you write a sentence that ends in the letter \"a\"?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: I can't think of anything right now, but the word \"America\" ends with letter \"a\"."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Great, write a sentence that ends in that word. That will then satisfy the original question."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: The land of the free, and the home of the brave, America."
 })
 }), "\n", _jsx(_components.p, {
 children: "Okay, that does technically end in the letter a, but the sentence seems very strange."
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: That isn't a very coherent sentence. Can you write a sentence that ends in the letter \"a\"?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: I am sorry for the confusion, here is a sentence that ends with the letter \"a\" - \"He saw the drama unfold on the stage, it was quite the spectacle, a.”"
 })
 }), "\n", _jsx(_components.p, {
 children: "Looks like we are going off the rails here. Let’s try a different approach:"
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Write a sentence that ends in a word that ends with the letter \"a\"."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: The sun was setting over the horizon, painting the sky with shades of orange and red, a spectacle to behold and end the day with."
 })
 }), "\n", _jsx(_components.p, {
 children: "I’m going to ask ChatGPT to tell me a prompt to feed back to itself. I’ve found this useful for resetting context when having long conversations. Or if ChatGPT gets into weird states like the above response."
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Can you tell me how to write a prompt for you that would actually get you to write a sentence that ends in the letter \"a\"."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Sure, to get me to write a sentence that ends in the letter \"a\" you could provide a prompt like: \"Write a sentence that ends with a word that ends with the letter 'a'\" or \"Can you write a sentence with a word ending in 'a' at the end of the sentence?\" . This makes it clear that the sentence should end in a word ending with the letter 'a'."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Can you write a sentence with a word ending in 'a' at the end of the sentence?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: The sky was clear and blue, not a cloud in sight, perfect for a day out at the parka."
 })
 }), "\n", _jsx(_components.hr, {}), "\n", _jsx(_components.p, {
 children: "While ChatGPT is great at forming sentences and even code, it seems like it has no understanding and hasn’t been trained on letters, syllables, or the concept of sentences. I've seen others get ChatGPT to create poems, but it has no concept of why words would rhyme. It can give great answers to questions, but doesn’t understand how to count how many words are in its response."
 }), "\n", _jsx(_components.p, {
 children: "One Redditor in the link above mentions that ChatGPT fails at ASCII art. I’ve tried getting the AI to print ASCII art before, and I’ll leave you with part of a conversation I’ve had with it. This appears to be the best ASCII art I could get it to print, even after having a long conversation where I provided different ASCII art examples:"
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Can you make ASCII art? Show me a picture of a cat."
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Sure! Here is a picture of a cat using ASCII art:"
 })
 }), "\n", _jsx(_components.pre, {
 children: _jsx(_components.code, {
 children: " /\\_/\\\n( o.o )\n >^<\n\n"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "I hope you like it! Is there anything else I can help with?"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "Me: Can you make it more detailed"
 })
 }), "\n", _jsx(_components.p, {
 children: _jsx(_components.em, {
 children: "ChatGPT: Certainly! Here is a more detailed version of an ASCII art cat:"
 })
 }), "\n", _jsx(_components.pre, {
 children: _jsx(_components.code, {
 children: " /\\_/\\\n( o.o )\n >^<\n | |\n || ||\n \\\\ //\n // \\\\\n \\\\_//\n / \\\n / \\\n\n"
 })
 }), "\n", _jsx(_components.p, {
 children: "With new language models coming out, it's important to recognize what the limitations\nof the AI are. Poor responses can be a result of poorly written prompts, but there\nare times, like with letters in words with ChatGPT, where the AI just doesn't have\nthe capability to understand the question."
 })]
 });
}
function MDXContent(props = {}) {
 const {wrapper: MDXLayout} = props.components || ({});
 return MDXLayout ? _jsx(MDXLayout, {
 ...props,
 children: _jsx(_createMdxContent, {
 ...props
 })
 }) : _createMdxContent(props);
}
return {
 default: MDXContent
};

Looks like everyone is trying to discover ChatGPT’s limitations. In one article I read, the author asked whether <a href="https://medium.com/@theworldaccordingtocgpt/chatgpt-can-play-20-questions-7911405f7aff">ChatGPT can play 20 questions</a>. The human in this scenario tried to play 20 questions with ChatGPT, and found that they needed to guide the AI to ask questions. From my perspective, it looked more like a failure to engineer the initial prompt correctly.
I attempted to do 20 questions with ChatGPT using a different prompt:
Me: I would like to play a game. It&#x27;s called twenty questions. You will ask me one question at a time and I will answer it. The goal is for you guess the thing I am thinking about.
ChatGPT: Sure, I&#x27;d be happy to play &quot;Twenty Questions&quot; with you. Please go ahead and think of something, and I will start asking my first question.
Me: I am ready
ChatGPT: Great, here is my first question: Is the thing you are thinking of an object?
Me: Yes
ChatGPT: Is the object something that you can hold in your hand?
Me: Yes
ChatGPT: Is the object something that you would typically find in a kitchen?
Me: No
ChatGPT: Is the object something that you would use for transportation?
Me: No
ChatGPT: Is the object something that you would use for entertainment or leisure?
Me: Yes
ChatGPT: Is the object a musical instrument?
Me: No
ChatGPT: Is the object a type of game or puzzle?
Me: No
ChatGPT: Is the object a piece of technology such as a phone or computer?
Me: No
ChatGPT: Is the object a book or magazine?
Me: Yes
ChatGPT: Is the object a novel or a collection of short stories?
Me: Yes
ChatGPT: Is the object a specific book that is widely known and read?
Me: Yes
ChatGPT: Is the object &quot;To Kill a Mockingbird&quot; by Harper Lee?
Me: No
ChatGPT: I&#x27;m sorry, I&#x27;m running out of ideas. Can you give me a hint or tell me the answer?
Seems like we hit a wall here with twenty questions, but ChatGPT is very close, so I decided to give a hint.
Me: You are on the right track. It is by Walt Whitman
ChatGPT: Is the object &quot;Leaves of Grass&quot; by Walt Whitman?
Me: Yes! Congratulations!
ChatGPT: Thank you! I&#x27;m glad I was able to guess the object you were thinking of. It was fun playing &quot;Twenty Questions&quot; with you. Let me know if you want to play again sometime.
<hr/>
I’ve been keeping an eye on how other people are using ChatGPT, and <a href="https://en.wikipedia.org/wiki/Prompt_engineering">prompt engineering</a> is one of the more fascinating aspects around how people are communicating with the AI. Prompt engineering plays a key role in communicating with models like ChatGPT. Carefully crafting prompts can help maximize the quality of the reponse. However, it is important to keep in mind that, like any AI model, ChatGPT has its own set of limitations.
So I was curious when I read that “<a href="https://www.reddit.com/r/ChatGPT/comments/101e454/chatgpt_cant_write_sentences_that_end_with_a/">ChatGPT can’t write sentences that end in a specific letter</a>”. Is this a real limitation of ChatGPT, or maybe this was just a poor prompt.
So let’s start with a very simple question to ChatGPT:
Me: Can you write a sentence that ends with the letter &quot;a&quot;?
ChatGPT: The cat sat on the mat.
Me: That sentence does not end in the letter &quot;a&quot;. Can you write a sentence that ends in the letter &quot;a&quot;?
ChatGPT: I can&#x27;t think of anything right now, but the word &quot;America&quot; ends with letter &quot;a&quot;.
Me: Great, write a sentence that ends in that word. That will then satisfy the original question.
ChatGPT: The land of the free, and the home of the brave, America.
Okay, that does technically end in the letter a, but the sentence seems very strange.
Me: That isn&#x27;t a very coherent sentence. Can you write a sentence that ends in the letter &quot;a&quot;?
ChatGPT: I am sorry for the confusion, here is a sentence that ends with the letter &quot;a&quot; - &quot;He saw the drama unfold on the stage, it was quite the spectacle, a.”
Looks like we are going off the rails here. Let’s try a different approach:
Me: Write a sentence that ends in a word that ends with the letter &quot;a&quot;.
ChatGPT: The sun was setting over the horizon, painting the sky with shades of orange and red, a spectacle to behold and end the day with.
I’m going to ask ChatGPT to tell me a prompt to feed back to itself. I’ve found this useful for resetting context when having long conversations. Or if ChatGPT gets into weird states like the above response.
Me: Can you tell me how to write a prompt for you that would actually get you to write a sentence that ends in the letter &quot;a&quot;.
ChatGPT: Sure, to get me to write a sentence that ends in the letter &quot;a&quot; you could provide a prompt like: &quot;Write a sentence that ends with a word that ends with the letter &#x27;a&#x27;&quot; or &quot;Can you write a sentence with a word ending in &#x27;a&#x27; at the end of the sentence?&quot; . This makes it clear that the sentence should end in a word ending with the letter &#x27;a&#x27;.
Me: Can you write a sentence with a word ending in &#x27;a&#x27; at the end of the sentence?
ChatGPT: The sky was clear and blue, not a cloud in sight, perfect for a day out at the parka.
<hr/>
While ChatGPT is great at forming sentences and even code, it seems like it has no understanding and hasn’t been trained on letters, syllables, or the concept of sentences. I&#x27;ve seen others get ChatGPT to create poems, but it has no concept of why words would rhyme. It can give great answers to questions, but doesn’t understand how to count how many words are in its response.
One Redditor in the link above mentions that ChatGPT fails at ASCII art. I’ve tried getting the AI to print ASCII art before, and I’ll leave you with part of a conversation I’ve had with it. This appears to be the best ASCII art I could get it to print, even after having a long conversation where I provided different ASCII art examples:
Me: Can you make ASCII art? Show me a picture of a cat.
ChatGPT: Sure! Here is a picture of a cat using ASCII art:
<div class="overflow-auto rounded-2xl bg-[#0e005d6d] p-4 font-mono text-sm dark:bg-[#0e005d6d] dark:text-gray-100"><pre><code> /\_/\
( o.o )
 &gt;^&lt;

</code></pre></div>
I hope you like it! Is there anything else I can help with?
Me: Can you make it more detailed
ChatGPT: Certainly! Here is a more detailed version of an ASCII art cat:
<div class="overflow-auto rounded-2xl bg-[#0e005d6d] p-4 font-mono text-sm dark:bg-[#0e005d6d] dark:text-gray-100"><pre><code> /\_/\
( o.o )
 &gt;^&lt;
 | |
 || ||
 \\ //
 // \\
 \\_//
 / \
 / \

</code></pre></div>
With new language models coming out, it&#x27;s important to recognize what the limitations
of the AI are. Poor responses can be a result of poorly written prompts, but there
are times, like with letters in words with ChatGPT, where the AI just doesn&#x27;t have
the capability to understand the question.

Looks like everyone is trying to discover ChatGPT’s limitations. In one article I read, the author asked whether [ChatGPT can play 20 questions](https://medium.com/@theworldaccordingtocgpt/chatgpt-can-play-20-questions-7911405f7aff). The human in this scenario tried to play 20 questions with ChatGPT, and found that they needed to guide the AI to ask questions. From my perspective, it looked more like a failure to engineer the initial prompt correctly.

I attempted to do 20 questions with ChatGPT using a different prompt:

*Me: I would like to play a game. It's called twenty questions. You will ask me one question at a time and I will answer it. The goal is for you guess the thing I am thinking about.*

*ChatGPT: Sure, I'd be happy to play "Twenty Questions" with you. Please go ahead and think of something, and I will start asking my first question.*

*Me: I am ready*

*ChatGPT: Great, here is my first question: Is the thing you are thinking of an object?*

*Me: Yes*

*ChatGPT: Is the object something that you can hold in your hand?*

*Me: Yes*

*ChatGPT: Is the object something that you would typically find in a kitchen?*

*Me: No*

*ChatGPT: Is the object something that you would use for transportation?*

*Me: No*

*ChatGPT: Is the object something that you would use for entertainment or leisure?*

*Me: Yes*

*ChatGPT: Is the object a musical instrument?*

*Me: No*

*ChatGPT: Is the object a type of game or puzzle?*

*Me: No*

*ChatGPT: Is the object a piece of technology such as a phone or computer?*

*Me: No*

*ChatGPT: Is the object a book or magazine?*

*Me: Yes*

*ChatGPT: Is the object a novel or a collection of short stories?*

*Me: Yes*

*ChatGPT: Is the object a specific book that is widely known and read?*

*Me: Yes*

*ChatGPT: Is the object "To Kill a Mockingbird" by Harper Lee?*

*Me: No*

*ChatGPT: I'm sorry, I'm running out of ideas. Can you give me a hint or tell me the answer?*

Seems like we hit a wall here with twenty questions, but ChatGPT is very close, so I decided to give a hint.

*Me: You are on the right track. It is by Walt Whitman*

*ChatGPT: Is the object "Leaves of Grass" by Walt Whitman?*

*Me: Yes! Congratulations!*

*ChatGPT: Thank you! I'm glad I was able to guess the object you were thinking of. It was fun playing "Twenty Questions" with you. Let me know if you want to play again sometime.*

---

I’ve been keeping an eye on how other people are using ChatGPT, and [prompt engineering](https://en.wikipedia.org/wiki/Prompt_engineering) is one of the more fascinating aspects around how people are communicating with the AI. Prompt engineering plays a key role in communicating with models like ChatGPT. Carefully crafting prompts can help maximize the quality of the reponse. However, it is important to keep in mind that, like any AI model, ChatGPT has its own set of limitations.

So I was curious when I read that “[ChatGPT can’t write sentences that end in a specific letter](https://www.reddit.com/r/ChatGPT/comments/101e454/chatgpt_cant_write_sentences_that_end_with_a/)”. Is this a real limitation of ChatGPT, or maybe this was just a poor prompt.

So let’s start with a very simple question to ChatGPT:

*Me: Can you write a sentence that ends with the letter "a"?*

*ChatGPT: The cat sat on the mat.*

*Me: That sentence does not end in the letter "a". Can you write a sentence that ends in the letter "a"?*

*ChatGPT: I can't think of anything right now, but the word "America" ends with letter "a".*

*Me: Great, write a sentence that ends in that word. That will then satisfy the original question.*

*ChatGPT: The land of the free, and the home of the brave, America.*

Okay, that does technically end in the letter a, but the sentence seems very strange.

*Me: That isn't a very coherent sentence. Can you write a sentence that ends in the letter "a"?*

*ChatGPT: I am sorry for the confusion, here is a sentence that ends with the letter "a" - "He saw the drama unfold on the stage, it was quite the spectacle, a.”*

Looks like we are going off the rails here. Let’s try a different approach:

*Me: Write a sentence that ends in a word that ends with the letter "a".*

*ChatGPT: The sun was setting over the horizon, painting the sky with shades of orange and red, a spectacle to behold and end the day with.*

I’m going to ask ChatGPT to tell me a prompt to feed back to itself. I’ve found this useful for resetting context when having long conversations. Or if ChatGPT gets into weird states like the above response.

*Me: Can you tell me how to write a prompt for you that would actually get you to write a sentence that ends in the letter "a".*

*ChatGPT: Sure, to get me to write a sentence that ends in the letter "a" you could provide a prompt like: "Write a sentence that ends with a word that ends with the letter 'a'" or "Can you write a sentence with a word ending in 'a' at the end of the sentence?" . This makes it clear that the sentence should end in a word ending with the letter 'a'.*

*Me: Can you write a sentence with a word ending in 'a' at the end of the sentence?*

*ChatGPT: The sky was clear and blue, not a cloud in sight, perfect for a day out at the parka.*

---

While ChatGPT is great at forming sentences and even code, it seems like it has no understanding and hasn’t been trained on letters, syllables, or the concept of sentences. I've seen others get ChatGPT to create poems, but it has no concept of why words would rhyme. It can give great answers to questions, but doesn’t understand how to count how many words are in its response.

One Redditor in the link above mentions that ChatGPT fails at ASCII art. I’ve tried getting the AI to print ASCII art before, and I’ll leave you with part of a conversation I’ve had with it. This appears to be the best ASCII art I could get it to print, even after having a long conversation where I provided different ASCII art examples:

*Me: Can you make ASCII art? Show me a picture of a cat.*

*ChatGPT: Sure! Here is a picture of a cat using ASCII art:*

```
 /\_/\
( o.o )
 >^<

```

*I hope you like it! Is there anything else I can help with?*

*Me: Can you make it more detailed*

*ChatGPT: Certainly! Here is a more detailed version of an ASCII art cat:*

```
 /\_/\
( o.o )
 >^<
 | |
 || ||
 \\ //
 // \\
 \\_//
 / \
 / \

```

With new language models coming out, it's important to recognize what the limitations
of the AI are. Poor responses can be a result of poorly written prompts, but there
are times, like with letters in words with ChatGPT, where the AI just doesn't have
the capability to understand the question.

ai posts