Sora - OpenAI Visual Studio Code Extension - idmontie's Portfolio

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = Object.assign({
    p: "p",
    a: "a",
    h2: "h2"
  }, props.components);
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "One of the first PC games I played was Starcraft and the expansion Starcraft: Broodwar. We didn’t have a PC, so I had to play it on a friend’s computer, but I remember being immersed in the real-time strategy gameplay."
    }), "\n", _jsxs(_components.p, {
      children: ["Later, I would get a PC and bought my own copy of Starcraft. This was years later, but the community for the game was still impressive. Joining one of the older Starcraft forums, I discovered “mods” for the game. The most popular ones were just graphical changes, but that concept of changing a game to display your own graphics was so interesting to me. I’m not sure how I stumbled on it, but I found ", _jsx(_components.a, {
        href: "http://www.staredit.net/",
        children: "http://www.staredit.net/"
      }), " (yes, it’s still active!) and I learned that you could do so much more than just change the graphics of Starcraft with mods. There were people in the community working on hooking into existing Starcraft code to modify the gameplay and graphics using C++."]
    }), "\n", _jsx(_components.h2, {
      children: "Diving deep"
    }), "\n", _jsx(_components.p, {
      children: "In order to create these mods, we needed to know the hex address of different functions that Starcraft would call during them game. And to do anything meaningful, we also needed to know the structure of units and sprites and where those were stored as well. A lot of this base work required using tools like OllyDbg to analyze the assembly of the Starcraft executable, and ArtMoney for analyzing the memory of the game while it was executing."
    }), "\n", _jsx(_components.p, {
      children: "I used OllyDbg to find what functions Starcraft would call during execution – from common functions like the game-loop, to highly specialized functions like checking supply limits. ArtMoney let us determine how structures like units were laid out – where each unit’s health was stored, how much damage each weapon would do, and more."
    }), "\n", _jsxs(_components.p, {
      children: ["Most of this information was shared in forums, chat rooms, and a few disparate sites for looking up hex addresses. Of course, no write up about modding Starcraft is complete without mentioning all the work from ShadowFlare (check out ", _jsx(_components.a, {
        href: "https://sfsrealm.hopto.org/",
        children: "ShadowFlare’s realm"
      }), " for all the work they did), the great people at Staredit.Net, and plenty of others who built amazing tools to work with Starcraft files. All these sites were amazing resources for figuring out file specs and working with the Starcraft engine."]
    }), "\n", _jsx(_components.p, {
      children: "However, each time I wanted to write a new mod or experiment with an idea, I’d have to look up all of this information across all of these sites."
    }), "\n", _jsx(_components.h2, {
      children: "Bringing it together"
    }), "\n", _jsx(_components.p, {
      children: "The goal with GPTP (General Plugin Template Project), was to take all of this work that the amazing modding community had done and bring it together into a C++ Visual Studio project that could be copied for a new mod."
    }), "\n", _jsx(_components.p, {
      children: "My initial idea was simple: take all of the code for injecting new function hooks into Starcraft and wrap it in some very friendly functions. When a modder would come in to create a new project, they would have three functions exposed to them to work with: gameStart, gameLoop, gameEnd (I don’t remember the exact names I gave them at the time)."
    }), "\n", _jsx(_components.p, {
      children: "I released the initial GPTP back in 2008-2009. This included the project setup for compiling and producing a QDP file that could be loaded into Starcraft. Additionally, it contained known structures and hex addresses that developers could use to build their mod. The Intellisense autocomplete feature really helped developers leverage these structures."
    }), "\n", _jsx(_components.h2, {
      children: "The power of open source"
    }), "\n", _jsx(_components.p, {
      children: "I made the original version of GPTP back in high school, but I didn’t have time to continue to work on it when I went to university. I returned to the community a decade later, and found that not only was the community still alive and working on mods, but that they were using GPTP. At this point, GPTP was unrecognizable from my original work; the goal was the same, but the quality of the code was greatly improved and the number of hooks, known structures, and addresses was much more impressive."
    }), "\n", _jsxs(_components.p, {
      children: ["You can view the project on Github: ", _jsx(_components.a, {
        href: "https://github.com/SCMapsAndMods/general-plugin-template-project",
        children: "general-plugin-template-project"
      }), "."]
    }), "\n", _jsx(_components.p, {
      children: "I’d like to thank open source for this sort of development. The amount of work that has been continually added onto this project couldn’t have been done without all the contributions from the community."
    }), "\n", _jsx(_components.h2, {
      children: "Does it still work?"
    }), "\n", _jsx(_components.p, {
      children: "I recently went back to Starcraft modding as a fun little project. I wanted to hop in and see how easy or difficult it would be to create a mod in 2020."
    }), "\n", _jsx(_components.p, {
      children: "One of the harder things to do in 2020 is find all the tools and initial setup for modding Starcraft. Some of the modding sites are down for good, and with them the knowledge they contained."
    }), "\n", _jsx(_components.p, {
      children: "But, I pulled together the tools (thanks PyMS), pulled together tutorials and other instructions into a Notion document, and created my first Starcraft mod!"
    }), "\n", _jsx(_components.p, {
      children: "Gathering all the tools was the hard part. After that, it was actually very straightforward creating a new GPTP template and mod. All of the known structures and hooks are pretty self-explanatory and I was able to even add my own hooks once I found the hex addresses using OllyDbg."
    }), "\n", _jsx(_components.h2, {
      children: "Looking forward"
    }), "\n", _jsx(_components.p, {
      children: "Starcraft modding is like an ancient art at this point. There isn’t a large audience for it, so in general, any new mod will be played by maybe 10 people. But to me, it’s a combination of nostalgia, hard work, tinkering, discovery, and creation that makes it so much fun. That feeling of finding a new function to hook onto, writing C++ code that injects itself into it, and then running the code in the Starcraft engine and seeing it work is a special kind of rewarding experience."
    }), "\n", _jsx(_components.h2, {
      children: "Thanks"
    }), "\n", _jsxs(_components.p, {
      children: ["I’d like to give thanks to everyone who has ever contributed to the modding community for Starcraft. There are so many names that I couldn’t possibly name them all, but please check of ", _jsx(_components.a, {
        href: "http://staredit.net",
        children: "http://staredit.net"
      }), " if any of this interests you."]
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, props)
  })) : _createMdxContent(props);
}
return {
  default: MDXContent
};


<p>One of the first PC games I played was Starcraft and the expansion Starcraft: Broodwar. We didn’t have a PC, so I had to play it on a friend’s computer, but I remember being immersed in the real-time strategy gameplay.</p>
<p>Later, I would get a PC and bought my own copy of Starcraft. This was years later, but the community for the game was still impressive. Joining one of the older Starcraft forums, I discovered “mods” for the game. The most popular ones were just graphical changes, but that concept of changing a game to display your own graphics was so interesting to me. I’m not sure how I stumbled on it, but I found <a href="http://www.staredit.net/">http://www.staredit.net/</a> (yes, it’s still active!) and I learned that you could do so much more than just change the graphics of Starcraft with mods. There were people in the community working on hooking into existing Starcraft code to modify the gameplay and graphics using C++.</p>
<h2>Diving deep</h2>
<p>In order to create these mods, we needed to know the hex address of different functions that Starcraft would call during them game. And to do anything meaningful, we also needed to know the structure of units and sprites and where those were stored as well. A lot of this base work required using tools like OllyDbg to analyze the assembly of the Starcraft executable, and ArtMoney for analyzing the memory of the game while it was executing.</p>
<p>I used OllyDbg to find what functions Starcraft would call during execution – from common functions like the game-loop, to highly specialized functions like checking supply limits. ArtMoney let us determine how structures like units were laid out – where each unit’s health was stored, how much damage each weapon would do, and more.</p>
<p>Most of this information was shared in forums, chat rooms, and a few disparate sites for looking up hex addresses. Of course, no write up about modding Starcraft is complete without mentioning all the work from ShadowFlare (check out <a href="https://sfsrealm.hopto.org/">ShadowFlare’s realm</a> for all the work they did), the great people at Staredit.Net, and plenty of others who built amazing tools to work with Starcraft files. All these sites were amazing resources for figuring out file specs and working with the Starcraft engine.</p>
<p>However, each time I wanted to write a new mod or experiment with an idea, I’d have to look up all of this information across all of these sites.</p>
<h2>Bringing it together</h2>
<p>The goal with GPTP (General Plugin Template Project), was to take all of this work that the amazing modding community had done and bring it together into a C++ Visual Studio project that could be copied for a new mod.</p>
<p>My initial idea was simple: take all of the code for injecting new function hooks into Starcraft and wrap it in some very friendly functions. When a modder would come in to create a new project, they would have three functions exposed to them to work with: gameStart, gameLoop, gameEnd (I don’t remember the exact names I gave them at the time).</p>
<p>I released the initial GPTP back in 2008-2009. This included the project setup for compiling and producing a QDP file that could be loaded into Starcraft. Additionally, it contained known structures and hex addresses that developers could use to build their mod. The Intellisense autocomplete feature really helped developers leverage these structures.</p>
<h2>The power of open source</h2>
<p>I made the original version of GPTP back in high school, but I didn’t have time to continue to work on it when I went to university. I returned to the community a decade later, and found that not only was the community still alive and working on mods, but that they were using GPTP. At this point, GPTP was unrecognizable from my original work; the goal was the same, but the quality of the code was greatly improved and the number of hooks, known structures, and addresses was much more impressive.</p>
<p>You can view the project on Github: <a href="https://github.com/SCMapsAndMods/general-plugin-template-project">general-plugin-template-project</a>.</p>
<p>I’d like to thank open source for this sort of development. The amount of work that has been continually added onto this project couldn’t have been done without all the contributions from the community.</p>
<h2>Does it still work?</h2>
<p>I recently went back to Starcraft modding as a fun little project. I wanted to hop in and see how easy or difficult it would be to create a mod in 2020.</p>
<p>One of the harder things to do in 2020 is find all the tools and initial setup for modding Starcraft. Some of the modding sites are down for good, and with them the knowledge they contained.</p>
<p>But, I pulled together the tools (thanks PyMS), pulled together tutorials and other instructions into a Notion document, and created my first Starcraft mod!</p>
<p>Gathering all the tools was the hard part. After that, it was actually very straightforward creating a new GPTP template and mod. All of the known structures and hooks are pretty self-explanatory and I was able to even add my own hooks once I found the hex addresses using OllyDbg.</p>
<h2>Looking forward</h2>
<p>Starcraft modding is like an ancient art at this point. There isn’t a large audience for it, so in general, any new mod will be played by maybe 10 people. But to me, it’s a combination of nostalgia, hard work, tinkering, discovery, and creation that makes it so much fun. That feeling of finding a new function to hook onto, writing C++ code that injects itself into it, and then running the code in the Starcraft engine and seeing it work is a special kind of rewarding experience.</p>
<h2>Thanks</h2>
<p>I’d like to give thanks to everyone who has ever contributed to the modding community for Starcraft. There are so many names that I couldn’t possibly name them all, but please check of <a href="http://staredit.net">http://staredit.net</a> if any of this interests you.</p>


One of the first PC games I played was Starcraft and the expansion Starcraft: Broodwar. We didn’t have a PC, so I had to play it on a friend’s computer, but I remember being immersed in the real-time strategy gameplay.

Later, I would get a PC and bought my own copy of Starcraft. This was years later, but the community for the game was still impressive. Joining one of the older Starcraft forums, I discovered “mods” for the game. The most popular ones were just graphical changes, but that concept of changing a game to display your own graphics was so interesting to me. I’m not sure how I stumbled on it, but I found [http://www.staredit.net/](http://www.staredit.net/) (yes, it’s still active!) and I learned that you could do so much more than just change the graphics of Starcraft with mods. There were people in the community working on hooking into existing Starcraft code to modify the gameplay and graphics using C++.

## Diving deep

In order to create these mods, we needed to know the hex address of different functions that Starcraft would call during them game. And to do anything meaningful, we also needed to know the structure of units and sprites and where those were stored as well. A lot of this base work required using tools like OllyDbg to analyze the assembly of the Starcraft executable, and ArtMoney for analyzing the memory of the game while it was executing.

I used OllyDbg to find what functions Starcraft would call during execution – from common functions like the game-loop, to highly specialized functions like checking supply limits. ArtMoney let us determine how structures like units were laid out – where each unit’s health was stored, how much damage each weapon would do, and more.

Most of this information was shared in forums, chat rooms, and a few disparate sites for looking up hex addresses. Of course, no write up about modding Starcraft is complete without mentioning all the work from ShadowFlare (check out [ShadowFlare’s realm](https://sfsrealm.hopto.org/) for all the work they did), the great people at Staredit.Net, and plenty of others who built amazing tools to work with Starcraft files. All these sites were amazing resources for figuring out file specs and working with the Starcraft engine.

However, each time I wanted to write a new mod or experiment with an idea, I’d have to look up all of this information across all of these sites.

## Bringing it together

The goal with GPTP (General Plugin Template Project), was to take all of this work that the amazing modding community had done and bring it together into a C++ Visual Studio project that could be copied for a new mod.

My initial idea was simple: take all of the code for injecting new function hooks into Starcraft and wrap it in some very friendly functions. When a modder would come in to create a new project, they would have three functions exposed to them to work with: gameStart, gameLoop, gameEnd (I don’t remember the exact names I gave them at the time).

I released the initial GPTP back in 2008-2009. This included the project setup for compiling and producing a QDP file that could be loaded into Starcraft. Additionally, it contained known structures and hex addresses that developers could use to build their mod. The Intellisense autocomplete feature really helped developers leverage these structures.

## The power of open source

I made the original version of GPTP back in high school, but I didn’t have time to continue to work on it when I went to university. I returned to the community a decade later, and found that not only was the community still alive and working on mods, but that they were using GPTP. At this point, GPTP was unrecognizable from my original work; the goal was the same, but the quality of the code was greatly improved and the number of hooks, known structures, and addresses was much more impressive.

You can view the project on Github: [general-plugin-template-project](https://github.com/SCMapsAndMods/general-plugin-template-project).

I’d like to thank open source for this sort of development. The amount of work that has been continually added onto this project couldn’t have been done without all the contributions from the community.

## Does it still work?

I recently went back to Starcraft modding as a fun little project. I wanted to hop in and see how easy or difficult it would be to create a mod in 2020.

One of the harder things to do in 2020 is find all the tools and initial setup for modding Starcraft. Some of the modding sites are down for good, and with them the knowledge they contained.

But, I pulled together the tools (thanks PyMS), pulled together tutorials and other instructions into a Notion document, and created my first Starcraft mod!

Gathering all the tools was the hard part. After that, it was actually very straightforward creating a new GPTP template and mod. All of the known structures and hooks are pretty self-explanatory and I was able to even add my own hooks once I found the hex addresses using OllyDbg.

## Looking forward

Starcraft modding is like an ancient art at this point. There isn’t a large audience for it, so in general, any new mod will be played by maybe 10 people. But to me, it’s a combination of nostalgia, hard work, tinkering, discovery, and creation that makes it so much fun. That feeling of finding a new function to hook onto, writing C++ code that injects itself into it, and then running the code in the Starcraft engine and seeing it work is a special kind of rewarding experience.

## Thanks

I’d like to give thanks to everyone who has ever contributed to the modding community for Starcraft. There are so many names that I couldn’t possibly name them all, but please check of [http://staredit.net](http://staredit.net) if any of this interests you.


Revisiting GPTP - the Starcraft modding toolkit

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = Object.assign({
    p: "p",
    code: "code",
    pre: "pre",
    img: "img",
    h2: "h2",
    strong: "strong",
    a: "a"
  }, props.components);
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "Github Copilot and other AI tools are hitting the scene. I decided to create my own Visual Studio Code extension, which is designed to use OpenAI’s APIs to bring some additional ChatGPT functionality into the code editor. The goal with Sora was to enable a developer to thoughtfully write a comment about the code they would like the AI to write, and then commit to it – rather than the real-time typeahead that Github Copilot provides."
    }), "\n", _jsxs(_components.p, {
      children: ["With that goal in mind, Sora provides two ways to activate OpenAI: by typing ", _jsx(_components.code, {
        children: "@OpenAI"
      }), " (formally ", _jsx(_components.code, {
        children: "@ChatGPT"
      }), ") or clicking “Send to OpenAI” when hovering over a comment. The other improvement is that Sora will read any relative link references to files in your project. A great way to use this is to have OpenAI write code in the style that already exists in your project, for example:"]
    }), "\n", _jsx(_components.pre, {
      children: _jsx(_components.code, {
        className: "language-tsx",
        children: "/**\n * Write tests for [my-file](./my-file.ts] using\n * [other-test](../something/other-file.test.ts) as an example\n */\n"
      })
    }), "\n", _jsx(_components.p, {
      children: "Using this extension lets users write specifications as comments, and have ChatGPT write the entire file for you."
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2023-06-06-sora/sora-preview.gif",
        alt: "Sora Preview"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["This extension leverages the OpenAI API to send any referenced files and a starting prompt to the ", _jsx(_components.code, {
        children: "gpt-3.5-turbo"
      }), " chat completion endpoint. The prompt mainly sets the context for creating working code using a given language and reference files."]
    }), "\n", _jsx(_components.p, {
      children: "Once the response comes back, the extension parses it and appends it to the original file."
    }), "\n", _jsx(_components.h2, {
      children: _jsx(_components.strong, {
        children: "Installation and Usage"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["You can install the extension by going to ", _jsx(_components.a, {
        href: "https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat",
        children: "the VSCode Marketplace"
      }), ", or searching for “Sora” in Visual Studio Code extensions."]
    }), "\n", _jsxs(_components.p, {
      children: ["Once installed, you will need to enter your own OpenAI API key. You can get your key by following ", _jsx(_components.a, {
        href: "https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key",
        children: "these instructions"
      }), ". Then just enter ", _jsx(_components.code, {
        children: "Sora: Set API Key"
      }), " into the Visual Studio Code command prompt."]
    }), "\n", _jsx(_components.p, {
      children: _jsx(_components.img, {
        src: "/media/2023-06-06-sora/sora-set-api-key.png",
        alt: "Sora Set API Key"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["You can review the code ", _jsx(_components.a, {
        href: "https://github.com/CapsuleCat/sora-by-capsule-cat",
        children: "on Github"
      }), "."]
    }), "\n", _jsx(_components.h2, {
      children: _jsx(_components.strong, {
        children: "Conclusion"
      })
    }), "\n", _jsxs(_components.p, {
      children: ["It was fun building my first extension. Please feel free to reach out on our ", _jsx(_components.a, {
        href: "https://github.com/CapsuleCat/sora-by-capsule-cat",
        children: "Github"
      }), " for feedback or questions."]
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, props)
  })) : _createMdxContent(props);
}
return {
  default: MDXContent
};


<p>Github Copilot and other AI tools are hitting the scene. I decided to create my own Visual Studio Code extension, which is designed to use OpenAI’s APIs to bring some additional ChatGPT functionality into the code editor. The goal with Sora was to enable a developer to thoughtfully write a comment about the code they would like the AI to write, and then commit to it – rather than the real-time typeahead that Github Copilot provides.</p>
<p>With that goal in mind, Sora provides two ways to activate OpenAI: by typing <code>@OpenAI</code> (formally <code>@ChatGPT</code>) or clicking “Send to OpenAI” when hovering over a comment. The other improvement is that Sora will read any relative link references to files in your project. A great way to use this is to have OpenAI write code in the style that already exists in your project, for example:</p>
<div class="overflow-auto rounded bg-gray-200 p-4 font-mono text-sm dark:bg-gray-800 dark:text-gray-100"><pre><code class="language-tsx">/**
 * Write tests for [my-file](./my-file.ts] using
 * [other-test](../something/other-file.test.ts) as an example
 */
</code></pre></div>
<p>Using this extension lets users write specifications as comments, and have ChatGPT write the entire file for you.</p>
<p><img alt="Sora Preview" src="/media/2023-06-06-sora/sora-preview.gif" style="max-height:500px;margin:auto;text-align:center"/></p>
<p>This extension leverages the OpenAI API to send any referenced files and a starting prompt to the <code>gpt-3.5-turbo</code> chat completion endpoint. The prompt mainly sets the context for creating working code using a given language and reference files.</p>
<p>Once the response comes back, the extension parses it and appends it to the original file.</p>
<h2><strong>Installation and Usage</strong></h2>
<p>You can install the extension by going to <a href="https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat">the VSCode Marketplace</a>, or searching for “Sora” in Visual Studio Code extensions.</p>
<p>Once installed, you will need to enter your own OpenAI API key. You can get your key by following <a href="https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key">these instructions</a>. Then just enter <code>Sora: Set API Key</code> into the Visual Studio Code command prompt.</p>
<p><img alt="Sora Set API Key" src="/media/2023-06-06-sora/sora-set-api-key.png" style="max-height:500px;margin:auto;text-align:center"/></p>
<p>You can review the code <a href="https://github.com/CapsuleCat/sora-by-capsule-cat">on Github</a>.</p>
<h2><strong>Conclusion</strong></h2>
<p>It was fun building my first extension. Please feel free to reach out on our <a href="https://github.com/CapsuleCat/sora-by-capsule-cat">Github</a> for feedback or questions.</p>

Github Copilot and other AI tools are hitting the scene. I decided to create my own Visual Studio Code extension, which is designed to use OpenAI’s APIs to bring some additional ChatGPT functionality into the code editor. The goal with Sora was to enable a developer to thoughtfully write a comment about the code they would like the AI to write, and then commit to it – rather than the real-time typeahead that Github Copilot provides.

With that goal in mind, Sora provides two ways to activate OpenAI: by typing `@OpenAI` (formally `@ChatGPT`) or clicking “Send to OpenAI” when hovering over a comment. The other improvement is that Sora will read any relative link references to files in your project. A great way to use this is to have OpenAI write code in the style that already exists in your project, for example:

```tsx
/**
 * Write tests for [my-file](./my-file.ts] using
 * [other-test](../something/other-file.test.ts) as an example
 */
```

Using this extension lets users write specifications as comments, and have ChatGPT write the entire file for you.

![Sora Preview](/media/2023-06-06-sora/sora-preview.gif)

This extension leverages the OpenAI API to send any referenced files and a starting prompt to the `gpt-3.5-turbo` chat completion endpoint. The prompt mainly sets the context for creating working code using a given language and reference files.

Once the response comes back, the extension parses it and appends it to the original file.

## **Installation and Usage**

You can install the extension by going to [the VSCode Marketplace](https://marketplace.visualstudio.com/items?itemName=CapsuleCat.sora-by-capsule-cat), or searching for “Sora” in Visual Studio Code extensions.

Once installed, you will need to enter your own OpenAI API key. You can get your key by following [these instructions](https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key). Then just enter `Sora: Set API Key` into the Visual Studio Code command prompt.

![Sora Set API Key](/media/2023-06-06-sora/sora-set-api-key.png)

You can review the code [on Github](https://github.com/CapsuleCat/sora-by-capsule-cat).

## **Conclusion**

It was fun building my first extension. Please feel free to reach out on our [Github](https://github.com/CapsuleCat/sora-by-capsule-cat) for feedback or questions.


Sora - OpenAI Visual Studio Code Extension

/*@jsxRuntime automatic @jsxImportSource react*/
const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0];
function _createMdxContent(props) {
  const _components = Object.assign({
    p: "p",
    pre: "pre",
    code: "code",
    h2: "h2",
    a: "a"
  }, props.components);
  return _jsxs(_Fragment, {
    children: [_jsx(_components.p, {
      children: "While working on the Clarity Hub NLP API, we had a common use-case where we would create embeddings from text, and use those embedding to determine cosine similarity with other embeddings. Doing this required loading all of the embeddings in-memory and then computing cosine similarity with the entire dataset. As the dataset grew, this operation would get incredibly slow."
    }), "\n", _jsx(_components.p, {
      children: "We worked on a fast way to do these lookups using ranges that can be performed in any database. This approach was never implemented, but we worked on multiple proof-of-concepts to test out our ideas. The goal was to take an input text, compute an embedding, load the entire embedding datasets loaded into an AWS lambda, find the most similar set of vectors, and return the top N similar vectors in one use-case. To tackle that, we came up with the following idea."
    }), "\n", _jsx(_components.p, {
      children: "Given a vector A, compute is similar to a unit vector U of the same dimension as A. So:"
    }), "\n", _jsx(_components.pre, {
      children: _jsx(_components.code, {
        className: "language-cpp",
        children: "dim(U) = dim(A)\n"
      })
    }), "\n", _jsx(_components.p, {
      children: "And"
    }), "\n", _jsx(_components.pre, {
      children: _jsx(_components.code, {
        className: "language-cpp",
        children: "S_u = cos(θ) = A · U / ||A|| x ||U||\n"
      })
    }), "\n", _jsx(_components.p, {
      children: "Where S_u is the similarity with the unit vector. The unit vector just needs to be the same across all samples."
    }), "\n", _jsx(_components.p, {
      children: "For each embedding, store the calculated S_u."
    }), "\n", _jsx(_components.p, {
      children: "If we want to find similar vectors for a new vector B, then we compute is similarity to the unit vector."
    }), "\n", _jsxs(_components.p, {
      children: ["Then, we can query the database for vectors within an interval of ", _jsx(_components.code, {
        children: "[S_u - ε, S_u + ε]"
      }), " . This will give us a subset of the dataset that have similar similarities with the unit vector."]
    }), "\n", _jsx(_components.p, {
      children: "We can re-query increasing or decreasing ε until the top N results are found."
    }), "\n", _jsx(_components.p, {
      children: "To further improve accuracy, we can also re-compute the similarity score using cosine similarity with the subset of vectors, which is still much faster then computing the similarity against the entire dataset."
    }), "\n", _jsxs(_components.p, {
      children: ["This approach begins to break down as the cosine similarity to the unit vector chosen gets very large (", _jsx(_components.code, {
        children: "> 0.4"
      }), ").  We end up with the possibility of matching against vectors that are of opposite directions – the least similar vectors to the original input vector."]
    }), "\n", _jsx(_components.p, {
      children: "One solution to workaround this could be to pre-compute the similarity of a vector against unit vectors for each dimension of the input vector. But this could be 512 or more cosine similarity calculations for modern embeddings just to precompute the data. Once all unit vector similarities are calculated and stored, the range query against the database would be made against the column for which the input vector’s similarity is closest to 0."
    }), "\n", _jsx(_components.p, {
      children: "There are a lot of real solutions to this problem, but this was a fun exercise to think about and work on."
    }), "\n", _jsx(_components.h2, {
      children: "Further reading"
    }), "\n", _jsxs(_components.p, {
      children: ["Vector similarity search is becoming increasingly popular and integrated into databases. Here are some resources to learn more: ", _jsx(_components.a, {
        href: "https://zilliz.com/blog/vector-similarity-search",
        children: "Vector Similarity Search"
      }), "."]
    })]
  });
}
function MDXContent(props = {}) {
  const {wrapper: MDXLayout} = props.components || ({});
  return MDXLayout ? _jsx(MDXLayout, Object.assign({}, props, {
    children: _jsx(_createMdxContent, props)
  })) : _createMdxContent(props);
}
return {
  default: MDXContent
};


<p>While working on the Clarity Hub NLP API, we had a common use-case where we would create embeddings from text, and use those embedding to determine cosine similarity with other embeddings. Doing this required loading all of the embeddings in-memory and then computing cosine similarity with the entire dataset. As the dataset grew, this operation would get incredibly slow.</p>
<p>We worked on a fast way to do these lookups using ranges that can be performed in any database. This approach was never implemented, but we worked on multiple proof-of-concepts to test out our ideas. The goal was to take an input text, compute an embedding, load the entire embedding datasets loaded into an AWS lambda, find the most similar set of vectors, and return the top N similar vectors in one use-case. To tackle that, we came up with the following idea.</p>
<p>Given a vector A, compute is similar to a unit vector U of the same dimension as A. So:</p>
<div class="overflow-auto rounded bg-gray-200 p-4 font-mono text-sm dark:bg-gray-800 dark:text-gray-100"><pre><code class="language-cpp">dim(U) = dim(A)
</code></pre></div>
<p>And</p>
<div class="overflow-auto rounded bg-gray-200 p-4 font-mono text-sm dark:bg-gray-800 dark:text-gray-100"><pre><code class="language-cpp">S_u = cos(θ) = A · U / ||A|| x ||U||
</code></pre></div>
<p>Where S_u is the similarity with the unit vector. The unit vector just needs to be the same across all samples.</p>
<p>For each embedding, store the calculated S_u.</p>
<p>If we want to find similar vectors for a new vector B, then we compute is similarity to the unit vector.</p>
<p>Then, we can query the database for vectors within an interval of <code>[S_u - ε, S_u + ε]</code> . This will give us a subset of the dataset that have similar similarities with the unit vector.</p>
<p>We can re-query increasing or decreasing ε until the top N results are found.</p>
<p>To further improve accuracy, we can also re-compute the similarity score using cosine similarity with the subset of vectors, which is still much faster then computing the similarity against the entire dataset.</p>
<p>This approach begins to break down as the cosine similarity to the unit vector chosen gets very large (<code>&gt; 0.4</code>).  We end up with the possibility of matching against vectors that are of opposite directions – the least similar vectors to the original input vector.</p>
<p>One solution to workaround this could be to pre-compute the similarity of a vector against unit vectors for each dimension of the input vector. But this could be 512 or more cosine similarity calculations for modern embeddings just to precompute the data. Once all unit vector similarities are calculated and stored, the range query against the database would be made against the column for which the input vector’s similarity is closest to 0.</p>
<p>There are a lot of real solutions to this problem, but this was a fun exercise to think about and work on.</p>
<h2>Further reading</h2>
<p>Vector similarity search is becoming increasingly popular and integrated into databases. Here are some resources to learn more: <a href="https://zilliz.com/blog/vector-similarity-search">Vector Similarity Search</a>.</p>


While working on the Clarity Hub NLP API, we had a common use-case where we would create embeddings from text, and use those embedding to determine cosine similarity with other embeddings. Doing this required loading all of the embeddings in-memory and then computing cosine similarity with the entire dataset. As the dataset grew, this operation would get incredibly slow.

We worked on a fast way to do these lookups using ranges that can be performed in any database. This approach was never implemented, but we worked on multiple proof-of-concepts to test out our ideas. The goal was to take an input text, compute an embedding, load the entire embedding datasets loaded into an AWS lambda, find the most similar set of vectors, and return the top N similar vectors in one use-case. To tackle that, we came up with the following idea.

Given a vector A, compute is similar to a unit vector U of the same dimension as A. So:

```cpp
dim(U) = dim(A)
```

And

```cpp
S_u = cos(θ) = A · U / ||A|| x ||U||
```

Where S_u is the similarity with the unit vector. The unit vector just needs to be the same across all samples.

For each embedding, store the calculated S_u.

If we want to find similar vectors for a new vector B, then we compute is similarity to the unit vector.

Then, we can query the database for vectors within an interval of `[S_u - ε, S_u + ε]` . This will give us a subset of the dataset that have similar similarities with the unit vector.

We can re-query increasing or decreasing ε until the top N results are found.

To further improve accuracy, we can also re-compute the similarity score using cosine similarity with the subset of vectors, which is still much faster then computing the similarity against the entire dataset.

This approach begins to break down as the cosine similarity to the unit vector chosen gets very large (`> 0.4`).  We end up with the possibility of matching against vectors that are of opposite directions – the least similar vectors to the original input vector.

One solution to workaround this could be to pre-compute the similarity of a vector against unit vectors for each dimension of the input vector. But this could be 512 or more cosine similarity calculations for modern embeddings just to precompute the data. Once all unit vector similarities are calculated and stored, the range query against the database would be made against the column for which the input vector’s similarity is closest to 0.

There are a lot of real solutions to this problem, but this was a fun exercise to think about and work on.

## Further reading

Vector similarity search is becoming increasingly popular and integrated into databases. Here are some resources to learn more: [Vector Similarity Search](https://zilliz.com/blog/vector-similarity-search).
