TLDR - You can test my implementation at https://privategpt.baldacchino.net
. If this appears slow to first load, what is happening behind the scenes is a 'cold start' within Azure Container Apps. Cold Starts happen due to a lack of load. To reduce costs, I have configured Azure Container Apps to tear down my container environment when there is no load to zero containers. The loading is due to the container starting (Alpine Linux and NodeJS). Cold Starts are not really a thing for workloads under regular load.
ChatGPT is amazing on so many levels, and it's free. But you know what they say. If something is free, then you are the product!
As part of OpenAI's T&C, your prompts (the questions you ask) will be used to further train OpenAI's LLM (Large Language Model). Are you okay with this valuable stream of data exiting your organisation? If you are reading this and are responsible for your organisaions security posture, how are you ensuring private IP is not being leaked out of your company so that you do not become the next Samsung? In order to prevent this, you can run a hosted version of this every popular LLM using Azure Open AI. There are many benefits of Azure OpenAI but the two I want to highlight are.
See this Microsoft Learn article for more details. Azure OpenAI is not free, but it is almost free and certainly is going to increase your organizations security posture. I say almost free, because the reality is that tokens for Azure OpenAI results in most conversations costing less than 10 cents. A link to Azure OpenAI pricing can be found here. In this blog post we will build a private ChatGPT like interface, to keep your prompts safe and secure using the Azure OpenAI service and a raft of other Azure services to provide you a private ChatGPT like offering.
The Building Blocks We are going do this in a way that is low on administrative effort as a result we will running any servers (IaaS) that we will need to water and feed. No scaling rules, no patching etc, this needs to be simple, so here is what we are going to use.
ghcr.io/mckaywrigley/chatbot-ui:main
, host this in your own container registry or pull from the source.Architecture This really is a simple architecture, whilst it is 12 steps it is an architecture that has been optimised for speed and low administrative effort. It will automatically update when new container versions are released and you have fine grain controls that you can toggle around not only updates but authentication and the ability to prevent public access.
Configuration This is a relatively simple architecture, and unlike a lot of my posts I am not going to go into great amount of detail. I will use the GUI, but of course this can be automated. If you have a question that is not addressed by the sections below, please leave a comment on this article.
ghcr.io/mckaywrigley/chatbot-ui:main
(you can obtain this from the repository at https://github.com/mckaywrigley/chatbot-ui), the port mapping of TCP 80 (HTTP) to TCP 3000 (Container runtime port).I have pasted in images showing ingress configuration (perhaps you want this private) and settings for authentication with a provider (OAuth etc), perhaps you want to restrict access to authorised viewers, ther are options to do so.You will need to define environment variables (ENV) for the container. These environment variables define variables such as the Azure OpenAI Model, Endpoints, Keys and so on.
When deploying the application, the following environment variables can be set:
Environment Variable | Default value | Description |
---|---|---|
OPENAI_API_KEY | The default API key used for authentication with OpenAI | |
OPENAI_API_HOST | https://api.openai.com |
The base url, for Azure use https://<endpoint>.openai.azure.com |
OPENAI_API_TYPE | openai |
The API type, options are openai or azure |
OPENAI_API_VERSION | 2023-03-15-preview |
Only applicable for Azure OpenAI |
AZURE_DEPLOYMENT_ID | Needed when Azure OpenAI, Ref Azure OpenAI API | |
OPENAI_ORGANIZATION | Your OpenAI organization ID | |
DEFAULT_MODEL | gpt-3.5-turbo |
The default model to use on new conversations, for Azure use gpt-35-turbo |
NEXT_PUBLIC_DEFAULT_SYSTEM_PROMPT | see here | The default system prompt to use on new conversations |
NEXT_PUBLIC_DEFAULT_TEMPERATURE | 1 | The default temperature to use on new conversations |
Test - You Are Done I hope you are like me, testing along the way. Test after Azure Container Apps is online, test after Azure Front Door is configured and finally test after you have a private domain. Feel free to have a poke around my instance at https://privategpt.baldacchino.net, I do have API limits which you will experience if you hit this too hard and I am using GPT-35-Turbo
Summary
Take control of your data, you IP and build your own ChatGPT like interface using Azure OpenAI and slew of other Azure services. It really isn't that hard. This quick walkthrough illustrated just how easy it is for you to do. If you are reading this and working for an organisation who is still accessing ChatGPT, I urge you to stop and think about how you can gain control of your data, so that your data doesn't become the product.
Thanks
Shane Baldacchino
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.