How to secure your OpenAI account
A wrapper to monitor and limit fine-grained API usage, enforce IAM permissions and optionally cache responses.
OpenAI have done an amazing job with ChatGPT, but frankly the luxury of being a monopoly has lead to some corner-cutting on the API design and support.
OpenAI does not currently provide any way to monitor or limit API usage costs by user, project or model. In fact, there is no concept of “project”, only users (which correspond to email addresses), organizations (which correspond to OpenAI accounts and must be individually funded) and API keys (which can be used interchangeably across any organizations to which a user belongs).
This leads to a proliferation of API keys and users opening up a wider attack surface from a security point of view. Furthermore, users cannot be forced to use MFA and may continue to use the API and create API keys, even if their email no longer exists.
Lastly, it is easy to make redundant calls to the API incurring unnecessary costs, especially when developing in an interactive environment such as a Jupyter notebook.
OpenAI Proxy [openai-wrapi]
This repo provides a wrapper which checks usage limits before passing on the request to the OpenAI API and records the usage costs per user, project, model and staging account. It leverages the IAM permission framework of AWS to control access to the OpenAI API, without exposing the unique API keys per staging account. Responses from the OpenAI API are cached by default.
Once it has been deployed, it is a simple matter to use it by running
pip install openai-wrapiand including the following line in your code before importing any packages (such as LangChain) which use the OpenAI package under the hood:
import openai_wrapi as openaiYou no longer need set the OpenAI API key or organization ID as these are securely stored in the corresponding Lambda functions.
Admins
Infrastructure As Code (IAC) is given in the repo to deploy the solution using a serverless architecture in AWS at a minimal extra cost and latency:
- A streaming Lambda function URL to proxy calls to the OpenAI API per staging account (dev, prod).
- A Lambda function to set usage limits and flush the cache per staging account (dev, prod).
- A DynamoDB table to store usage and limits.
- An optional ElastiCache Memcached cluster to cache OpenAI API responses.
Convenience functions are available in the Python package to set usage limits per user, project and model as well as to flush the cache. Also a simple Streamlit app is provided to monitor usage.
If you want to use the proxy from somewhere other than Python, you can use the URL of the Lambda function in place of the OpenAI endpoint, provided you authenticate with AWS appropriately. In fact, you can even make the Lambda function URL public and restrict the access with CORS, so that it can be used directly in a frontend application.
You can find the repo here:
