Trying to deploy chromadb as a Google Cloud Run Service

TL;DR

Deploying chromadb 0.4.18 on Google Cloud Run and hitting HTTPError on client connect plus permissions errors downloading the embedding model: maintainer-verified fixes. 2) Downgrade the Cloud Run build pack to Python 3.11 (3.12 broke it).

Steps

1. Deploying chromadb 0.4.18 on Google Cloud Run and hitting HTTPError on client connect plus permissions errors downloading the embedding model: maintainer-verified fixes. 1) allowreset is a server-side setting: set env var ALLOWRESET=TRUE on the Cloud Run service. 2) Downgrade the Cloud Run build pack to Python 3.11 (3.12 broke it). 3) Install onnxruntime and tokenizers (missing OTEL deps in chromadb-client also caused failures). 4) Set ef.DOWNLOADPATH = "models/ONNXMiniLML6_V2" to a writable path. Caveat: downloading an 80MB model on every cold start is a non-starter on serverless, so for production switch the embedding function to an external API (OpenAI, Cohere, Vertex, or HuggingFace) instead of the local ONNX model.

When to use

You are seeing this: Issue chroma-core/chroma#1429 (closed, 22 comments): ### What happened? Use this skill when you run into "Trying to deploy chromadb as a Google Cloud Run Service".

When not to use

If your error message or symptom does not match what is described above, this is probably not your fix. Search for your exact error text instead of forcing this one to fit.

Versions

Versions mentioned in the source: PYTHON_VERSION 3.10.13, chromadb 0.4.18. If you are on something much newer or older, the details may have shifted.

Why this happens

The original report does not dig into a root cause. It documents the symptom and the fix that resolved it.

Edge cases

• Caveat: downloading an 80MB model on every cold start is a non-starter on serverless, so for production switch the embedding function to an external API (OpenAI, Cohere, Vertex, or HuggingFace) instead of the local ONNX model.

In the GitHub issue gh:chroma-core/chroma#1429 (https://github.com/chroma-core/chroma/issues/1429), the problem reported was: Issue chroma-core/chroma#1429 (closed, 22 comments): ### What happened? Created a service. Deployed directly docker from Docker Hub. Now trying connect via HttpClient: `` chroma_client = chromadb.HttpClient(host="[MY_HOST]", port='443', ssl=True, settings=Settings(allow_reset=True, anonymized_telemetry=False), headers={'authorization': 'PASS'}) ### Versions PYTHON_VERSION=3.10.13 chromadb 0.4.18 ### Relevant log output #### Client size ``shell HTTPError Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/chromadb/api/fastapi.py in raisechromaerror(resp) 627 try: --> 628 resp.raiseforstatus() 629 except requests. The verified resolution was: Deploying chromadb 0.4.18 on Google Cloud Run and hitting HTTPError on client connect plus permissions errors downloading the embedding model: maintainer-verified fixes. 1) allowreset is a server-side setting: set env var ALLOWRESET=TRUE on the Cloud Run service. 2) Downgrade the Cloud Run build pack to Python 3.11 (3.12 broke it). 3) Install onnxruntime and tokenizers (missing OTEL deps in chromadb-client also caused failures). 4) Set ef.DOWNLOADPATH = "models/ONNXMiniLML6_V2" to a writable path. Caveat: downloading an 80MB model on every cold start is a non-starter on serverless, so for production switch the embedding function to an external API (OpenAI, Cohere, Vertex, or HuggingFace) instead of the local ONNX model.