Reliability guide
Google Search API retries and rate limits
A reliable search integration retries only temporary failures, waits between attempts, and stops after a small retry budget. Invalid requests should be fixed instead of repeated.
Published by Mentorsko Ltd on . Last reviewed .
Retry only temporary responses
Retry HTTP 429, 502, 503, and 504 when the calling workflow can wait. A 429 response means a request limit was reached, while the 5xx responses represent temporary service or deadline failures.
A timeout or temporary network error can also be retried when the total request deadline still leaves enough time for another attempt.
Do not retry HTTP 400, 401, 402, 413, or 415 until the request, credentials, available credits, body size, or content type has been corrected.
Use a capped retry loop
The example makes no more than three attempts within a 30 second total deadline. It waits for the complete numeric Retry-After delay when that delay fits. If the delay is longer than the remaining deadline, it stops instead of retrying early.
Node.js retry helper ยท javascript
const retryableStatuses = new Set([429, 502, 503, 504]);
const maxAttempts = 3;
const attemptTimeoutMs = 10_000;
const totalTimeoutMs = 30_000;
function isTemporaryFetchError(error) {
return error instanceof TypeError ||
(error instanceof Error && ["AbortError", "TimeoutError"].includes(error.name));
}
async function waitForRetry(delayMs, deadline, requestId) {
if (delayMs >= deadline - Date.now()) {
throw new Error("Retry delay exceeds search deadline (" + requestId + ")");
}
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
async function searchWithRetry(input) {
const apiKey = process.env.GSEARCH_API_KEY;
if (!apiKey) throw new Error("GSEARCH_API_KEY is not set");
const deadline = Date.now() + totalTimeoutMs;
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
const remainingMs = deadline - Date.now();
if (remainingMs <= 0) throw new Error("Search deadline exceeded");
let response;
let body = null;
try {
response = await fetch("https://gsearch.dev/api/v1/search", {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-API-Key": apiKey
},
body: JSON.stringify(input),
signal: AbortSignal.timeout(Math.min(attemptTimeoutMs, remainingMs))
});
try {
body = await response.json();
} catch (error) {
if (isTemporaryFetchError(error)) throw error;
}
} catch (error) {
if (!isTemporaryFetchError(error) || attempt === maxAttempts - 1) throw error;
await waitForRetry(500 * 2 ** attempt, deadline, "no request ID");
continue;
}
if (response.ok) {
if (!body || !Array.isArray(body.organic)) {
throw new Error("Search API returned an invalid JSON response");
}
return body;
}
const requestId = body?.error?.requestId ?? "no request ID";
if (!retryableStatuses.has(response.status) || attempt === maxAttempts - 1) {
throw new Error("Search failed with HTTP " + response.status + " (" + requestId + ")");
}
const retryAfterSeconds = Number.parseInt(
response.headers.get("retry-after") ?? "",
10
);
const delayMs = Number.isFinite(retryAfterSeconds) && retryAfterSeconds >= 0
? retryAfterSeconds * 1_000
: 500 * 2 ** attempt;
await waitForRetry(delayMs, deadline, requestId);
}
}Keep retry traffic inside the plan limit
Every plan has a shared account request limit per minute. A retry is another request, so keep the attempt count small and avoid starting the same retry loop in many workers at once.
Set a timeout for every attempt and a wider deadline for the complete user action. Stop retrying when the wider deadline is nearly reached.
- Cap retries at a small fixed number.
- Increase the delay between attempts.
- Respect response rate-limit headers.
- Avoid retrying the same request from several workers at once.
Keep useful diagnostics without exposing the key
Error responses include code, message, and requestId. Log the HTTP status, error code, requestId, and attempt number, but never log the API key.
Failed requests do not use a search credit. A successful response uses one credit, including a response served from cache.