Reliability guide
Google Search API retries and rate limits
A reliable search integration retries only temporary failures, waits between attempts, and stops after a small retry budget. Invalid requests should be fixed instead of repeated.
Published by Mentorsko Ltd on . Last reviewed .
Retry only temporary responses
Retry HTTP 429, 502, 503, and 504 when the calling workflow can wait. A 429 response means a request limit was reached, while the 5xx responses represent temporary service or deadline failures.
Do not automatically retry a timeout or network error when no HTTP response was received. The first request may have completed and used credits even though its response was lost. Check the account activity before deciding whether to retry manually.
Do not retry HTTP 400, 401, 402, 413, or 415 until the request, credentials, available credits, body size, or content type has been corrected.
Use a capped retry loop
The example retries only explicit HTTP 429, 502, 503, and 504 responses. It makes no more than three attempts. Each attempt scales with the requested page count, and the total deadline is three times that attempt limit. It waits for the complete numeric Retry-After delay when that delay fits.
Node.js retry helper ยท javascript
const retryableStatuses = new Set([429, 502, 503, 504]);
const maxAttempts = 3;
function unknownTransportOutcome(error) {
const detail = error instanceof Error ? error.message : String(error);
return new Error(
"Search outcome is unknown after a network or timeout error. " +
"The request may have completed and used credits. Do not retry automatically. " + detail
);
}
async function waitForRetry(delayMs, deadline, requestId) {
if (delayMs >= deadline - Date.now()) {
throw new Error("Retry delay exceeds search deadline (" + requestId + ")");
}
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
async function searchWithRetry(input) {
const apiKey = process.env.GSEARCH_API_KEY;
if (!apiKey) throw new Error("GSEARCH_API_KEY is not set");
const pages = input.pages ?? 1;
const attemptTimeoutMs = Math.ceil(pages / 10)
* 30_000
+ 15_000;
const totalTimeoutMs = attemptTimeoutMs * maxAttempts;
const deadline = Date.now() + totalTimeoutMs;
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
const remainingMs = deadline - Date.now();
if (remainingMs <= 0) throw new Error("Search deadline exceeded");
let response;
let body = null;
try {
response = await fetch("https://gsearch.dev/api/v1/search", {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-API-Key": apiKey
},
body: JSON.stringify(input),
signal: AbortSignal.timeout(Math.min(attemptTimeoutMs, remainingMs))
});
try {
body = await response.json();
} catch (error) {
if (response.ok) throw unknownTransportOutcome(error);
}
} catch (error) {
throw unknownTransportOutcome(error);
}
if (response.ok) {
if (!body || !Array.isArray(body.organic)) {
throw new Error("Search API returned an invalid JSON response");
}
return body;
}
const requestId = body?.error?.requestId ?? "no request ID";
if (!retryableStatuses.has(response.status) || attempt === maxAttempts - 1) {
throw new Error("Search failed with HTTP " + response.status + " (" + requestId + ")");
}
const retryAfterSeconds = Number.parseInt(
response.headers.get("retry-after") ?? "",
10
);
const delayMs = Number.isFinite(retryAfterSeconds) && retryAfterSeconds >= 0
? retryAfterSeconds * 1_000
: 500 * 2 ** attempt;
await waitForRetry(delayMs, deadline, requestId);
}
}Keep retry traffic inside the plan limit
Every plan has separate shared account limits for requests and requested pages per minute. A retry counts as one more request and adds its full page count to the requested-page limit, so keep the attempt count small and avoid starting the same retry loop in many workers at once.
Set a timeout for every attempt and a wider deadline for the complete user action. Stop retrying when the wider deadline is nearly reached.
- Cap retries at a small fixed number.
- Increase the delay between attempts.
- Respect response rate-limit headers.
- Do not automatically retry when no HTTP response was received.
- Avoid retrying the same request from several workers at once.
Keep useful diagnostics without exposing the key
Error responses include code, message, and requestId. Log the HTTP status, error code, requestId, and attempt number, but never log the API key.
Explicit failed HTTP responses do not use search credits. A transport failure has an unknown outcome because the request may have completed after the response was lost. Each requested page in a successful response uses one credit, including a page served from cache.