Long texts, caching, limits and retries

Bound requests, preserve sentence order and retry only errors that can recover.

The current input cap is 20000 Unicode code points, including an emoji as one character. JSON request bodies are capped at 120,000 bytes by default. Limits can be changed by the operator; OpenAPI and response headers describe the deployed settings. Never count UTF-8 bytes or JavaScript UTF-16 .length as equivalent to the text character limit.

The server automatically divides accepted generative text into bounded sequential chunks. Default provider chunks are at most 6,000 code points and 5,000 UTF-8 bytes; per-tool token caps apply to each provider call. Paraphrase/translation preserve chunk order. Generative summaries summarize bounded sections separately. This prevents large input from silently exceeding a provider context window. A request can still time out after four minutes; use smaller client-side sections for large documents.

For input above the API cap, split at sentence/paragraph boundaries, process sections sequentially and join results in order. This complete Node program handles long individual sentences without splitting surrogate pairs:

import {readFile} from 'node:fs/promises';
const key=process.env.METAPHRASE_API_KEY;
if(!key)throw new Error('Set METAPHRASE_API_KEY');
const limit=20000;
const source=await readFile(process.argv[2]??'input.txt','utf8');
const chunks=[];let current='';
for(const {segment} of new Intl.Segmenter(undefined,{granularity:'sentence'}).segment(source)){
  const points=[...segment];
  for(let index=0;index<points.length;index+=limit){
    const part=points.slice(index,index+limit).join('');
    if([...current].length+[...part].length>limit){if(current.trim())chunks.push(current);current='';}
    current+=part;
  }
}
if(current.trim())chunks.push(current);
const results=[];
for(const text of chunks){
  const response=await fetch('https://metaphrase.msalman.com/api/v1/paraphrase',{method:'POST',headers:{Authorization:'Bearer '+key,'Content-Type':'application/json'},body:JSON.stringify({text,mode:'standard'}),signal:AbortSignal.timeout(250000)});
  const value=await response.json();
  if(!response.ok)throw new Error(value.error.code+': '+value.error.message+' '+value.error.docs_url);
  results.push(value.result.text);
}
console.log(results.join('\n\n'));

Save as long-text.mjs; run node long-text.mjs input.txt. Each section consumes its own accepted-call quota. Very large numbers of tiny requests cost more in prompt overhead, so avoid splitting every sentence into a separate API call.

Cache and usage

Identical tool inputs share a private 24-hour result cache. Streaming and JSON forms share the same generative result. Cache hits have usage.cached=true, zero new tokens and zero new provider cost; they still consume a key's request quota. Cache admission may be skipped under memory pressure, so repeated requests are not guaranteed to hit cache. Concurrent identical work may return 429 with Retry-After: 2 rather than start a second paid provider call.

Usage is exact when the provider reports tokens; otherwise usage_estimated=true. Accepted failed or cancelled upstream attempts may be billed. A provider fallback can therefore include measured/estimated units from more than one attempt. estimated_cost_usd uses operator-configured per-million token prices and is an estimate, not a payment charge or invoice. Free grammar, citation and extractive summaries have zero LLM tokens.

Retry policy

• invalid_request, invalid_json, invalid_source, text_too_long and text_too_short: correct input before retrying.
• unauthorized: replace the key; do not loop on the same revoked key.
• provider_not_configured: operator configuration is required; changing your key does not fix it.
• rate_limited: wait the response's Retry-After seconds and add jitter. It can mean per-key rate, duplicate work, or occupied shared slots.
• quota_exceeded / budget_exceeded: wait for reset or ask the operator. Do not repeatedly retry a monthly/daily cap.
• upstream_unavailable, service_unavailable: use bounded exponential backoff; a later retry may succeed.
• upstream_timeout, upstream_error, output_truncated: preserve any partial preview and retry a shorter section as a separate request. An earlier attempt may have incurred tokens.

The official SDK does not automatically retry paid work. A simple bounded helper for ordinary JSON calls:

import {setTimeout as delay} from 'node:timers/promises';
const key=process.env.METAPHRASE_API_KEY;
if(!key)throw new Error('Set METAPHRASE_API_KEY');
async function callWithRetry(payload){
  for(let attempt=0;attempt<3;attempt++){
    const response=await fetch('https://metaphrase.msalman.com/api/v1/paraphrase',{method:'POST',headers:{Authorization:'Bearer '+key,'Content-Type':'application/json'},body:JSON.stringify(payload),signal:AbortSignal.timeout(250000)});
    const value=await response.json();
    if(response.ok)return value;
    const code=value.error?.code;
    if(!['rate_limited','upstream_unavailable','service_unavailable'].includes(code)||attempt===2)throw new Error(JSON.stringify(value.error));
    const retry=Number(response.headers.get('retry-after'));
    const wait=Number.isFinite(retry)&&retry>0?retry*1000:500*2**attempt;
    if(wait>60000)throw new Error(JSON.stringify(value.error));
    await delay(wait+Math.random()*250);
  }
}
console.log(await callWithRetry({text:'A clear idea deserves careful expression.'}));

Save as retry.mjs; run node retry.mjs. No outgoing webhooks are implemented. Use ordinary HTTP responses or SSE; Zapier/Make can trigger an incoming workflow and then call this API.