Community Discussion
Best practices for fetching realtime campaign content metrics with hourly granularity
3 days ago
Hi Taboola Team,
We're using the Realtime Top Campaign Content API endpoint:
GET /backstage/api/1.0/account_id/reports/realtime-top-campaign-content/dimensions/by_item?start_date=YYYY-MM-DDTHH:mm:ss&end_date=YYYY-MM-DDTHH:mm:ss
We'd like to clarify a few points and ask for recommendations.
Our use case:
We need to sync ad metrics with hourly granularity for internal reporting and performance tracking. Our setup: a single network account contains approximately 40 ad accounts. We run an hourly cron job (with 10-minute max execution time) to fetch metrics for all ad accounts under this network account.
Challenges we're facing:
- Rate limiting (HTTP 429) - Since all ad account requests use the same access token (obtained for the network account), we believe the rate limit applies at the network account level. With 40 ad accounts requiring hourly metrics, we quickly hit rate limits. We currently wait 8 seconds between requests and 61 seconds on 429 responses, but this significantly extends sync time - often exceeding our 10-minute execution window.
- No pagination or cursor support - The endpoint returns up to 1000 items per request. If results exceed this limit, there's no way to fetch the remaining data for that time range.
Questions:
- Rate limit details: What are the exact rate limits for this endpoint? Is the limit per network account (access token) or per ad account? What's the recommended delay between requests to avoid 429 errors?
- Pagination/cursors: Are there plans to add pagination support? This would help when results exceed 1000 items.
- Metrics aggregation delay: How long after an hour ends should we wait before fetching metrics? We currently use a 1-hour buffer (e.g., at 15:30 we fetch data up to 14:00). Is this sufficient for accurate data, or would 30 minutes work?
- Recommended approach: Given our scale (40 ad accounts under one network account, hourly sync), what's the recommended approach to efficiently fetch hourly metrics without hitting rate limits?
Any guidance would be greatly appreciated.
Thank you!
