# AI crawler guidelines for OpenAlgo
User-agent: *
Allow: /
Crawl-delay: 1
Sitemap: https://openalgo.com/sitemap.xml
Sitemap: https://openalgo.com/sitemaps/core.xml
Sitemap: https://openalgo.com/sitemaps/researchers.xml
Sitemap: https://openalgo.com/sitemaps/reports.xml
Sitemap: https://openalgo.com/sitemaps/events.xml
Contact: mailto:info@openalgo.com
Content: https://openalgo.com/researchers
Content: https://openalgo.com/research-reports
Content: https://openalgo.com/research-areas
Content: https://openalgo.com/ai-progress
Content: https://openalgo.com/events
Content: https://openalgo.com/insights
API: https://openalgo.com/api/search (GET, cacheable, read-only)
API: https://openalgo.com/api/ai-index/researchers (GET, machine-readable, NDJSON)
API: https://openalgo.com/api/ai-index/reports (GET, machine-readable, NDJSON)
API: https://openalgo.com/api/ai-index/events (GET, machine-readable, NDJSON)
API: https://openalgo.com/api/ai-index/highlights (GET, compact summaries)
Updates: researchers=daily; research-reports=weekly; events=weekly; ai-progress=weekly
Policy: embeddings-allowed
Policy: summarization-allowed
Rate-limit: standard=30 requests/minute; heavy=10 requests/minute
Preferred-refresh: 86400 seconds
Data-usage: cite OpenAlgo with source URLs when surfaced in AI answers
Change-notify: https://openalgo.com/api/ai-index/changes (experimental feed)