Live from the official portal
Read in real time from the portal's public API (cached for up to 1 hour).
What this is
- Source: transparencia-cubatao.smarapd.com.br (SMARAPD "PAI" portal).
- How: we call the same public JSON API the official site uses in your browser. No scraping first, no login, no private data.
- Where it goes: a nightly job (02:00, Brasília time) stores everything in a PostgreSQL database, starting with contracts and payroll from 2025 on.
- Stack: Python collector (httpx2, tenacity, psycopg) in Docker; this page runs on Cloudflare Pages with small Functions that talk to the APIs.
Roadmap
- Done Step 1 — Skeleton. Project structure, local Docker with PostgreSQL 16, base API client (Referer header, ~1 request/second, retries), CLI stubs, nightly job at 02:00.
- Done Presentation layer. This page on Cloudflare Pages, with an
/apilayer that reads the portal's public API. - Next Step 2 — Contracts. Collect 2025+ contracts and amendments with full descriptions into the database.
- Planned Step 3 — Payroll. Monthly payroll from 2025 on (name, employee id, salary), plus totals by role and department.
- Planned Step 4 — Incremental sync. Only download a table when the portal reports a new update.
- Planned Step 5 — Public site. Simple search, filters and CSV downloads, served from our own API.