We’re setting up a BigQuery table for event data, and records can arrive hours or occasionally days after they were generated. Most reports filter by the event’s timestamp, but our ingestion pipeline also needs to find recently loaded rows.
Would you partition on the event timestamp and accept that late records land in older partitions, or use ingestion time and rely on filters for reporting? I’m also curious whether clustering changes the trade-off much for a table that grows by several million rows a day.
Can you answer this question?
Write Answer0 Answers