Work in progress · Personal project · Synthetic data
Building an AI analyst that asks for evidence before it answers.
A lab project about a practical boundary: the model can decide what narrow evidence it needs, while Python remains responsible for calculating the numbers.
A briefing built from fixed weekly and aggregate summaries can look technically successful and still miss a short event hidden inside those summaries.
The useful question became more precise: how can the assistant request a small daily slice when it needs more evidence, without gaining open-ended access to the dataset or inventing the calculation?
The first live briefing produced valid output but missed the planted event in the synthetic reference data. That is analytical failure even though the API call and JSON parsing worked.
I separated the work into two responsibilities:
- Python calculates. It reads the synthetic CSV, validates the request, aggregates rows by date, and returns daily spend, clicks, and conversions.
- The model requests and interprets. It may ask for one advertiser, one campaign, and one bounded date range, then explain what the returned evidence supports.
The local data function accepts exact names, inclusive ISO dates, a valid start/end order, and a maximum 31-day range. It is read-only and does not receive the private evaluation answer key.
The daily query and its validation work locally. Connecting it to a model-selected tool request, then evaluating the full loop, remains in progress.
- The reference dataset is synthetic and reproducible enough for a bounded course exercise.
- A baseline live briefing missed the short planted event, exposing a real context-granularity problem.
- The local daily query returns deterministic date-level evidence without calling a model.
- Invalid names, non-ISO dates, reversed ranges, and oversized ranges fail clearly.
This establishes that the narrow data boundary works locally. It does not establish that the complete AI workflow is reliable.
The project uses synthetic data, covers a narrow scenario, has no demonstrated customer or business impact, and is not deployed. Tool-loop behavior and repeatable evaluation still need to be completed.
Next: connect the validated data function to a model-selected request, simulate valid and invalid calls, then measure whether the added evidence improves the missed case without creating unsupported conclusions.
← Back to projects