Work in progress · Personal project · Synthetic data

Building an AI analyst that asks for evidence before it answers.

A lab project about a practical boundary: the model can decide what narrow evidence it needs, while Python remains responsible for calculating the numbers.

GoalGround analysis in inspectable campaign evidence
My roleProblem framing, tool design, implementation, evaluation
Current stateBounded local data tool built; model loop in progress

A briefing built from fixed weekly and aggregate summaries can look technically successful and still miss a short event hidden inside those summaries.

The useful question became more precise: how can the assistant request a small daily slice when it needs more evidence, without gaining open-ended access to the dataset or inventing the calculation?

The first live briefing produced valid output but missed the planted event in the synthetic reference data. That is analytical failure even though the API call and JSON parsing worked.

I separated the work into two responsibilities:

  • Python calculates. It reads the synthetic CSV, validates the request, aggregates rows by date, and returns daily spend, clicks, and conversions.
  • The model requests and interprets. It may ask for one advertiser, one campaign, and one bounded date range, then explain what the returned evidence supports.

The local data function accepts exact names, inclusive ISO dates, a valid start/end order, and a maximum 31-day range. It is read-only and does not receive the private evaluation answer key.

01User asks an analysis question
02Model chooses a bounded evidence request
03Application validates the arguments
04Python calculates from synthetic CSV data
05Compact daily evidence returns to the model
06Model answers with the evidence and limits

The daily query and its validation work locally. Connecting it to a model-selected tool request, then evaluating the full loop, remains in progress.

  • The reference dataset is synthetic and reproducible enough for a bounded course exercise.
  • A baseline live briefing missed the short planted event, exposing a real context-granularity problem.
  • The local daily query returns deterministic date-level evidence without calling a model.
  • Invalid names, non-ISO dates, reversed ranges, and oversized ranges fail clearly.

This establishes that the narrow data boundary works locally. It does not establish that the complete AI workflow is reliable.

The project uses synthetic data, covers a narrow scenario, has no demonstrated customer or business impact, and is not deployed. Tool-loop behavior and repeatable evaluation still need to be completed.

Next: connect the validated data function to a model-selected request, simulate valid and invalid calls, then measure whether the added evidence improves the missed case without creating unsupported conclusions.

← Back to projects