Skip to content

[bot] Merge master/a0e06bc5 into rel/dev - #1754

Merged
yenkins-admin merged 2 commits into
rel/devfrom
snapshot-master-a0e06bc5-to-rel/dev
Aug 24, 2026
Merged

[bot] Merge master/a0e06bc5 into rel/dev#1754
yenkins-admin merged 2 commits into
rel/devfrom
snapshot-master-a0e06bc5-to-rel/dev

Conversation

@yenkins-admin

Copy link
Copy Markdown
Contributor

🚀 Automated PR to perform merge from master into rel/dev with changes up to a0e06bc (created by https://github.com/gooddata/gooddata-python-sdk/actions/runs/32703319578).

… failed one

_extract_metric_result (and conversation.py's own copy of it) returned on
the first create_metric tool call in a turn, regardless of isError. When
the agent self-corrects an invalid MAQL by retrying create_metric within
the same turn, this captured the failed first attempt's error payload
instead of the successful retry's payload:

- In conversation.py, turn_outputs then had no metric_id, so any later
  fixture turn referencing $ref:create_metric.* raised inside
  _resolve_refs and was silently [SKIP]ped -- never sent to the server.
- In metric_skill.py, the run reported maql_correct=False even though
  the metric was created, and metric_id_to_delete stayed unset, so the
  metric the successful retry created leaked into the shared workspace.

Fix, consolidated into one implementation that conversation.py now
imports instead of keeping its own copy:
- Skip a create_metric result when the decoded payload is not a dict,
  is an error, or is empty -- callers unconditionally call .get() on it.
- Prefer the most recent successful call in the turn.
- conversation._check_output_present's metric branch now requires an
  actually-extracted result, not just any create_metric call having been
  made -- a turn where every attempt failed no longer reports success
  while the real failure surfaces as the next turn being skipped.
- metric_skill._execute_single_metric_run now tracks every metric id any
  create_metric call in the run produced (via the existing
  _extract_created_metric_ids) and deletes all of them, instead of only
  the one derived from the (possibly failed) primary candidate.

evaluators/metric_skill.py's _find_create_metric has the same first-match
pattern but is only reachable from gooddata-eval's standalone CLI, not
from the ai-agent-tests-staging.yml pipeline -- left as-is.

Verified locally against gdc-nas: ran the full agent_metric_skill and
agent_conversations suites (18 fixtures) in parallel the way daily CI
does; the workspace's metric list matched its pre-run baseline afterward.

JIRA: QA-29053
risk: low
…ction

fix(gooddata-eval): keep retrying create_metric result, not the first failed one
@yenkins-admin
yenkins-admin merged commit 3769281 into rel/dev Aug 24, 2026
1 check passed
@yenkins-admin
yenkins-admin deleted the snapshot-master-a0e06bc5-to-rel/dev branch August 24, 2026 07:50
@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 49add6d3-06e5-411d-ac28-d383eb22a6c2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 80.20%. Comparing base (fa00fc3) to head (a0e06bc).
⚠️ Report is 572 commits behind head on rel/dev.

Files with missing lines Patch % Lines
...val/src/gooddata_eval/core/agentic/conversation.py 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           rel/dev    #1754      +/-   ##
===========================================
+ Coverage    80.16%   80.20%   +0.03%     
===========================================
  Files          272      272              
  Lines        19120    19117       -3     
===========================================
+ Hits         15327    15332       +5     
+ Misses        3793     3785       -8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants