Rajeev Pentyala – Technical Blog on Power Platform, Azure and AI

The Power CAT Copilot Studio Kit is a comprehensive set of capabilities designed to augment Microsoft Copilot Studio.

The kit helps makers test custom copilots, use large language model to validate AI-generated content, track aggregated key performance indicators.

By running individual tests against the Direct Line API, the copilot responses are evaluated against expected results.

Today, the tool supports following types of tests:

• Response exact match.
• Attachments match.
• Topic match (requires Dataverse enrichment).
• Generative answers (requires AI Builder for response analysis, and Azure Application Insights for details on why an answer was or was not generated).

Setup instructions and documentation

• Prerequisites
• Installation instructions
• Configure users and teams
• Configure copilots
• Configure tests
• Run tests
• Analyze test results
• Troubleshooting

Please refer this link for more details.

🙂