Role Summary

We are looking for a Power BI SDET who combines strong manual testing fundamentals with a growing automation and AI-assisted quality engineering practice. You will own the quality of Power BI semantic models, reports, and data pipelines for lululemon's retail analytics, making sure the numbers business users rely on are accurate, performant, and trusted.

You will also help shape how the team uses Microsoft Copilot (Power BI and Fabric) and agentic AI to validate reports, generate test assets, and speed up regression cycles, while testing the AI-driven experiences themselves.

Key Responsibilities

Manual and functional testing (core of the role)

- Design and execute test plans, cases, and exploratory sessions for Power BI reports, dashboards, and semantic models.
- Validate DAX measures, calculated columns, KPIs, and business logic against source systems and business definitions (sell-through, units per transaction, inventory turns, comp sales, margin, and similar).
- Perform data reconciliation between source systems (ERP, POS, e-commerce, warehouse) and the reporting layer.
- Test row-level security (RLS) and object-level security by persona, such as store manager, regional lead, merchandiser, and executive.
- Test slicers, drill-through, bookmarks, cross-filtering, tooltips, and visual behavior across browsers, devices, and the Power BI mobile app.
- Validate refresh behavior (scheduled and incremental), gateway connectivity, and data freshness.
- Log, triage, and track defects in Azure DevOps or Jira, with clear repro steps and business impact.
- Run UAT with business stakeholders and sign off on release readiness.

Test automation

- Build automated data validation and regression suites using SQL, Python, and/or PySpark.
- Automate semantic model testing with DAX queries, Tabular Editor and Best Practice Analyzer rules, and the XMLA endpoint.
- Integrate automated checks into CI/CD pipelines (Azure DevOps or GitHub Actions) with Power BI deployment pipelines and Git-based source control (PBIP / TMDL).
- Automate UI and report smoke tests where practical (Playwright, Selenium, or equivalent).




- Establish performance baselines using Performance Analyzer, DAX Studio, and Fabric capacity metrics.

AI-infused quality engineering

- Copilot: Use Copilot in Power BI / Fabric to accelerate test design, DAX review, measure explanation, and report summaries, and independently verify its outputs.
- Testing AI outputs: Test Copilot-generated content (narratives, DAX suggestions, Q&A; responses) for accuracy, hallucination, consistency, and security/data-exposure risks.
- Agentic AI: Apply agentic patterns, such as agents that generate test cases from requirements, run reconciliation checks, triage failures, or draft defect reports, with a human in the loop.
- Guardrails: Help define guardrails, evaluation criteria, and acceptance thresholds for AI-assisted analytics features.
- Responsible AI: Document responsible-AI practices for testing, including data privacy, prompt and output logging, and bias checks.

Collaboration

- Work with BI developers, data engineers, product owners, and lululemon business stakeholders in an Agile/Scrum environment.
- Contribute to test strategy, definition of done, and quality gates for the analytics platform.
- Mentor junior testers on moving from manual to automated and AI-assisted approaches.

Required Qualifications

- 5+ years in software or data testing, with substantial hands-on manual testing experience (functional, regression, exploratory, UAT).
- Practical experience testing Power BI reports and semantic models, including DAX, RLS, and refresh/deployment behavior.
- Strong SQL skills for data validation and reconciliation.
- Solid understanding of data warehousing and BI concepts (star schema, ETL/ELT, dimensions and facts, slowly changing dimensions).




- Experience with defect and test management tools (Azure DevOps, Jira, or Xray).
- Working knowledge of at least one scripting language, preferably Python, for test automation.
- Exposure to CI/CD concepts and Agile delivery.
- AI: Working knowledge of Copilot (Power BI, Microsoft 365, or GitHub Copilot) and a explicit understanding of agentic AI concepts: agents, tools, orchestration, human-in-the-loop, and evaluation.
- Excellent communication skills and comfort working directly with client stakeholders.

Preferred Qualifications

- Microsoft Fabric experience (Lakehouse, Warehouse, Dataflows Gen2, Pipelines, Direct Lake).
- Experience with Tabular Editor, DAX Studio, ALM Toolkit, or Power BI REST/XMLA APIs.
- Hands-on experience with agent frameworks or tooling (Copilot Studio, Azure AI Foundry, Semantic Kernel, AutoGen, LangChain) or prompt-based test generation.
- Experience with data quality frameworks (Great Expectations, dbt tests, Soda).
- Retail, e-commerce, or consumer brand analytics experience (merchandising, inventory, omni-channel, loyalty).
- Certifications: PL-300 (Power BI Data Analyst), DP-600 (Fabric Analytics Engineer), AI-900 or AI-102, ISTQB.

What Success Looks Like (First 6-12 Months)

- A documented, repeatable Power BI test strategy covering manual, automated, and AI-assisted validation.
- Measurable reduction in report defects reaching production and in regression cycle time.
- Automated data reconciliation checks running in the deployment pipeline.
- A governed approach to testing Copilot and AI-generated analytics, with defined accuracy and safety criteria.
- Stakeholder trust in the numbers, reflected in fewer "which number is right?" escalations.

Tools and Technologies

BI and data platform

Power BI Desktop/Service, Microsoft Fabric, DAX, SQL

Automation and scripting

Python, PySpark, Playwright / Selenium

Model and performance tooling

Tabular Editor, DAX Studio, ALM Toolkit, Performance Analyzer

DevOps and test management

Azure DevOps, GitHub, Jira / Xray

AI and Copilot

Copilot (Power BI, Fabric, GitHub, Copilot Studio), Azure AI Foundry

📌 Power BI SDET / Test Automation Engineer (Bengaluru)
🏢 Orcapod Consulting Services
📍 Bengaluru