30 Oct
|
Algoworks
|
Noida
About the Company
- Algoworks is an award-winning artificial intelligence, engineering services and experience transformation firm with offices across the United States, Europe, South America and India. We bring together a global team of engineers, architects, designers, researchers and operators united by rigor, accountability and a commitment to delivering measurable results. For over 20 years, Algoworks has partnered with Fortune 500 organizations across the Americas, Europe and Asia to define, build and run technology that drives meaningful business outcomes. Our work combines human-centered design, engineering excellence and AI-powered capabilities to solve complex challenges with clarity and precision. Innovation, particularly in the responsible application of AI, is embedded in how teams approach problem-solving and continuous improvement. At Algoworks, growth is continuous and closely tied to impact. Teams collaborate across geographies and disciplines, strengthening outcomes through shared insight and collective expertise. The culture values transparency, open dialogue and an workplace where every voice is heard and contribution is recognized. Through collaboration, accountability and a focus on results, Algoworks operates at the intersection of technology and people, building not only advanced systems but strong global teams that elevate performance and create lasting impact. Role Overview
- We are seeking an ETL QA Data Engineer to ensure the accuracy, completeness, and reliability of data pipelines and analytics outputs across high-volume, event-driven domains such as emergency response, public safety, and healthcare operations. The role involves validating data ingestion and transformation logic, reconciling records between source and target systems, detecting anomalies, and ensuring reporting and dashboards reflect trustworthy data.
Key Responsibilities
- 1. Data Validation & Reconciliation
• Perform source-to-target reconciliation for event
- and transaction-style datasets across ingestion and curated layers.
• Validate completeness, timeliness, and consistency of daily and near-real-time ingestion pipelines.
• Identify and investigate data anomalies, including missing records, spikes or drops, duplicates, out-of-range values, and schema drift.
• Validate the correctness of KPIs and metrics used in dashboards and reports, including volumes, rates, turnaround times, and error trends.
• Ensure referential integrity across key entities such as events/encounters, units/resources, locations, and agencies/sites.
• Partner with engineering teams to debug discrepancies, identify root causes, and validate fixes. 2. Automation & Framework Development
• Build and maintain automated data validation frameworks using Python and testing frameworks such as PyTest or Unittest.
• Develop reusable validation assets, including reconciliation templates, data quality rules, and regression suites for critical pipelines.
• Create data quality rules covering nulls, ranges, uniqueness, and referential integrity.
• Integrate validation into CI/CD and orchestration workflows, including validation gates for pipeline runs.
• Produce validation artifacts such as test reports, defect summaries, and data quality scorecards for ongoing monitoring. 3. AI-Assisted Validation
• Use AI tools and LLM-based solutions, such as Qodo, to accelerate validation query generation, reconciliation logic, and test case creation.
• Leverage AI-assisted techniques to identify edge cases, explore anomaly detection ideas, and expand test coverage.
• Validate all AI-generated outputs against actual data and documented business rules before adoption. 4. Data Quality & Reporting Reliability
• Ensure reporting layers and dashboards accurately reflect validated source data and business definitions.
• Identify and report data quality issues that may affect downstream analytics and operational decision-making.
• Support the development of repeatable and documented validation rules aligned with business metrics.
• Contribute to early detection of anomalies and reduction of downstream reporting issues. 5. Collaboration & Continuous Improvement
• Collaborate with data engineering, analytics, and other relevant teams to investigate pipeline failures and data discrepancies.
• Support incident triage and fix verification to improve resolution efficiency.
• Contribute to continuous improvement of data testing practices, validation frameworks, and quality monitoring. Required Technical Skills & Competencies
• Strong SQL skills for data reconciliation and validation, including joins, aggregations, window functions, and data profiling.
• Hands-on Python experience for test automation and data validation.
• Experience in ETL/ELT QA, data testing, analytics validation, or data quality engineering.
• Understanding of event-driven and transaction-based systems and common pipeline failure patterns, including late-arriving data, corrections, and duplicates.
• Experience validating large datasets and reporting layers in environments with high reliability expectations. Must-Have Skills
-
• SQL
• Python
• ETL/ELT QA
• Data Validation & Reconciliation
• Data Quality Testing
• Test Automation
• Source-to-Target Validation
• Data Anomaly Investigation Good to Have Skills
-
• Great Expectations or custom Python validation frameworks.
• dbt tests or similar data validation frameworks.
• Familiarity with cloud data platforms such as Snowflake, BigQuery, or Redshift.
• Exposure to Databricks or Spark.
• Familiarity with orchestration tools such as Airflow, Azure Data Factory (ADF), or Prefect.
• Experience with sensitive data handling and privacy/security considerations.
• Exposure to interoperability and integration patterns such as APIs, HL7/FHIR, EDI, and batch feeds, as applicable to the environment.
• Familiarity with AI-assisted testing tools and LLM-based validation support. Desired Attributes
-
• Strong analytical and problem-solving skills with attention to data accuracy and detail.
• Ability to investigate complex data discrepancies and identify root causes.
• A quality-focused mindset with an emphasis on data reliability and reporting integrity.
• Ability to collaborate effectively with engineering and analytics teams.
• Commitment to documenting validation rules and maintaining reusable testing assets.
• Sound judgment when using AI-assisted tools, with a focus on human verification. What Success Looks Like
-
• Automated validation coverage for critical pipelines, including freshness, completeness, and reconciliation.
• Early detection of anomalies and measurable reduction in downstream reporting issues.
• Repeatable, documented validation rules aligned with business definitions and metrics.
• Efficient collaboration with engineering teams to shorten incident triage and fix verification cycles.
📌 ETL QA Data Engineer (Noida)
🏢 Algoworks
📍 Noida