31 Oct
|
Algoworks Solutions
|
Noida
31 Oct
Algoworks Solutions
Noida
About the Company -
Algoworks is an award-winning artificial intelligence, engineering services and experience
transformation firm with offices across the United States, Europe, South America and
India. We bring together a global team of engineers, architects, designers, researchers and
operators united by rigor, accountability and a commitment to delivering measurable
results.
For over 20 years, Algoworks has partnered with Fortune 500 organizations across the
Americas, Europe and Asia to define, build and run technology that drives meaningful
business outcomes. Our work combines human-centered design, engineering excellence
and AI-powered capabilities to solve complex challenges with clarity and precision.
Innovation, particularly in the responsible application of AI, is embedded in how teams
approach problem-solving and continuous improvement.
At Algoworks, growth is continuous and closely tied to impact. Teams collaborate across
geographies and disciplines, strengthening outcomes through shared insight and
collective expertise. The culture values transparency, open dialogue and an environment
where every voice is heard and contribution is recognized.
Through collaboration, accountability and a focus on results, Algoworks operates at the
intersection of technology and people, building not only advanced systems but strong
global teams that elevate performance and create lasting impact.
Role Overview-
We are seeking an ETL QA Data Engineer to ensure the accuracy, completeness, and
reliability of data pipelines and analytics outputs across high-volume, event-driven
domains such as emergency response, public safety, and healthcare operations. The role
involves validating data ingestion and transformation logic, reconciling records between
source and target systems, detecting anomalies, and ensuring reporting and dashboards
reflect trustworthy data.
Key Responsibilities-
1. Data Validation & Reconciliation
• Perform source-to-target reconciliation for event- and transaction-style datasets
across ingestion and curated layers.
• Validate completeness, timeliness, and consistency of daily and near-real-time
ingestion pipelines.
• Identify and investigate data anomalies, including missing records, spikes or drops,
duplicates, out-of-range values, and schema drift.
• Validate the correctness of KPIs and metrics used in dashboards and reports,
including volumes, rates, turnaround times, and error trends.
• Ensure referential integrity across key entities such as events/encounters,
units/resources, locations, and agencies/sites.
• Partner with engineering teams to debug discrepancies, identify root causes, and
validate fixes.
2. Automation & Framework Development
• Build and maintain automated data validation frameworks using Python and testing
frameworks such as PyTest or Unittest.
• Develop reusable validation assets, including reconciliation templates, data quality
rules, and regression suites for critical pipelines.
• Create data quality rules covering nulls, ranges, uniqueness, and referential
integrity.
• Integrate validation into CI/CD and orchestration workflows, including validation
gates for pipeline runs.
• Produce validation artifacts such as test reports, defect summaries, and data
quality scorecards for ongoing monitoring.
3. AI-Assisted Validation
• Use AI tools and LLM-based solutions, such as Qodo, to accelerate validation query
generation, reconciliation logic, and test case creation.
• Leverage AI-assisted techniques to identify edge cases, explore anomaly detection
ideas, and expand test coverage.
• Validate all AI-generated outputs against actual data and documented business
rules before adoption.
4. Data Quality & Reporting Reliability
• Ensure reporting layers and dashboards accurately reflect validated source data
and business definitions.
• Identify and report data quality issues that may affect downstream analytics and
operational decision-making.
• Support the development of repeatable and documented validation rules aligned
with business metrics.
• Contribute to early detection of anomalies and reduction of downstream reporting
issues.
5. Collaboration & Continuous Improvement
• Collaborate with data engineering, analytics, and other relevant teams to
investigate pipeline failures and data discrepancies.
• Support incident triage and fix verification to improve resolution efficiency.
• Contribute to continuous improvement of data testing practices, validation
frameworks, and quality monitoring.
Required Technical Skills & Competencies
• Strong SQL skills for data reconciliation and validation, including joins,
aggregations, window functions, and data profiling.
• Hands-on Python experience for test automation and data validation.
• Experience in ETL/ELT QA, data testing, analytics validation, or data quality
engineering.
• Understanding of event-driven and transaction-based systems and common
pipeline failure patterns, including late-arriving data, corrections, and duplicates.
• Experience validating large datasets and reporting layers in environments with high
reliability expectations.
Must-Have Skills-
• SQL
• Python
• ETL/ELT QA
• Data Validation & Reconciliation
• Data Quality Testing
• Test Automation
• Source-to-Target Validation
• Data Anomaly Investigation
Positive to Have Skills-
• Great Expectations or custom Python validation frameworks.
• dbt tests or similar data validation frameworks.
• Familiarity with cloud data platforms such as Snowflake, BigQuery, or Redshift.
• Exposure to Databricks or Spark.
• Familiarity with orchestration tools such as Airflow, Azure Data Factory (ADF), or
Prefect.
• Experience with sensitive data handling and privacy/security considerations.
• Exposure to interoperability and integration patterns such as APIs, HL7/FHIR, EDI,
and batch feeds, as applicable to the environment.
• Familiarity with AI-assisted testing tools and LLM-based validation support.
Desired Attributes-
• Strong analytical and problem-solving skills with attention to data accuracy and detail.
• Ability to investigate complex data discrepancies and identify root causes.
• A quality-focused mindset with an emphasis on data reliability and reporting integrity.
• Ability to collaborate effectively with engineering and analytics teams.
• Commitment to documenting validation rules and maintaining reusable testing assets.
• Sound judgment when using AI-assisted tools, with a focus on human verification.
What Success Looks Like-
• Automated validation coverage for critical pipelines, including freshness, completeness, and reconciliation.
• Early detection of anomalies and measurable reduction in downstream reporting issues.
• Repeatable, documented validation rules aligned with business definitions and metrics.
• Efficient collaboration with engineering teams to shorten incident triage and fix verification cycles.
📌 ETL QA Data Engineer (Noida)
🏢 Algoworks Solutions
📍 Noida