Overview:
Sr Data Engineer

Position Summary

We are looking for a self-sufficient BI Analyst / Data Engineer to join a small, high-impact analytics team. In this role you will own the full lifecycle of business intelligence delivery—from building and maintaining ETL pipelines in AWS to designing interactive dashboards in Amazon QuickSight. You will work closely with onshore stakeholders to translate business requirements into reliable, well-documented data products that drive operational and strategic decision-making.
- Core Responsibilities

ETL / Data Pipeline Development

Design, build, and maintain AWS Glue (PySpark) ETL jobs that ingest data from multiple source systems such as Salesforce, JIRA, Azure DevOps, and Zendesk into a layered data lake architecture (raw core
- presentation).
- Develop and optimize AWS Step Functions workflows for pipeline orchestration, including parallel execution, error handling, retry logic, and concurrency management.
- Implement incremental loading strategies, schema evolution handling, and data quality checks within Glue scripts.




- Work with Apache Iceberg table format on S3 for ACID-compliant analytics tables, including MERGE operations, compaction, snapshot expiration, and orphan file cleanup.
- Integrate supporting AWS services such as EventBridge, SQS, SNS, Athena, and CloudWatch into pipeline architectures.
- Troubleshoot production pipeline failures, diagnose performance bottlenecks (OOM errors, API timeouts, DPU tuning), and implement fixes with mínimal guidance.
- Write and maintain transparent technical documentation for all pipeline components.
- Dashboard & Reporting Development
- Build, publish, and maintain Amazon QuickSight dashboards and analyses that serve operational and executive audiences.
- Translate business requirements and existing Tableau/Excel reports into QuickSight visuals, ensuring accuracy and usability.
- Optimize dashboard performance, evaluating SPICE vs. Direct Query modes and managing refresh schedul

📌 Senior Data Engineer (Bengaluru)
🏢 HealthEdge
📍 Bengaluru