Job Description:

The Senior Data Engineer will be responsible for designing, developing, and maintaining scalable cloud-based data platforms and data pipelines. The role requires strong expertise in AWS, Snowflake, Databricks, Python, SQL, and PySpark, along with experience in batch and real-time data processing. The candidate will collaborate with cross-functional teams to build reliable, secure, and highperformance data solutions that support analytics, reporting, and AI/ML initiatives.

Key Responsibilities:

- Design, develop, and maintain scalable data pipelines and ETL/ELT frameworks.
- Build and optimize batch and real-time streaming solutions using PySpark and cloud-native services.
- Develop and manage enterprise data platforms using AWS, Snowflake, and Databricks.
- Design and implement robust data models, data lakes, data warehouses, and lakehouse architectures.
- Integrate data from multiple sources while ensuring data quality, governance, and security.
- Optimize performance and scalability of data processing workloads.
- Collaborate with Data Scientists, Analysts, Architects, and business stakeholders to deliver data solutions.
- Implement monitoring, alerting, and automated data validation processes
- Contribute to architecture reviews, technical documentation, and best practices.
- Mentor junior engineers and participate in code reviews.
- Stay updated on emerging cloud and data engineering technologies.

Required Skills and Experience:

- 5-8 years of relevant experience in Data Engineering, Big Data, and Cloud Data Platform development.
- Strong proficiency in Python, SQL, and PySpark.
- Hands-on experience designing, developing, and maintaining scalable data platforms on AWS Cloud.
- Strong expertise in Snowflake and Databricks for enterprise-scale data engineering solutions.
- Experience building and optimizing batch and real-time streaming data pipelines.




- Strong understanding of Data Modeling concepts, including Dimensional
- Modeling, Star Schema, Snowflake Schema, and Data Vault.
- Experience designing and implementing ETL/ELT pipelines for large-scale enterprise data platforms.
- Robust expertise in Apache Spark and distributed data processing frameworks.
- Experience building and managing Data Lakes, Data Warehouses, and Lakehouse architectures.
- Hands-on experience with AWS services such as S3, Glue, EMR, Lambda, Redshift, Athena, EC2, ECS/EKS, Kinesis, MSK, Step Functions, CloudWatch, IAM, RDS, DynamoDB, EventBridge, SNS, and SQS.
- Experience integrating data from multiple structured and unstructured data sources.
- Strong understanding of data governance, security, data quality, and performance optimization best practices.
- Experience with version control systems such as Git and collaborative development workflows.
- Proficiency with CI/CD pipelines and software engineering best practices.
- Strong problem-solving skills with the ability to debug, troubleshoot, and resolve complex data engineering challenges.
- Ability to design scalable and maintainable data solutions using modern architectural patterns and best practices.
- Proven ability to work collaboratively in cross-functional teams, including participation in code reviews, technical discussions, and knowledge-sharing initiatives.
- Experience working in Agile/Scrum delivery environments.
- Strong communication skills and ability to interact with business and technical stakeholders.
- Experience in Healthcare, Pharma, or Real-World Evidence (RWE) data environments (good to have).
- Experience with Infrastructure as Code (Terraform/CloudFormation) (good to have).
- Experience with Docker and Kubernetes (good to have).
- Knowledge of Data Mesh, CDC, and Event-Driven Architectures (good to have).
- Exposure to MLOps and AI/ML platform integration (good to have).
- AWS, Snowflake, or Databricks certifications (good to have).

📌 Senior Data Engineer (India)
🏢 MathCo India
📍 India