Role: Senior Data Engineer (PySpark, AWS, Databricks)
Location: Bangalore
Experience: 7-10 Years
Notice Period: Immediate to 30 Days Preferred
About the Role
We are looking for an experienced Data Engineer to design, develop, and optimize large-scale data platforms and pipelines. The ideal candidate should have strong expertise in PySpark, AWS, Databricks, Python, and SQL, with experience building scalable batch and streaming data solutions.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and data processing frameworks.
- Build and optimize batch and real-time data processing solutions using PySpark and Spark.
- Develop robust ETL/ELT workflows and data integration processes.
- Work with AWS cloud services to build and maintain data platforms.
- Implement and optimize streaming and batch data pipelines.
- Collaborate with cross-functional teams to deliver high-quality data solutions.
- Ensure data quality, performance, scalability,
and reliability.
- Implement CI/CD best practices and version control processes.
- Mentor junior engineers and contribute to technical leadership initiatives.
Required Skills
- Strong experience in PySpark / Apache Spark
- Solid proficiency in Python and SQL
- Strong Hands-on experience with AWS
- Experience with Databricks
- Experience with Kafka and Airflow
- Knowledge of DBT
- Experience with Batch and Streaming Data Pipelines
- Strong understanding of ETL/ELT concepts
- Experience with GitHub and CI/CD practices
- Team handling or leadership experience