Responsibilities

- Design, build, and maintain robust ETL/ELT data pipelines using Apache Spark on Databricks.

- Implement data Lakehouse architecture using Delta Lake for cost-effective data storage and analytics.

- Use Databricks Workflows for orchestrating batch and streaming pipelines.

- Develop and maintain CI/CD pipelines for data applications using tools such as AWS, GitHub Actions or Databricks Repos.

- Monitor pipeline performance and troubleshoot data issues in real-time and batch environments.

- Document solutions, workflows, and technical standards.

Requirements

- Experience in data engineering with a strong focus on AWS Databricks and Apache Spark.

- Proficiency in PySpark, SQL, and Python.

- Experience with Delta Lake, Databricks SQL, and Unity Catalog.

- Hands-on experience with cloud platforms





- Familiarity with data lakehouse architecture, data warehousing and streaming data

- Strong understanding of ETL best practices, data partitioning, and performance tuning.

- Experience with CI/CD for data pipelines.

- Experience in Machine learning and Data Analytics is added advantage.

- Solid exposure and understanding on the AI and Agentic AI technologies.

- Excellent problem-solving and communication skills

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 EFS FCRM Professional (Hyderabad)
🏢 ADP Private
📍 Hyderabad