Description:
• Design, develop, and maintain scalable data pipelines using PySpark on Databricks
• Build ETL/ELT workflows to process structured and unstructured data
• Optimize Spark jobs for performance, cost, and scalability
• Work with Delta Lake, implement data versioning, and optimize storage
• Integrate data from multiple sources such as APIs, databases, and streaming platforms
• Develop and manage workflows using Databricks Workflows / Apache Airflow
• Ensure data quality through validation, monitoring, and logging
• Collaborate with data scientists, analysts, and stakeholders
• Implement best practices for data governance, security, and compliance
• Handle batch and real-time data processing (streaming). Total Experience (Ex. 5-7 Years) 5
- 8 Relevant Experience 4
- 7 Mandatory skills
• Databricks
• Pyspark
• Spark
• SQL
• CI/CD (GitHub or similar). Desired skills
• Understanding of Datawarehousing concepts.

📌 Data Engineer (Pune)
🏢 Amazure Technologies
📍 Pune