We are looking for a highly skilled Cloud Engineer with a specialization in Apache Spark and Databricks to join our energetic team. The ideal candidate will have extensive experience working with cloud platforms such as AWS, Azure, and GCP, and a deep understanding of data engineering, ETL processes, and cloud-native tools. Your primary responsibility will be to design, develop, and maintain scalable data pipelines using Spark and Databricks, while optimizing performance and ensuring data integrity across diverse settings.
Key Responsibilities:
Design and Development:
Architect, develop, and maintain scalable ETL pipelines using Databricks , Apache Spark (Scala, Python), and other cloud-native tools such as AWS Glue , Azure Data Factory , and GCP Dataflow .
Design and build data lakes and data warehouses on cloud platforms (AWS, Azure, GCP).
Implement effective data ingestion, transformation, and processing workflows with Spark and Databricks.
Optimize the performance of ETL processes for faster data processing and lower costs.




Develop and manage data pipelines using other ETL tools such as Informatica , SAP Data Intelligence , and others as needed.

Data Integration and Management:
Integrate structured and unstructured data sources (relational databases, APIs, ERP systems) into the cloud data infrastructure.
Ensure data quality, validation, and integrity through rigorous testing.
Perform data extraction and integration from SAP or ERP systems , ensuring seamless data flow.

Performance Optimization:
Monitor, troubleshoot, and enhance the performance of Spark/Databricks pipelines.
Implement best practices for data governance, security, and compliance across data workflows.

Collaboration and Communication:
Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to define data requirements and deliver scalable solutions.
Provide technical guidance and recommendations on cloud data engineering

📌 Cloud Data Engineer Spark Pune
🏢 Brighttier
📍 Pune