Position Senior Data Engineer We are seeking an experienced Senior Data Engineer to design and implement high-performance data pipelines ensuring the smooth flow of large datasets across systems In this role you will leverage your expertise in Databricks Pyspark and ETL design to build efficient data solutions You will also contribute to optimizing database performance and ensure that data systems meet both scalability and security standards Key Responsibilities Develop optimize and maintain scalable ETL pipelines for processing large datasets using Databricks and Pyspark Apply advanced techniques such as chunking and partitioning to handle large file volumes efficiently Tuning and optimizing databases for better performance and storage efficiency Collaborate with cross-functional teams to ensure the architecture meets business requirements and data quality standards Design and implement solutions with an eye toward scalability and long-term performance Work with Azure services with exposure to Azure Function Apps Azure Data Factory and Cosmos DB preferred Communicate effectively with stakeholders to align on project goals and deliverables Key Skills and Experience Databricks advanced-level skills in data engineering workflows Pyspark intermediate-level skills for processing big data Robust ETL design skills particularly in partitioning chunking and database tuning for large datasets Azure experience a plus including Function Apps Cosmos DB and Azure Data Factory Excellent communication skills for effective collaboration with teams and stakeholders

📌 Data Engineer- 4 (Karnataka)
🏢 DBiz ai solutions
📍 Karnataka