- Infosys is looking for a highly skilled Senior Databricks Consultant with 6 8 years of experience in building scalable data engineering solutions
- The ideal candidate should possess robust expertise in Databricks PySpark Apache Spark Python SQL ETL development and modern Data Engineering practices
Key Responsibilities:
- Design develop and maintain scalable Databricks based data engineering solutions
- Build and optimize complex ETL ELT pipelines using Databricks PySpark Spark SQL and Python
- Develop robust data ingestion and transformation frameworks for structured and unstructured data
- Implement and manage Delta Lake and Lakehouse Architecture solutions
- Design monitor and optimize enterprise grade data pipelines for performance scalability and reliability
- Perform performance tuning and troubleshooting of Databricks and Apache Spark workloads
- Develop reusable frameworks and best practices for Data Engineering and Data Integration
- Collaborate with architects analysts and business stakeholders to translate business requirements into technical solutions
- Ensure data quality governance security and compliance standards are followed
Technical Requirements:
- 6 8 years of experience in Data Engineering and Big Data technologies
- Strong hands on experience with Databricks and PySpark
- Expertise in Apache Spark Spark SQL and distributed data processing
- Strong programming skills in Python and SQL
- Experience building and managing enterprise scale ETL ELT pipelines
- Hands on experience with Delta Lake and Lakehouse Architecture
- Strong understanding of Data Warehousing Data Modeling and Data Integration concepts
- Experience working with large scale structured and unstructured datasets