10 Oct
|
Infosys
|
Bengaluru
Job Summary
We are seeking a highly motivated Databricks Data Engineer to design, develop, and optimize up-to-date data platforms leveraging Databricks and cloud-based technologies. The ideal candidate will have hands-on experience in building large-scale data pipelines, implementing Lakehouse architectures, and enabling advanced analytics solutions using Databricks, Spark, and cloud ecosystems.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
- Build and optimize ETL/ELT workflows for processing large-scale structured and unstructured datasets.
- Develop data ingestion frameworks to integrate data from multiple sources, including databases, APIs, applications, and cloud storage.
- Implement and manage Lakehouse architecture using Databricks and Delta Lake.
- Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
- Collaborate with analytics, BI, and business teams to understand data requirements and deliver reusable data assets.
- Implement data quality frameworks, monitoring solutions, and governance controls across data pipelines.
- Troubleshoot production issues and provide timely resolution to ensure business continuity.
- Participate in architecture reviews, code reviews, and technical discussions to promote best practices.
- Create and maintain technical documentation,
reusable components, and deployment standards.
Technical Requirements
- 9-11 years of strong hands-on experience with Databricks.
- Proficiency in PySpark or Spark SQL for distributed data processing.
- Strong SQL skills with experience in query optimization and performance tuning.
- Experience in designing and developing ETL/ELT pipelines.
- Hands-on experience with Delta Lake, DataFrames, and Spark architecture.
- Knowledge of data warehousing concepts, dimensional modeling, and data lake architectures.
- Experience working with cloud platforms such as Azure, AWS, or GCP.
- Familiarity with Git-based source code management and DevOps practices.
- Strong analytical, debugging, and problem-solving skills.
Additional Responsibilities
Infosys Data Analytics practice helps organizations modernize their data landscape through cloud-native architectures, advanced analytics, AI, and intelligent data platforms. As a Databricks Data Engineer, you will work on enterprise-scale data modernization programs involving Lakehouse implementations, cloud migrations, real-time analytics, and AI-ready data ecosystems.
Educational Requirements
Bachelor of Engineering
Preferred Skills
- Technology->Data Engineering->Databricks
Service Line
Data Analytics Unit
📌 Azure Databricks Data Engineer (Bengaluru)
🏢 Infosys
📍 Bengaluru