15 Nov
|
Infosys
|
Bengaluru
Job highlights
- The ideal candidate should have strong expertise in Databricks,PySpark,Spark,SQL,cloud platforms (AWS / Azure/GCP),and modern data engineering practices to build scalable and high-performance data pipelines.
- . Technical Requirement
- Experience with Kafka / Event Hub for streaming data processing is preferred
- . Educational Requirement MCA,MSc,MTech,BTech.
We are looking for an experienced Databricks Engineer with 5-8 years of experience in designing, developing, and optimizing large-scale data solutions. The ideal candidate should have strong expertise in Databricks, PySpark, Spark, SQL, cloud platforms (AWS/Azure/GCP), and modern data engineering practices to build scalable and high-performance data pipelines.
Roles & Responsibilities
- Roles & Responsibilities
- Design, develop, and maintain data pipelines using Databricks.
- Build ETL/ELT workflows for large-scale data processing.
- Develop and optimize data transformation logic using PySpark and Spark SQL.
- Implement and manage Delta Lake for reliable and effective data storage.
- Work with batch and real-time data ingestion frameworks.
- Monitor and optimize data pipelines for performance and scalability.
- Collaborate with data analysts, data scientists, and business stakeholders.
- Ensure data quality, security, and governance standards are followed.
- Troubleshoot production issues and provide ongoing support.
Technical Requirement
- 5-8 years of experience in Data Engineering and Big Data technologies.
- Strong hands-on experience with Databricks, PySpark, and Spark SQL.
- Experience with Delta Lake, Databricks Workflows, and Unity Catalog.
- Proficiency in Python and SQL.
- Experience with Azure/AWS/GCP cloud platforms.
- Knowledge of Azure Data Factory (ADF), Apache Airflow, or similar orchestration tools.
- Understanding of data warehousing concepts and ETL/ELT methodologies.
- Experience with Kafka/Event Hub for streaming data processing is preferred.
- Familiarity with Git, CI/CD pipelines, and DevOps practices.
- Strong problem-solving, analytical, and communication skills.
Educational RequirementMCA,MSc,MTech,BTech Preferred SkillsTechnology->Data Engineering->Databricks
Service LineData & Analytics Unit
📌 Databricks Engineer (Bengaluru)
🏢 Infosys
📍 Bengaluru