Primary skills:Technology->Big Data - Data Processing->Spark Technology->Data Engineering->Databricks Technology->Functional Programming->Scala

Key Responsibilities: Data Engineering & ETL Development

- Design, develop, and maintain ETL pipelines using Spark-Scala on Databricks for batch and incremental processing.
- Implement data transformations, joins, aggregations, and validations to ensure accurate and consistent outputs.
- Build reusable Spark components and follow best practices for modular, maintainable code. Performance, Reliability & Operations
- Tune Spark jobs (partitioning, caching, shuffle optimization) to improve performance and cost efficiency.
- Monitor job execution, troubleshoot failures, and provide timely production support with root-cause analysis.
- Implement logging, error handling, and data quality checks to improve pipeline reliability. Collaboration & Delivery
- Work with cross-functional teams to gather requirements and translate them into technical solutions.
- Participate in code reviews, documentation, and knowledge sharing to uplift team standards.
- Support release cycles by validating outputs, ensuring backward compatibility, and maintaining deployment readiness.

Minimum

Qualifications:





- Bachelor’s/Master’s degree (BE/BTech/MSc/MCA/MTech or equivalent).
- 3–5 years of experience in data engineering or ETL development roles.
- Solid hands-on experience with Spark using Scala and working on Databricks.
- Solid understanding of ETL concepts, data transformations, and pipeline troubleshooting.
- Ability to write clean, testable code and collaborate effectively with technical and non-technical stakeholders.

Preferred

Qualifications:

- Experience building end-to-end pipelines on Databricks including notebooks, jobs/workflows, and cluster configuration basics.
- Strong SQL skills and experience integrating Spark pipelines with structured data sources and curated datasets.
- Familiarity with data quality frameworks, reconciliation strategies, and automated validation checks.
- Exposure to CI/CD practices for data engineering (version control, automated testing, release management).
- Proven ability to optimize distributed workloads and deliver measurable improvements in runtime and stability.

Good to have skills: SQL, Delta Lake, Apache Airflow, Azure Data Lake Storage (ADLS), Git

📌 Spark-Scala, Databricks (Bengaluru)
🏢 Infosys
📍 Bengaluru