Data Engineer

Experience: 3+ Years

Location: Bengaluru(Bengaluru based candidates preferred for a F2F Interview)

Immediate Joiners

Candidates should be available for the F2F Interview.

About the role:

We are looking for a skilled Data Engineer with 3+ years of experience in building and maintaining scalable data pipelines and data platforms. The ideal candidate should have hands-on expertise in Apache Spark, SQL, Python, and Microsoft Azure services, with a strong understanding of data engineering best practices and cloud-based data solutions.

Key Responsibilities

- Design, develop, and optimize scalable ETL/ELT data pipelines using Spark and Python.
- Build, maintain, and enhance data ingestion frameworks for structured and unstructured data.
- Write efficient and optimized SQL queries for data extraction, transformation, and reporting.
- Develop and manage data workflows and pipelines on Microsoft Azure.
- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
- Monitor data pipelines, troubleshoot production issues, and ensure data quality, reliability, and availability.




- Implement data validation, logging, monitoring, and alerting mechanisms.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Work with large-scale datasets in distributed computing environments.
- Follow data governance, security, privacy, and compliance standards.
- Participate in code reviews and contribute to continuous improvement of engineering practices.

Required Skills

- 3 + years of experience in Data Engineering.
- Strong hands-on experience with Apache Spark (PySpark or Spark SQL).
- Proficiency in Python for data processing, automation, and application development.
- Advanced SQL skills with experience in query optimization and performance tuning.
- Hands-on experience with Microsoft Azure data serviceS
- Experience building batch and/or streaming data pipelines.
- Knowledge of data modeling and data warehousing concepts.
- Experience with Git and CI/CD practices.
- Solid analytical, problem-solving, and debugging skills.

📌 Data Engineer (Bengaluru)
🏢 Bct Consulting
📍 Bengaluru