Location: Bengaluru (Preferred Bengaluru based candidates to apply)
Should be available to attend the Face to Face Interview
Immediate Joiners Preferred
About the Role
We are looking for a skilled Data Engineer with 4 - 8 years of experience in building and maintaining scalable data pipelines and data platforms. The ideal candidate should have hands-on expertise in AWS, Spark, SQL, Python with a strong understanding of data engineering best practices and cloud-based data solutions.
Key Responsibilities
- Design, develop, and optimize scalable ETL/ELT data pipelines using Spark and Python.
- Build, maintain, and enhance data ingestion frameworks for structured and unstructured data.
- Write efficient and optimized SQL queries for data extraction, transformation, and reporting.
- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.
- Monitor data pipelines, troubleshoot production issues, and ensure data quality and reliability.
- Implement data validation, logging, and monitoring mechanisms.
- Optimize Spark jobs for performance and cost efficiency.
- Work with large-scale datasets in distributed computing environments.
- Follow data governance, security, and compliance standards.
- Participate in code reviews and contribute to continuous improvement of engineering practices.
Required Skills
- 4 - 8 years of experience in Data Engineering.
- Robust hands-on experience with Apache Spark (PySpark or Spark SQL).
- Proficiency in Python for data processing and automation.
- Advanced SQL skills with experience in query optimization.
- Experience in building batch and/or streaming data pipelines.
- Knowledge of data modeling and data warehousing concepts.
- Experience with Git and CI/CD practices.
- Strong analytical, problem-solving, and debugging skills.
Preferred Qualifications
- Experience with Apache Airflow or similar workflow orchestration tools.
- Exposure to Kafka or other streaming technologies.
- Familiarity with Delta Lake, Apache Iceberg, or Apache Hudi.
- Experience with Infrastructure as Code (Terraform or CloudFormation) is a plus.
- AWS Certification is an added advantage.
📌 Aws Data Engineer (Bengaluru)
🏢 Bct Consulting
📍 Bengaluru
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.