30 Oct
|
PW (PhysicsWallah)
|
Noida
30 Oct
PW (PhysicsWallah)
Noida
Role: Data Engineer SDE-1
Location:
Noida and Bangalore
About the Role
We are looking for an experienced Data Engineer to design, build, and optimize scalable data infrastructure and pipelines. The ideal candidate will have strong expertise in ETL/ELT processes, distributed data processing, and data lake and warehouse architectures. They will work closely with cross-functional teams to build reliable, high-performance data solutions that support business intelligence, analytics, and data-driven decision-making.
Key Responsibilities
● Design and implement scalable, efficient ETL/ELT pipelines to ingest, transform, and process data from multiple sources. ● Architect and maintain robust data lake and warehouse solutions, aligning with business and analytical needs. ● Own the development and optimization of distributed data processing systems using Spark, AWS EMR, or similar technologies. ● Collaborate with cross-functional teams (data science, analytics, product) to gather requirements and implement data-driven solutions. ● Ensure high levels of data quality, security, and availability across systems. ● Evaluate emerging technologies and tools for data processing and workflow orchestration. ● Build reusable components, libraries, and frameworks to enhance engineering efficiency and reliability.
● Drive performance tuning, cost optimization, and automation of data infrastructure. ● Mentor junior engineers, review code, and set standards for development practices. Required Skills &
Qualifications:
● 1+ years of qualified experience in data engineering or backend systems with a focus on scalable systems. ● Strong hands-on experience with Python or Scala, and writing efficient, production
- grade code. ● Deep understanding of data engineering concepts: data modeling, data warehousing, data lakes, streaming vs. batch processing, and metadata management. ● Solid experience with AWS (S3, Redshift, EMR, Glue, Lambda) or equivalent cloud platforms. ● Experience working with orchestration tools like Apache Airflow (preferred) or similar (Azkaban, Luigi). ● Proven expertise in working with big data tools such as Apache Spark, and managing Kubernetes clusters. ● Proficient in SQL and working with both relational (Postgres, Redshift) and NoSQL (MongoDB) databases. ● Ability to understand API-driven architecture and integrate with backend services as part of data pipelines. ● Strong problem-solving skills, with a proactive attitude towards ownership and continuous improvement.
📌 Data Engineer (Noida)
🏢 PW (PhysicsWallah)
📍 Noida