25 Sep
|
WOW Softech
|
Bengaluru
25 Sep
WOW Softech
Bengaluru
Role: Hadoop Data Engineer
Experience: 8+ Years
Location: Bangalore
Work Mode: Hybrid
POSITION SUMMARY
We are seeking a highly skilled and driven Senior Data Engineer to join our growing Data Engineering team. The ideal candidate will have robust expertise in building scalable data solutions, modern cloud-based data platforms, and distributed data processing systems.
ROLES AND RESPONSIBILITIES
Design, develop, and maintain data services that integrate seamlessly with LendingClub products and external vendor systems.
Build and manage scalable data pipelines using modern technologies such as Hadoop, Spark, DBT, Trino, Airflow/Dagster, and cloud-native AWS services.
Develop solutions on modern data platforms including Databricks and Snowflake.
Identify opportunities to automate manual processes, optimize data delivery, improve system efficiency, and reduce cloud infrastructure costs.
Implement and enhance frameworks for Data Quality, Observability, Governance, Lineage, and Monitoring.
Support production environments, troubleshoot critical issues, and perform root cause analysis (RCA) to ensure platform reliability.
Follow engineering best practices including Test-Driven Development (TDD), unit and integration testing, documentation,
and knowledge-sharing through technical wikis.
REQUIRED QUALIFICATIONS
Strong hands-on experience with distributed data systems and big data technologies, including Hadoop, Spark, Hive, DBT, and Airflow/Dagster.
Minimum 4+ years of production-grade Python development experience, with a focus on data engineering and pipeline development.
Experience working with cloud platforms, preferably AWS.
Hands-on experience with modern data platforms such as Databricks and/or Snowflake.
Proficiency in Git, Jenkins, JIRA, Shell Scripting, and CI/CD practices.
Solid understanding of Agile methodologies, source control management, automated testing, and Test-Driven Development.
Experience collaborating with cross-functional teams in quick-paced, dynamic environments.
Solid analytical thinking, problem-solving abilities, and communication skills.
Proven track record of designing and delivering scalable, reliable, high-performance data pipelines and data platforms.
EDUCATION: Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience
KEY SKILLS: Python, PySpark, Hadoop, Spark, Hive, Data Engineering, Big Data, SQL
📌 Hadoop Data Engineer Bengaluru
🏢 WOW Softech
📍 Bengaluru