Hiring for a MNC - client

Hyderabad and Bengaluru

3- 7 years experience

Software Development Engineer (SDE) – Data Engineering & Data Lake Experience: 4–8 Years

Employment Type: Full-Time

Role: Software Development Engineer – Data Engineering

About the Role We are looking for a hands-on Software Development Engineer (SDE) with strong experience in Data Engineering, Data Lakes and large-scale data migration. The ideal candidate should have strong programming and SQL fundamentals, practical experience with Apache Spark/PySpark, and a proven track record of working on data migration, reconciliation, validation and data quality initiatives.

The role involves designing, developing and optimizing scalable data pipelines and supporting data lake/lakehouse migration initiatives.

Key Responsibilities

- Design, develop and maintain scalable data engineering pipelines and data lake solutions.
- Develop complex and optimized SQL queries for large datasets.
- Build and maintain data processing applications using Python or Java and Apache Spark/PySpark.
- Work on data migration and data lake migration projects, including source-to-target mapping and migration validation.
- Perform data reconciliation, validation and data quality checks across source and target systems.
- Implement SCD Type 2 and temporal data models for historical data management.
- Work with large-scale datasets in distributed data environments.
- Design solutions involving partitioning, clustering and performance optimization.
- Work with Snowflake, Hadoop/HDFS/Hive and modern lakehouse technologies.
- Work with Apache Iceberg and lakehouse architectures.




- Handle structured and semi-structured data formats including Parquet, Avro and JSON.
- Support schema evolution and changes in data structures.
- Participate in CI/CD processes for data engineering applications.
- Troubleshoot performance, data quality and pipeline issues.
- Collaborate with data architects, developers, QA teams and business stakeholders.

Mandatory / Non-Negotiable Skills Candidates must have hands-on experience with:

- Strong SQL
- Python or Java
- Apache Spark / PySpark
- Data Engineering / Data Lake
- Hands-on Data Migration / Data Lake Migration
- Data Reconciliation / Validation / Data Quality
- SCD Type 2 / Temporal Data Modeling

High-Priority Skills Strong preference for candidates with experience in:
- Snowflake
- Hadoop / HDFS / Hive
- Apache Iceberg / Lakehouse Architecture

Technical Skills – Good to Have
- Kubernetes
- Apache Kafka
- FTP
- Sybase IQ

Additional Technical Knowledge Experience with the following will be advantageous:
- Schema Evolution
- Partitioning
- Clustering
- Query and Data Pipeline Performance Optimization
- Parquet
- Avro
- JSON
- CI/CD

Candidate Profile The ideal candidate should:
- Have robust hands-on development experience rather than purely managerial exposure.
- Be comfortable working with large datasets and distributed data processing.
- Have strong problem-solving and debugging skills.
- Understand data migration challenges and source-to-target reconciliation.
- Be capable of independently developing and troubleshooting data pipelines.
- Have good understanding of data modeling and historical data management.
- Be comfortable working in an Agile development environment.

📌 SDE- Data Engineer (Hyderabad)
🏢 Magneton Technologies
📍 Hyderabad