Job Description: Data Engineer

Role: Data Engineer

Experience: 6+ Years
Location: As per project requirement ( Bangalore, Hyderabad, Pune, Kolkata

Must-Have Skills

- Strong hands-on experience in AWS, particularly S3, Glue, Lambda, and Athena.
- Hands-on experience with Snowflake, including data modeling and performance tuning.
- Solid SQL skills with experience in complex queries and query optimization.
- Proficiency in Python and/or PySpark.
- Strong experience in ETL/ELT pipeline development.
- Hands-on experience with Apache Airflow, including DAG development and scheduling.
- Experience with incremental and batch data processing.
- Good understanding of file formats such as Parquet/ORC, partitioning, and data processing optimization.
- Good-to-Have Skills

- Experience with REST API integration.
- Knowledge of BIP (BI Publisher) reports.
- Experience with AWS Glue optimization and cost tuning.
- Knowledge of CDC (Change Data Capture) and data versioning.
- Experience with Athena performance tuning.
- Exposure to CI/CD pipelines.




- Understanding of data governance practices.
- Exposure to Oracle Fusion/EBS data.
- Key Responsibilities

- Design, develop, and optimize AWS-based data pipelines using services such as S3, Glue, Lambda, and Athena.
- Develop and maintain Snowflake data models and optimize their performance.
- Design, develop, and manage Apache Airflow DAGs for data pipeline orchestration and scheduling.
- Implement incremental and batch data processing mechanisms.
- Optimize data pipelines for cost, performance, scalability, and reliability.
- Develop complex SQL queries and perform query optimization.
- Build ETL/ELT processes using Python/PySpark.
- Troubleshoot and resolve production data pipeline issues.
- Collaborate with Data Analysts, QA teams, and other stakeholders to deliver reliable data solutions.
- Follow best practices for data quality, governance, security, and deployment.

📌 Data Engineer (Hyderabad)
🏢 Alike Thoughts
📍 Hyderabad