We are looking for a skilled Data Engineer to design, build, and optimize scalable data pipelines and systems. The ideal candidate will have hands-on experience with data lake development, orchestration tools, and big data technologies. A strong foundation in Python and SQL is essential, along with expertise in optimizing data processing frameworks.
Responsibilities
- Design and implement robust data pipelines to support analytics and reporting.
- Develop and maintain data lakes for efficient data storage and processing.
- Optimize Spark/PySpark and Hive applications for performance and scalability.
- Implement and manage workflow orchestration using Apache Airflow.
- Collaborate with cross-functional teams to ensure data quality, reliability, and accessibility.
Required Skills
- Programming: Advanced proficiency in Python.
- SQL Expertise: Solid ability to write and optimize complex SQL queries.
- Big Data Tools: Hands-on experience with Spark and Hive.
- Data Lakes:
Proven experience in data lake development.
- Orchestration: Experience with Apache Airflow or similar tools.
Preferred Skills
- Familiarity with Trino or AWS Athena.
- Experience with Snowflake.
- Understanding of data quality frameworks and best practices.
- Knowledge of file storage solutions like Amazon S3.
Qualifications
- Bachelor s or Master s degree in Computer Science, Data Engineering, or a related field.
- Demonstrated ability to design and optimize scalable data solutions.
- Strong problem-solving and analytical skills.
Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their official channels before applying.
📌 Data Engineer (Pune)
🏢 Valuedx Technologies
📍 Pune
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.