We are looking for a skilled Data Engineer to design, build, and optimize scalable data pipelines and systems. The ideal candidate will have hands-on experience with data lake development, orchestration tools, and big data technologies. A solid foundation in Python and SQL is essential, along with expertise in optimizing data processing frameworks.
Responsibilities
Design and implement robust data pipelines to support analytics and reporting.
Develop and maintain data lakes for effective data storage and processing.
Optimize Spark/PySpark and Hive applications for performance and scalability.
Implement and manage workflow orchestration using Apache Airflow.
Collaborate with cross-functional teams to ensure data quality, reliability, and accessibility.
Required Skills
Programming: Advanced proficiency in Python.
SQL Expertise: Strong ability to write and optimize complex SQL queries.
Big Data Tools: Hands-on experience with Spark and Hive.
Data Lakes:
Proven experience in data lake development.
Orchestration: Experience with Apache Airflow or similar tools.
Preferred Skills
Familiarity with Trino or AWS Athena.
Experience with Snowflake.
Understanding of data quality frameworks and best practices.
Knowledge of file storage solutions like Amazon S3.
Qualifications
Bachelor s or Master s degree in Computer Science, Data Engineering, or a related field.
Demonstrated ability to design and optimize scalable data solutions.
Robust problem-solving and analytical skills.
Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their official channels before applying.
📌 Data Engineer Pune
🏢 Valuedx Technologies
📍 Pune
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.