Company : Quation Solution
Position: Data Engineer
Location : Bangalore (WFO)
JOB DESCRIPTION
We are looking for a skilled and passionate Data Engineer to join our growing team. As a Data Engineer, you will play a critical role in designing, developing, and maintaining our data infrastructure, ensuring the smooth flow of data and enabling data-driven decision-making across the organization.
Responsibilities:
• Design, build, and maintain efficient and reliable data pipelines using SQL and
Python, leveraging tools like Apache Airflow for orchestration.
• Develop ETL (Extract, Transform, Load) processes to move data between various sources and destinations.
• Implement data quality checks and validation routines to ensure data accuracy and integrity.
• Collaborate with data analysts and scientists to understand their data needs and translate them into actionable data solutions.
• Optimize existing data pipelines for performance and scalability using Airflow's features for monitoring and optimization.
• Develop and maintain documentation for data pipelines and processes,
including Airflow workflows.
• Contribute to the development and maintenance of data governance policies.
• Stay up-to-date with the latest trends and technologies in the data engineering field, including advancements in orchestration tools like Airflow.
Qualifications:
• Bachelor's degree in Computer Science, Engineering, or a related field.
• 2-3 years of experience as a Data Engineer or in a similar role.
• Solid proficiency in SQL for data querying, manipulation, and analysis.
• Excellent programming skills in Python, including experience with data manipulation libraries (Pandas, NumPy) and data engineering libraries (Dask, Airflow).
• Proficiency in using Apache Airflow for data pipeline orchestration, scheduling, and monitoring.
• Familiarity with cloud platforms (AWS, Azure, GCP) and data warehousing solutions (Snowflake, Redshift).