- Develop, maintain, and monitor data pipelines, ensuring reliability and prompt resolution of failures as they occur.
- Collaborate with stakeholders to gather requirements, provide updates, and participate in design discussions.
- Design and implement custom ETL processes and automation scripts using AWS, Databricks, Python, and PySpark.
- Prepare architecture and design documentation for data engineering projects.
- Utilize expertise in cloud-based big data integration and infrastructure, with a strong focus on Databricks and the Apache Spark framework.
- Demonstrate proficiency in Python and other scripting languages for data engineering tasks.
- Leverage strong SQL and PL/SQL skills to write complex procedures and queries for dataframes and notebooks.
- Maintain a data engineering mindset to ensure quality, reliability, and efficiency across data systems and processes.
Minimum Qualifications :
- B.E. or Masters degree in Computer Science, Information Systems, Computer Engineering, or Systems Engineering.
- 4+ years of relevant skilled experience in data engineering or related fields.
- At least 2 years of hands-on experience with Databricks, Spark, Databricks workflows, and Autoloader.
- Proficiency in additional scripting languages such as Scala and JavaScript, as well as experience with API development.
- Familiarity with data orchestration and ETL tools, including Alteryx, Snowflake, and Azure Data Factory.
- Experience with business intelligence tools such as Tableau, Power BI, or QuickSight.
📌 Data Engineer (Chennai)
🏢 YO IT CONSULTING
📍 Chennai
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.