22 Nov
|
Ness Digital Engineering
|
India
22 Nov
Ness Digital Engineering
India
Sr. Python Data Engineer; Must Have: Strong hands-on Python ETL integration experience. Nice to have
- AWS Glue, PySpark, Snowpark Requirements and Responsibilities What You’ll Do
• Develop Python-based solutions for data extraction, transformation, and integration across multiple systems
• Work with legacy systems (Data Warehouses, ETL tools, BI tools) to extract and standardize metadata, data assets, code base and migrate to a modern cloud platform.
• Build resilient, fault-tolerant data integration and processing workflows
• Build modular, reusable code.
• Establish robust orchestration and monitoring using AWS and Snowflake services.
• Ensure governance, security, and cost efficiency.
• Implement Data quality checks, Tests in pipelines.
• Support CI/CD integration for data pipelines and contribute to DevOps practices
• Implement logging, monitoring, and alerting frameworks.
• Optimize pipelines for performance and cost.
• Lead troubleshooting and root cause analysis of production issues.
• Mentor junior engineers and support knowledge sharing within the team
• Collaborate with architects, analysts.
• Drive best practices in data engineering standards. What You’ll Bring
• Bachelor’s degree or higher in computer science, Information Technology, or a related field.
• Minimum 7+ years of relevant experience in data engineering, data integration.
• Minimum 5+ years of hands-on experience with Python for data engineering, data integration and automation.
• Robust hands-on experience handling structured and semi-structured data (e.g. JSON, XML)
• Strong SQL skills and experience working with large-scale datasets
• Experience with PySpark/Snowpark is a plus.
• Experience with AWS services such as Glue, S3, Lambda, and SNS is a plus.
• Experience working with ETL tools and enterprise data integration platforms (e.g. Informatica, SSIS, DataStage)
• Strong understanding of data warehousing concepts and practices.
• Familiarity with CI/CD practices for data pipelines.
• Experience with Agile/Scrum environments.
• Experience with Snowflake and dbt in production environments is a plus.
• Excellent analytical and problem-solving skills.
• Ability to work independently and guide junior resources.
• Strong communication skills in English (written and verbal).
📌 Flagstar -EDH- Data Engineer (AWS, Snowflake, ETL) - Rajesh Arugollu Replacement (India)
🏢 Ness Digital Engineering
📍 India