6+ years of hands-on data engineering development experience in Python Advanced knowledge of Python and Linux shell scripting Proficiency in SQL and Data Structures, experience in handling (transformations, read / write) huge volume of data using Python
Experience in performance improvement on data pipelines and a good understanding of Volume, Variety, Veracity, Velocity, and Value concepts in Big Data
- Valuable knowledge of algorithms and software design principles
- Experience in CI/CD pipelines with one of: Azure DevOps, AWS Code Pipeline, GitHub Actions Familiarity with Docker, Kubernetes, and cloud services (Azure, AWS, GCP) Proficient with Kafka, Flink, RabbitMQ or similar technologies Self-motivated, able to self-improve,
learn new and complex Technologies
- In-depth knowledge and experience in agile processes and principles
- Summary: This JD is targeting a senior-level data engineer with extensive experience in building scalable data infrastructures, working with large datasets, optimizing performance, and applying cutting-edge technologies
- Strong Python and cloud experience are critical, as well as proficiency in Big Data tools and design patterns
- You will also need to be proactive in improving systems and aligning your work with business goals