Primary Responsibilities:
Data Pipeline Development: Design, develop, and maintain productive and scalable data pipelines using Python and Azure technologies
Data Integration: Integrate data from various sources, ensuring data quality and consistency
Azure Cloud Services: Utilize Azure cloud services for data storage, processing, and analytics, including Azure Data Factory, Azure Databricks, and Azure SQL Database
CI/CD Implementation: Implement and manage CI/CD pipelines to automate the deployment of data solutions
Data Modeling: Develop and maintain data models to support business intelligence and analytics needs
Performance Optimization: Optimize data processing and storage solutions for performance and cost-efficiency
Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs
Documentation: Create and maintain comprehensive documentation for data pipelines, processes, and solutions
Troubleshooting:



Identify and resolve data-related issues and bottlenecks

Qualifications - External

Required Qualifications:
Bachelor's or Master's degree in Computer Science, Information Technology, or equivalent in related field
7+ years of experience in data engineering, with a solid focus on Python, Azure, CI/CD, and deployment
Proficiency in Python for data processing and automation
Experience with Azure cloud services (Azure Data Factory, Azure Databricks, Azure SQL Database, etc.)
Experience with containerization and orchestration tools (e.g., Docker, Kubernetes)
Knowledge of SQL and NoSQL databases
Familiarity with data warehousing concepts and technologies
Solid understanding of CI/CD principles and tools (e.g., Jenkins, GitLab CI/CD)
Proven excellent problem-solving and analytical skills
Proven solid communication and collaboration abilities
Proven ability to work independently and as part of a team

📌 Data Engineer Uttar Pradesh
🏢 Optum
📍 Uttar Pradesh