13 Sep
|
Persistent Systems
|
Pune
13 Sep
Persistent Systems
Pune
About Position:
We are looking for a skilled Data Engineer with 6 to 9 years of experience in building scalable, secure, and high-performance data platforms. The ideal candidate will possess strong expertise in Databricks, Python, Spark, and Azure cloud technologies, with hands-on experience in developing distributed data pipelines and modern data processing solutions. This role requires a strong understanding of cloud-native architecture, containerized deployments, data security controls, and data engineering best practices.
- Role: Data Engineer
- Location: Pune
- Experience: 6 to 9 years
- Job Type: Full time Employment
What You‘ll Do:
- Design, build, and optimize distributed data pipelines using Python and Apache Spark.
- Develop, manage, and maintain large-scale data processing workloads on Azure Databricks.
- Build scalable ETL and ELT solutions for data ingestion, transformation, and processing.
- Manage data storage and cloud resources using Azure Storage, Azure Container Registry (ACR), and Azure Kubernetes Service (AKS).
- Develop container-based deployment solutions for data engineering workloads.
- Orchestrate data workflows using Apache Airflow and enterprise scheduling platforms.
- Implement robust data security controls including access management, encryption, and secure credential handling.
- Support vulnerability remediation activities across data platforms using tools such as CodeQL, Cycode, and related security frameworks.
- Monitor data pipelines and troubleshoot performance, reliability, and operational issues.
- Collaborate with data architects, business analysts, and application teams to deliver business-driven data solutions.
- Maintain source code repositories using GitHub and ensure proper documentation in Confluence.
- Participate in code reviews and enforce data engineering best practices and development standards.
- Support CI/CD processes and deployment automation for data platform components.
- Contribute to Agile ceremonies includi
📌 Data Engineer (Pune)
🏢 Persistent Systems
📍 Pune