10 Aug
|
Objectways
|
Bengaluru
10 Aug
Objectways
Bengaluru
Job Title - Data Engineer
Years of Experience - 5 to 8 Years
Location - Bangalore
Notice Period - Immediate to 30 Days
About the Role
We are looking for a skilled Data Engineer with hands-on experience in building and managing scalable data pipelines across cloud platforms. The ideal candidate should have strong expertise in ETL frameworks and exposure to multiple cloud environments.
Key Responsibilities
• Design, develop, and maintain scalable end-to-end ETL/ELT data pipelines (batch and near real-time) across AWS, Azure, and GCP environments (AWS preferred).
• Implement robust data ingestion patterns including CDC, API-based, file-based, and streaming integrations ensuring data quality, reliability, and performance.
• Develop and maintain cloud-native data solutions using MySQL/SQL Server, Azure Data Factory, MFTS, GCP BigQuery, and GCS (preferred: AWS Glue, Redshift, EC2, RDS).
• Build and manage workflow orchestration using Apache Airflow (or equivalent tools) for scheduling, monitoring, and dependency management.
• Work on data ingestion, transformation,
and integration across multiple enterprise systems and platforms.
• Develop CI/CD pipelines for data infrastructure and ETL code using Git-based workflows and Infrastructure-as-Code (IaC) best practices.
• Collaborate with cross-functional teams to understand data requirements and deliver scalable and effective data solutions.
• Work closely with security and compliance teams to implement encryption, IAM least privilege access, network segmentation (VPN/VPC), and governance controls.
• Build monitoring, alerting, and auditing mechanisms for data pipelines including latency tracking, failure monitoring, retries, DLQ, and data quality metrics.
• Optimize data storage, partitioning, and query performance for analytics workloads in BigQuery and Redshift environments.
• Troubleshoot production issues, perform root-cause analysis, and implement corrective and preventive actions.
• Produce and maintain technical doc
📌 Data Engineer – Azure (Bengaluru)
🏢 Objectways
📍 Bengaluru