24 Oct
|
Accenture in India
|
Kochi
24 Oct
Accenture in India
Kochi
Job Title
- Lead Data Engineer – Specialist
- ACS SONG Management Level: Level 9
- Specialist
Location:
Kochi, Coimbatore, Trivandrum Must have
skills:
GCP services, BigQuery, Python, and SQL. Positive to have
skills:
BI and Analytical Tool
Experience:
5 -8 years of experience is required Educational Qualification: Graduation Job Summary We are seeking a Lead Data Engineer specializing in Google Cloud Platform (GCP) and BigQuery with 5+ years of experience in data engineering, cloud-based data platforms, data pipelines, SQL development, and large-scale data processing. The role will focus on designing, developing, and optimizing scalable data pipelines and analytics solutions using BigQuery and other GCP data services. Roles And Responsibilities
• The ideal candidate should have strong hands-on experience with BigQuery, Python, SQL, Cloud Storage, Dataflow, Pub/Sub, Dataproc, and Cloud Composer, along with a solid understanding of ETL/ELT, data warehouse, data lake, and modern cloud data architecture principles.
• This position requires the ability to independently design and deliver production-grade data solutions, optimize performance and cost, troubleshoot complex data issues, and collaborate closely with architects, data scientists, analysts, application teams, and business stakeholders.
• Design, develop, and maintain scalable data pipelines and data processing solutions using GCP services, BigQuery, Python, and SQL.
• Build reliable and high-performance data transformation and integration workflows supporting reporting, analytics, machine learning, and downstream business applications.
• Drive improvements in data quality, pipeline performance, BigQuery optimization, cost efficiency, monitoring, security, and production stability.
• Provide technical guidance and contribute to the adoption of modern GCP data engineering architecture, standards, and development best practices.
• Design, develop, and maintain batch and real-time data pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Cloud Composer, Python, and SQL.
• Develop ETL/ELT solutions for ingesting, transforming, and integrating structured, semi-structured, and unstructured data from multiple source systems.
• Design and maintain scalable BigQuery datasets, tables, views, materialized views, partitioning, clustering, and data models for analytics and reporting workloads.
• Optimize BigQuery SQL queries, storage structures, data processing workloads, and pipeline architecture to improve performance and control cloud consumption costs.
• Build and maintain batch and streaming data processing solutions using services such as Dataflow, Pub/Sub, Dataproc, and BigQuery.
• Design curated data layers, reusable datasets, and data models supporting enterprise analytics, BI, machine learning, and downstream applications.
• Implement data validation, reconciliation, data quality checks, error handling, lineage, monitoring, alerting, and operational controls.
• Orchestrate data pipelines and workflows using Cloud Composer / Apache Airflow, Workflows, or equivalent orchestration technologies.
• Collaborate with solution architects, data architects, data scientists, analysts, application teams, and business stakeholders to translate requirements into scalable technical solutions.
•
• Follow and promote engineering best practices covering Git-based development, automated testing, CI/CD, Infrastructure as Code, documentation, monitoring, security, and production support.
📌 Lead Data Engineer 2 (Kochi)
🏢 Accenture in India
📍 Kochi