Key Responsibilities:

Build, and optimize scalable data pipelines and ETL/ELT workflows for large, complex datasetsDesign and implement foundational data architecture supporting identity resolution and ID graph systemsDevelop and enhance systems supporting identity resolution and ID graph construction (data ingestion, normalization, matching, and deduplication)Process and unify multi-source datasets (cookies, device IDs, behavioral data, third-party and proprietary data)Write productive, testable, and maintainable code using Python and SQL for large-scale data processingOptimize data models, queries, and storage strategies for performance, scalability, and cost efficiencyBuild and maintain data validation, monitoring, and alerting systems to ensure data quality and reliabilityTroubleshoot, debug, and improve existing data pipelines and infrastructureOwn and drive complex data problems end-to-end, from initial design through production deploymentMake and influence key technical decisions related to data architecture, scalability, and system designCollaborate with data, platform, DevOps, and product teams to deliver scalable, production ready solutionsTranslate business and product requirements into practical, performant data solutionsDocument data pipelines, systems, and workflows clearlyContinuously improve system performance, data quality, and pipeline resilience.Contribute to building new capabilities that improve how customers understand and leverage data insights.

Key Requirements:





8-12+ years of hands-on experience in data engineering or large-scale data processingProven experience building and maintaining production-grade data pipelines and distributed systemsDemonstrated experience architecting and delivering large-scale data platforms or mission critical data systemsStrong expertise in: o SQL and relational databases (Postgres, BigQuery, Redshift, etc.) Python for data processing and analysisExperience with Google Cloud Platform (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Cloud Functions) and/or AWS (S3, Redshift, EMR, RDS)Experience working with large-scale datasets (hundreds of millions to billions of records)Strong understanding of data modeling, partitioning, indexing, and query optimizationExperience with distributed data processing and parallelization techniquesExperience moving large volumes of data across systems and architecturesFamiliarity with CI/CD, containerization, and orchestration tools (Docker, Kubernetes, GitHub Actions, etc.)Strong debugging and troubleshooting skills in complex data environmentsExperience with version control (Git) and Agile tools (Jira, Confluence, etc.)Highly analytical with strong attention to detail and a data-driven mindsetAbility to hit the ground running, quickly understand systems, and deliver independentlyComfortable working in a remote, fast-paced, and collaborative environmentProven ability to drive system design and implementation.

📌 Principal Data Engineer (Mumbai)
🏢 Dun & Bradstreet India
📍 Mumbai