Job DescriptionData Engineer L2

NExperience: 3-10 years in data engineering.

NLocation: Bangalore.

NWork Mode: Bangalore - Hybrid

NRole Overview

nJoin CoffeeBeans Consulting as a Data Engineer L2 and immerse yourself in a transformative role where your expertise will directly contribute to the future of AI. Located in Bangalore, this position offers a unique opportunity to work at the forefront of data engineering, shaping the way businesses leverage their data to drive innovation. With 4–7 years of experience, you will play a pivotal role in building and optimizing scalable data pipelines that empower analytics and AI/ML solutions.

This is not just a job;

it's a chance to elevate your career in a company that values engineering excellence and client impact.

NKey Responsibilities

n

N
- Design and implement enterprise-grade Databricks Lakehouse architectures using Delta Lake and Unity Catalog.
N
- Build scalable batch and real-time data ingestion pipelines using Lakeflow Connect, SDP, Auto Loader, Spark, and Kafka.
N
- Design and implement CDC architectures using Debezium, Kafka/Kafka Connect, and relational databasessuch as PostgreSQL, MySQL, SQL Server, and Oracle.
N
- Implement streaming andevent-driven data pipelines using Kafka, Spark Structured Streaming, and related technologies.
N
- Design and manage schema evolution and data contracts using Karapace / Schema Registry.
N
- Implement centralized governance using Unity Catalog, including catalogs, schemas, RBAC, row/column-level security, lineage, and data access policies.
N
- Develop metadata-driveningestion frameworks, data quality, reconciliation, profiling,



and observability solutions.
N
- Design Bronze, Silver, and Gold data layers and appropriate data modeling strategies for analytical workloads.
N
- Establish engineering best practices covering CI/CD, testing, deployment, monitoring, logging, and operational support.
N
- Use Databricks Asset Bundles (DAB) and CI/CD tools such as Jenkins/GitHub Actions for automated deployment.
N
- Work with cloud services such as AWS S3, IAM, networking, monitoring, and security services.
N
- Lead technical discussions with clients, translate business requirements into technical solutions, and drive architecture decisions.
N
- Troubleshoot complex data engineering, CDC, streaming, performance, and production issues.
N
- Mentor engineers and provide technical direction across data engineering initiatives. Must-Have Skills
N
- Strong hands-on experience with Databricks and Lakehouse architecture.
N
- Advanced Python and SQLskills.
N
- Strong expertise in Apache Spark / PySpark and distributed data processing.
N
- Hands-on experience with Unity Catalog and Delta Lake.
N
- Experience with Lakeflow Connect, SDP / Spark Declarative Pipelines, and Auto Loader.
N
- Strong understanding ofCDC architectures using Debezium and Kafka.
N




- Hands-on experience with Kafka / Kafka Connect.
N
- Experience with Karapace or Schema Registry and schema evolution.
N
- Strong understanding ofETL/ELT, data modeling, data warehousing, streaming, and data integration patterns.
N
- Experience with production-grade data pipelines and orchestration.
N
- Strong understanding ofcloud-native data services, particularly AWS.
N
- Experience with CI/CD and Databricks Asset Bundles (DAB).
N
- Experience leading technical implementations and working directly with business/client stakeholders.
N

nGood to Have

n

N
- Experience with Snowflake and dbt.
N
- Experience with Apache Flink or other real-time processing frameworks.
N
- Experience implementingdata governance, lineage, security, data quality, and observability.
N
- Experience with AWS S3,IAM, Glue, MSK/Kafka, and cloud networking.
N
- Experience designing metadata-driven data platforms.
N
- Experience with AI/ML data platforms and GenAI workloads.
N
- Databricks certifications, particularly Databricks Certified Data Engineer Professional.
N
- AWS Data Engineering/Data Analytics certifications.

Other

Expectations

N
- Robust ownership and problem-solving mindset.
N
- Ability to balance hands-on engineering with architecture and technical leadership.
N
- Strong client-facing and communication skills.
N
- Ability to mentor and guide engineering teams.
N
- Willingness to adapt tonew technologies and client environments.
N
- Willingness to travel within India and internationally for short/medium-term client assignments.
N

📌 Senior Data Engineer (Karnataka)
🏢 CoffeeBeans
📍 Karnataka