Data Engineer/Developer

Role Summary
Design, build, and optimize Azure data pipelines and lakehouse solutions.
Deliver secure, reliable datasets with strong governance, automation, and
documentation. Collaborate across teams and contribute to standards in an Agile
setting.
Must-Have (Day 1)
Experience:
4–6 years in data engineering
Core
Platform: Databricks with Python, Spark, Pandas (notebooks and modular
code)
Orchestration:
Azure Data Factory (pipelines, integration runtimes); ingest from diverse
sources
Lakehouse:
Delta Lake fundamentals; Medallion architecture (bronze/silver/gold) in
production
Storage/SQL/Performance:
Azure Data Lake Storage (ADLS); solid SQL; performance-aware design
Data
Patterns: ETL/ELT; data modeling (e.g., dimensional/star schema)
DevOps
& Security: CI/CD for data projects (Azure DevOps or GitHub
Enterprise); familiarity with Azure Entra ID for SSO/RBAC; secure
workspace/data access
Quality
& Observability: Data validation/testing, code reviews, and basic
monitoring/alerting for jobs/pipelines
Ways of
Working:



Agile/Scrum (Jira/Confluence); transparent pipeline and data contract
documentation
Collaboration:
Effective stakeholder engagement; support/mentor junior team members;
explicit communication
Generative
AI (Day 1):
Prompt
design for data tasks (ingestion, transformations, documentation) with
clear objectives and constraints
Use of
Copilot/ChatGPT to scaffold notebooks/jobs, generate tests, and optimize
SQL/Spark—validates outputs before merging

Nice-to-Have (Train within 60–90 days)
Unity
Catalog migration (Hive to Unity) and permissions/governance
Databricks
DevOps (cluster configuration, secret management, workspace automation)
Azure
Functions (C# or Python) for orchestration/integration
Synapse
dedicated SQL pools or dbt; Delta Live Tables
Financial
services domain exposure

Shared Expectations
Work
independently with minimal supervision while contributing to team outcomes
Commitment
to secure p

📌 Data Engineer Hyderabad
🏢 Kumaran Systems
📍 Hyderabad