KEY RESPONSIBILITIES

- System Architecture: Design scalable, resilient, secure, and cost-effective data lakehouses and warehouses using modern patterns such as Medallion Architecture (Bronze/Silver/Gold).

- Technical Leadership: Set engineering standards, conduct architecture and code reviews, establish reusable frameworks, and mentor data engineers on best practices.

- Data Enablement: Deliver reliable, high-performance data platforms that support BI, data science, machine learning, and operational analytics.

- Cloud Architecture: Design end-to-end Azure/cloud infrastructure blueprints covering storage, compute, RBAC, security, network isolation, scalability, and cost optimization.

- Big Data Processing: Architect and optimize Spark/Databricks/Snowflake workloads, including Delta Lake design, cluster sizing, partitioning, Z-Ordering, Vacuuming, clustering, and performance tuning.

- Data Pipelines Ingestion: Build and standardize production-grade PySpark, Python, and SQL pipelines supporting batch, micro-batch, streaming, and CDC ingestion patterns.

- Data Modeling: Establish organizational data-modeling standards and design Star/Snowflake schemas, Kimball dimensional models, Data Vault patterns, and Lakehouse storage layers.

- Orchestration: Architect workflow dependencies, retries, alerts, monitoring, and backfilling using Azure Data Factory, Airflow, or Databricks Workflows.

- DevOps CI/CD: Implement Git-based development standards and automated CI/CD pipelines using Azure DevOps or GitHub Actions for reliable data-platform deployments.

- Infrastructure-as-Code:



Design and implement infrastructure provisioning using Terraform, Bicep, or equivalent IaC frameworks.

- Governance Security: Implement metadata catalogs, lineage, data quality controls, access policies, and regulatory compliance practices using technologies such as Unity Catalog and Microsoft Purview.

- Performance FinOps: Benchmark workloads, resolve bottlenecks, optimize memory and compute configurations, monitor cloud consumption, and drive cost-efficiency across data platforms.

MINIMUM REQUIREMENTS

- Overall Experience: 10 to 12+ years of IT/software engineering experience, with at least 5 years in Azure data architecture, cloud platform design, or equivalent data-platform architecture.

- Distributed Computing: 10+ years of experience working with distributed data processing or modern cloud data platforms, with strong hands-on exposure to Databricks, Spark, Snowflake, or equivalent technologies.

- Data Architecture: Strong understanding of Lakehouse/Warehouse architecture, Medallion Architecture, scalability, resiliency, security, and cost optimization.

- Data Modelling: Solid mastery of Kimball dimensional modelling, Star/Snowflake schemas, Data Vault, and analytical data design.





- Programming: Robust hands-on expertise in Python/PySpark and SQL, including production-grade transformations and performance optimization.

- Orchestration: Strong experience with Airflow, Azure Data Factory, Databricks Workflows, or equivalent orchestration platforms.

- DevOps: Hands-on experience with Git, CI/CD, Infrastructure-as-Code, Terraform/Bicep, and automated deployment practices.

- Cloud Security: Robust understanding of Azure services, RBAC, identity, network isolation, security controls, monitoring, and enterprise governance.

- Communication Leadership: Ability to communicate architecture decisions clearly, lead technical discussions, mentor engineers, and work effectively with cross-functional stakeholders.

GOOD TO HAVE

- Governance: Experience with Unity Catalog, Microsoft Purview, metadata management, data lineage, data quality, GDPR, HIPAA, or other regulated-data environments.

- Monitoring: Experience with Azure Monitor, Log Analytics, operational alerting, and platform observability.

- Cloud Breadth: Exposure to AWS or GCP data platforms in addition to Azure.

- Modern Data Engineering: Experience with CDC, streaming architectures, reusable ingestion frameworks, and platform engineering practices.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Data Engineer - Architect (Hyderabad)
🏢 Nstarx India
📍 Hyderabad