- We are seeking an experienced Databricks Snowflake and PySpark Architect to design architect and deliver scalable cloud based data platforms and modern analytics solutions
- The ideal candidate will have deep expertise in data architecture cloud native technologies data engineering best practices and large scale data transformation programs
- The role involves collaborating with business stakeholders solution architects and engineering teams to build high performance secure and cost productive data ecosystems leveraging Databricks Snowflake and PySpark
Key Responsibilities:
- Define and drive enterprise scale data architecture and modernization strategies
- Design end to end cloud data platforms using Databricks Snowflake and modern data engineering frameworks
- Architect scalable batch and real time data processing pipelines for large datasets
- Lead data lake lakehouse and data warehouse implementations
- Provide architectural guidance on performance optimization scalability governance security and reliability
- Work closely with business stakeholders to understand analytical and reporting requirements
- Review technical designs conduct architecture assessments and establish best practices
- Mentor data engineers and technical leads across project engagements
- Lead technology evaluations proof of concepts POCs and solution accelerators
- Drive cloud migration and modernization initiatives from legacy data platforms
- Ensure adherence to enterprise security compliance and data governance standards
- Collaborate with DevOps and platform teams to implement CI CD and infrastructure automation for data solutions
Technical Requirements:
- 11 years of experience in Data Engineering Analytics or Data Platform Architecture
- Strong expertise in Databricks Lakehouse Platform
- Extensive hands on experience with Snowflake Data Cloud
- Expert level proficiency in PySpark and distributed data processing
- Strong experience in designing enterprise scale data lakes data warehouses and lakehouse architectures
- Hands on experience with cloud platforms such as
- Azure ADF ADLS Gen2 Synapse Azure Databricks
- AWS S3 Glue EMR Redshift
- GCP BigQuery DataProc Cloud Storage
- Strong SQL and data modeling skills Dimensional Modeling Data Vault Star Schema
- Experience building ETL ELT frameworks and reusable data engineering accelerators
- Knowledge of Data Governance Metadata Management Data Lineage and Data Quality frameworks
- Experience with orchestration tools such as Airflow Azure Data Factory or similar
- Expertise in performance tuning and cost optimization within Databricks and Snowflake environments
- Experience implementing CI CD pipelines using Azure DevOps GitHub Actions Jenkins or similar tools