Job Description Data Engineer (Databricks | Snowflake | Python)

Position: Senior Data Engineer – Databricks, Snowflake & Python

Experience: 5–10 Years

Location: Hyderabad, India

Employment Type: Full-Time

About the Role

We are seeking a highly skilled Data Engineer with strong expertise in Databricks, Snowflake, and Python to design, build, and optimize scalable data platforms and modern data pipelines. The ideal candidate should have hands-on experience working with cloud-based data engineering solutions, data lakes, ETL/ELT frameworks, and large-scale analytics platforms.

This role requires close collaboration with data architects, analysts, AI/ML engineers, and business stakeholders to deliver reliable, high-performance data solutions.

Key Responsibilities

- Design and develop scalable data pipelines using Databricks and Python.
- Build and optimize ETL/ELT workflows for structured and semi-structured data.
- Develop and maintain data models in Snowflake.
- Implement Delta Lake architecture and data lakehouse solutions.
- Create high-performance Spark applications using PySpark.
- Develop reusable data ingestion frameworks.
- Optimize Snowflake warehouses for performance and cost.
- Integrate multiple data sources including APIs, databases, files, and streaming platforms.
- Implement data quality, validation, and monitoring frameworks.
- Build orchestration workflows using Azure Data Factory, Airflow, or similar tools.
- Collaborate with BI teams to support reporting and analytics.
- Implement CI/CD pipelines for data engineering projects.
- Ensure security, governance, and compliance of enterprise data.
- Troubleshoot production issues and optimize existing data pipelines.
- Mentor junior engineers and participate in design reviews.

Required Skills

Databricks





- Azure Databricks / AWS Databricks
- Delta Lake
- Unity Catalog
- Databricks Workflows
- Databricks SQL
- Delta Live Tables (DLT)
- MLflow (preferred)
- Lakehouse Architecture

Snowflake

- Snowflake Architecture
- Virtual Warehouses
- Data Sharing
- Snowpipe
- Streams & Tasks
- Time Travel
- Cloning
- Snowpark
- Query Optimization
- Performance Tuning
- Data Modeling
- RBAC Security

Python

- Advanced Python Programming
- PySpark
- Pandas
- NumPy
- REST API Integration
- Object-Oriented Programming
- Logging & Exception Handling
- Unit Testing
- Packaging & Modular Development

SQL

- Advanced SQL
- Complex Joins
- Window Functions
- CTEs
- Stored Procedures
- Performance Optimization

Cloud Platforms (One or More)

- Microsoft Azure
- AWS
- Google Cloud Platform

Data Engineering

- ETL / ELT
- Data Warehousing
- Data Lake
- Lakehouse
- Data Modeling
- Batch Processing
- Streaming Data
- Metadata Management
- Data Governance

DevOps

- Git
- Azure DevOps
- Jenkins
- GitHub Actions
- CI/CD Pipelines
- Docker (preferred)

Preferred Technologies

- Azure Data Factory
- Apache Airflow
- Kafka
- Event Hub
- Terraform
- Kubernetes
- Power BI
- Tableau
- dbt
- Great Expectations
- Microsoft Fabric (preferred)

Qualifications

- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
- 5–10 years of experience in Data Engineering.




- At least 3 years of hands-on experience with Databricks.
- At least 3 years of experience with Snowflake.
- Strong Python development experience.
- Excellent SQL skills.
- Experience with cloud-native data platforms.
- Knowledge of data security and governance best practices.

Nice to Have

- Microsoft Fabric
- Azure Synapse Analytics
- AI/ML pipelines
- Generative AI data preparation
- LangChain
- Vector Databases
- OpenAI APIs
- Apache Iceberg
- Apache Hudi

Soft Skills

- Excellent communication skills
- Robust analytical and problem-solving abilities
- Ability to work independently
- Stakeholder management
- Team collaboration
- Agile/Scrum experience
- Documentation skills

Success Metrics

- High-performance, scalable data pipelines
- Optimized Snowflake compute costs
- Reliable and automated ETL workflows
- High data quality and governance standards
- Timely delivery of data engineering projects
- Strong collaboration with analytics and AI teams

Preferred Candidate Profile

- Hands-on experience with modern cloud data platforms.
- Strong understanding of Lakehouse architecture and enterprise data management.
- Experience supporting AI/ML workloads using Databricks and Snowflake.
- Ability to design scalable, secure, and cost-efficient data solutions.
- Exposure to healthcare, life sciences, retail, or financial services domains is an advantage.

Expected Certification

- Databricks: Databricks Certified Data Engineer Associate (minimum), Professional preferred.
- Snowflake: SnowPro Core (minimum), SnowPro Advanced: Data Engineer preferred.
- Cloud: Azure DP-203 or AWS Data Engineer – AssociateRole & responsibilities

Preferred candidate profile

📌 Senior Data Engineer-Snowflake,Databricks,Python (Telangana)
🏢 Avirasoft Digital Technologies
📍 Telangana