Key Responsibilities

Data Pipeline Architecture: Design, build, and optimize scalable ELT/ETL data pipelines to ingest structured and semi-structured data from various sources into Snowflake.

Data Transformation & Processing: Write efficient, production-grade PySpark and Python code to process, clean, and transform large-scale datasets.

Snowflake Management: Develop and optimize data models, views, and stored procedures within Snowflake. Implement best practices for data warehousing, warehouse sizing, query performance tuning, and cost optimization.

Database Management: Write advanced SQL queries for complex data manipulation, aggregation, and troubleshooting.

Data Quality & Governance: Implement data validation rules, monitor pipeline health, and ensure data integrity, security, and compliance across all data environments.

Collaboration: Work closely with cross-functional teams, including Data Scientists, Data Analysts, and business stakeholders, to translate business requirements into technical data solutions.

Required Qualifications & Skills

Experience: Minimum of 5 years of hands-on experience in Data Engineering, Data Warehousing, or a closely related role.

SQL:



Advanced proficiency in SQL with a strong ability to write complex queries, optimize performance, and model data effectively.

Python: Robust programming skills in Python for data manipulation, scripting, and automation.

PySpark: Extensive experience using PySpark for distributed data processing and large-scale data transformations.

Snowflake: Deep expertise in Snowflake Cloud Data Warehouse, including its unique architecture, features (e.g., Snowpipe, Streams, Tasks, Time Travel), and query optimization techniques.

Education: Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or a related field.

Preferred / Nice-to-Have Skills

Experience with cloud computing platforms such as AWS (Glue, S3, Lambda)

Familiarity with data orchestration and scheduling tools like Apache Airflow.

Knowledge of modern data engineering tools like dbt (Data Build Tool).

Experience using Version Control (Git) and deploying code through CI/CD pipelines.

Relevant certifications (e.g., SnowPro Core Certification, Databricks Certified Data Engineer).

📌 Lead Software Engineer (Bengaluru)
🏢 Virtusa
📍 Bengaluru