04 Sep
|
Virtusa
|
Hyderabad
Key Responsibilities
Data Pipeline Architecture Design, build, and optimize scalable ELT/ETL data pipelines to ingest structured and semi-structured data from various sources into Snowflake.
Data Transformation & Processing Write efficient, production-grade PySpark and Python code to process, clean, and transform large-scale datasets.
Snowflake Management Develop and optimize data models, views, and stored procedures within Snowflake. Implement best practices for data warehousing, warehouse sizing, query performance tuning, and cost optimization.
Database Management Write advanced SQL queries for complex data manipulation, aggregation, and troubleshooting.
Data Quality & Governance Implement data validation rules, monitor pipeline health, and ensure data integrity, security, and compliance across all data environments.
Collaboration Work closely with cross-functional teams, including Data Scientists, Data Analysts, and business stakeholders, to translate business requirements into technical data solutions.
Required Qualifications & Skills
Experience Minimum of 5 years of hands-on experience in Data Engineering, Data Warehousing, or a closely related role.
SQL Advanced proficiency in SQL with a strong ability to write complex queries, optimize performance, and model data effectively.
Python Solid programming skills in Python for data manipulation, scripting, and automation.
PySpark Extensive experience using PySpark for distributed data processing and large-scale data transformations.
Snowflake Deep expertise in Snowflake Cloud Data Warehouse, including its unique architecture, features (e.g., Snowpipe, Streams, Tasks, Time Travel), and query optimization techniques.
Education Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or a related field.
Preferred / Nice-to-Have Skills
Experience with cloud computing platforms such as AWS (Glue, S3, Lambda)
Familiarity with data orchestration and scheduling tools like Apache Airflow.
Knowledge of modern data engineering tools like dbt (Data Build Tool).
Experience using Version Control (Git) and deploying code through CI/CD pipelines.
Relevant certifications (e.g., SnowPro Core Certification, Databricks Certified Data Engineer).
📌 Tech Lead (Hyderabad)
🏢 Virtusa
📍 Hyderabad