12 Oct
|
Spasnik Solutions
|
Hyderabad
12 Oct
Spasnik Solutions
Hyderabad
Job Title: Cloud Data Engineer
Experience: 5 Years
We are looking for an experienced Cloud Data Engineer with strong hands-on experience in Databricks, AWS, data lake architecture, SQL/PL-SQL, and contemporary data engineering practices.
The role will focus on building and maintaining scalable cloud data solutions, supporting data warehouse migration and consolidation, developing data pipelines, and transforming legacy business logic into scalable solutions using Databricks, Spark SQL, Delta Lake, and AWS S3.
Key Responsibilities
- Analyze and understand existing data warehouse implementations to support migration and consolidation initiatives.
- Reverse-engineer legacy PL/SQL and SQL stored procedures and translate business logic into scalable Spark SQL code within Databricks notebooks.
- Design and develop data lake solutions on AWS using S3 and Delta Lake.
- Leverage Databricks for data processing, transformation, and development of data products.
- Build and maintain robust data pipelines using ETL tools, with ingestion into S3 and processing in Databricks.
- Collaborate with data architects to implement ingestion and transformation frameworks aligned with enterprise standards.
- Evaluate and optimize data models including Star, Snowflake, and Flattened models for performance and scalability.
- Document ETL processes, data flows, and transformation logic.
- Perform data administration activities including job scheduling, error troubleshooting, performance tuning, and backup coordination.
- Work closely with cross-functional teams to support the transition and integration of data sources into a unified platform.
- Triage, debug,
and resolve technical issues related to data lakes and data pipelines.
- Maintain and manage code repositories using Git.
- Participate in Agile ceremonies including sprint planning, retrospectives, backlog grooming, and other development activities.
- Contribute to the scalability, reliability, and cost optimization of the cloud data platform.
Required Skills
- 5 years of hands-on experience with Databricks.
- Strong experience with Spark SQL and Delta Lake.
- 3 years of experience designing and implementing data lake architectures on Databricks.
- Strong SQL and PL/SQL skills.
- Ability to interpret, reverse-engineer, and refactor legacy stored procedures.
- Hands-on experience with data modeling and data warehouse design principles.
- Proficiency in at least one programming language such as Python, Scala, or Java.
- Strong understanding of AWS data lake concepts, particularly Amazon S3.
- Experience building and maintaining ETL/data pipelines.
- Experience working in Agile environments and iterative development cycles.
- Strong troubleshooting and problem-solving skills.
- Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
Good to Have
- Databricks certification.
- Exposure to enterprise data governance.
- Knowledge of metadata management practices.
- Experience with cloud data platform migration or modernization.
- Experience with performance and cost optimization of cloud data platforms.
Interested candidates can share their updated resume at [email protected].
Pay: ₹1,500,000.00 - ₹2,500,000.00 per year
Work Location: Hybrid remote in Hyderabad, Telangana
📌 Cloud Data Engineer (Hyderabad)
🏢 Spasnik Solutions
📍 Hyderabad