Education Qualification: Bachelor's degree in Computer Science or related field or higher with minimum 3 years of relevant experience.
- Position Description: Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and PySpark.
- Develop data processing solutions using Python and Spark for large-scale data transformation.
- Write complex SQL queries for data extraction, transformation, validation, and reporting.
- Build and optimize data workflows to improve performance, scalability, and reliability.
- Work with structured and semi-structured data from multiple data sources.
- Collaborate with data analysts, business stakeholders, and development teams to understand data requirements.
- Perform data quality checks, troubleshooting, and root cause analysis for data issues.
- Optimize Spark jobs and SQL queries for improved performance.
- Follow coding standards, documentation, and best practices for data engineering.
- Participate in code reviews and support production deployments.
Responsibilities:
- Minimum 3 years of experience in Data Engineering.
- Solid hands-on experience with Databricks.
- Good experience in PySpark and Apache Spark.
- Strong programming skills in Python.
- Proficiency in SQL, including complex queries, joins, window functions, and performance tuning.
- Experience in developing ETL/ELT pipelines.
- Knowledge of data warehousing concepts and data modeling.
- Experience with Git or other version control systems.
- Strong analytical, problem-solving, and debugging skills.
LI-IP2
Must-Have Skills:
- Strong experience Databricks
- Hands-on experience Data & Delta Lake
- Experience with PySpark and Apache Spark.
- Experience with Python.
- Experience with ETL/ELT pipeline
📌 Software Data Engineer (Hyderabad)
🏢 CGI
📍 Hyderabad
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.