Role & responsibilities

- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and PySpark.
- Develop data processing solutions using Python and Spark for large-scale data transformation.
- Write complex SQL queries for data extraction, transformation, validation, and reporting.
- Build and optimize data workflows to improve performance, scalability, and reliability.
- Work with structured and semi-structured data from multiple data sources.
- Collaborate with data analysts, business stakeholders, and development teams to understand data requirements.
- Perform data quality checks, troubleshooting, and root cause analysis for data issues.
- Optimize Spark jobs and SQL queries for improved performance.
- Follow coding standards, documentation, and best practices for data engineering.




- Participate in code reviews and support production deployments.

Required Skills

- Minimum 3 years of experience in Data Engineering.
- Solid hands-on experience with Databricks.
- Good experience in PySpark and Apache Spark.
- Strong programming skills in Python.
- Proficiency in SQL, including complex queries, joins, window functions, and performance tuning.
- Experience in developing ETL/ELT pipelines.
- Knowledge of data warehousing concepts and data modeling.
- Experience with Git or other version control systems.
- Strong analytical, problem-solving, and debugging skills.

Preferred candidate profile

📌 Data bricks engineer (Telangana)
🏢 CGI
📍 Telangana