04 Oct
|
Durapid Technologies Private
|
India
04 Oct
Durapid Technologies Private
India
Job Summary We are looking for an experienced Senior ETL / Data Engineer with solid expertise in PySpark, modern cloud data platforms, NoSQL databases, Graph Databases, and large-scale ETL pipelines . The ideal candidate should have hands-on experience designing, developing, and optimizing scalable data processing solutions across cloud data warehouse and data lake environments.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using PySpark and Python.
- Develop and optimize large-scale data processing jobs using Apache Spark / PySpark.
- Work with cloud data platforms such as Databricks, Snowflake, and Amazon Redshift.
- Build data ingestion and transformation pipelines from structured and unstructured data sources.
- Work with NoSQL databases for high-volume and distributed data workloads.
- Design and implement solutions using Graph Databases and graph-based data models.
- Perform data transformation, cleansing, validation, reconciliation, and quality checks.
- Optimize ETL pipelines, Spark jobs, queries, and data storage for performance and scalability.
- Implement data integration across databases, APIs, files, data lakes, and cloud platforms.
- Troubleshoot data pipeline failures and perform root-cause analysis.
- Collaborate with Data Architects, Data Scientists, BI teams, and application teams.
Required Skills
- 7+ years of experience in Data Engineering / ETL development.
- Strong hands-on experience with PySpark / Apache Spark.
- Experience with one or more cloud data platforms: Databricks, Snowflake, or Amazon Redshift.
- Strong experience in ETL/ELT pipeline development.
- Hands-on experience with NoSQL databases.
- Experience with Graph Databases and graph data modeling.
- Strong SQL and Python skills.
- Experience with data transformation, data integration, and performance optimization.
- Good understanding of data warehousing and data lake concepts.
- Experience working with large-scale datasets and distributed processing environments.
Good to Have Databricks | Snowflake | Amazon Redshift | PySpark | Apache Spark | NoSQL | Graph Database | Python | SQL | ETL/ELT | Data Lake | Data Warehouse | Cloud Data Engineering Skills: databricks,graph databases,cloud data engineering,nosql,amazon redshift,snowflake,python,sql,data lakes,data warehouse
📌 Senior ETL / Data Engineer (India)
🏢 Durapid Technologies Private
📍 India