Responsibilities
Design, develop, and maintain scalable data pipelines using PySpark, Spark, Databricks, Azure Data Factory (ADF), and AWS Glue.
Build and optimize ETL/ELT workflows for processing large-scale structured and unstructured data.
Develop and manage cloud-based data solutions on AWS, Azure, and GCP platforms.
Work with Databricks, Delta Lake, ADLS, S3, BigQuery, Redshift, and Snowflake for data ingestion, transformation, and storage.
Implement data quality, monitoring, and performance optimization best practices.
Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver scalable solutions.
Ensure data security, governance, and compliance standards are followed across cloud platforms.
Optimize Spark jobs and SQL queries for improved performance and cost efficiency.
Support real-time and batch data processing frameworks.
Participate in code reviews, troubleshooting, and production support activities.
Create technical documentation and maintain data engineering standards and best practices.

📌 Data Engineer Pyspark Databricks Azure Data Factory Hyderabad
🏢 Infosys
📍 Hyderabad