Design and optimize large scale data processing solutions as a senior developer using Spark and AWS based services in a hybrid work arrangement. Apply advanced expertise in Spark optimization AWS Glue EMR Redshift S3 IAM and CloudWatch to build reliable data pipelines and analytical platforms. Collaborate with cross functional teams in day shifts to deliver secure efficient and business aligned data products that support strategic decision making.
Responsibilities
Design and implement scalable data processing pipelines using Apache Spark and AWS Glue to efficiently transform and prepare data for analytics and reporting across the organization.
Optimize Spark jobs by tuning configurations improving partition strategies and refining data formats to consistently reduce runtime control resource consumption and improve overall reliability of data workflows.
Build and manage AWS EMR clusters to run complex distributed data processing workloads while maintaining cost efficiency and ensuring high performance for critical business processes.
Develop robust data ingestion and storage solutions using Amazon S3 as the central data lake enforcing folder structures and retention practices that support governance reusability and effective querying.
Create and maintain analytical data models in Amazon Redshift that enable fast and accurate reporting ensuring proper distribution keys sort keys and compression settings are applied for query performance optimization.
Configure and maintain AWS CloudWatch monitoring for data pipelines and compute resources setting up detailed metrics and alarms that enable proactive detection and resolution of operational issues before they impact business stakeholders.
Administer AWS IAM policies for data engineering components to ensure secure access control applying fine grained permissions that protect sensitive data while allowing teams to work efficiently within approved boundaries.
Collaborate with product owners analysts and engi
📌 Sr. Developer (Chennai)
🏢 Cognizant
📍 Chennai
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.