We are seeking a skilled Big Data Engineer to join our team in India. The ideal candidate will have extensive experience in designing and implementing robust big data solutions that support our data-driven decision-making processes.
Job description
Design, build, and maintain scalable ETL/ELT pipelines using AWS Glue, Lambda, Step Functions, and Apache Spark .
Develop and optimize data lake and data warehouse architectures using AWS S3, Redshift, and Athena .
Implement data ingestion, transformation, and storage solutions using AWS Glue, Kinesis, and Kafka .
Work with structured and unstructured data to develop productive data pipelines .
Optimize query performance in SQL-based and NoSQL databases (Redshift, DynamoDB, Aurora, PostgreSQL, etc.).
Ensure data security, compliance, and governance using IAM roles, KMS encryption, and AWS Lake Formation .
Automate data workflows using Terraform, CloudFormation, or CDK .
Monitor and troubleshoot data pipelines,



ensuring high availability and performance.
Collaborate with Data Scientists, Analysts, and Engineers to provide robust data solutions.

Required Skills:
Robust experience with AWS Data Services (Glue, Redshift, S3, Lambda, Kinesis, Athena).
Proficiency in Python, SQL, and Scala
Experience with ETL/ELT pipeline design and data warehousing concepts.
Strong knowledge of relational and NoSQL databases (PostgreSQL, MySQL, DynamoDB).
Familiarity with DevOps practices and CI/CD tools (Jenkins, GitHub Actions).
Experience with Kafka or Kinesis for real-time data streaming.
Knowledge of Terraform, CloudFormation, or CDK for infrastructure automation.
Hands-on experience with data security and governance .
Solid analytical and problem-solving skills

📌 Big Data Engineer Pune
🏢 Purview India Consulting And Services
📍 Pune