07 Oct
|
McBird Technologies
|
Bengaluru
07 Oct
McBird Technologies
Bengaluru
Introduction
We are seeking an experienced Data Engineer with deep expertise in AWS-based data engineering, ETL pipeline design, and large-scale data management.
The ideal candidate will have a solid foundation in Python, SQL, and AWS data services, and will be responsible for architecting and maintaining data systems that power advanced analytics and machine learning initiatives.
You will work closely with data scientists, analysts, and DevOps teams to ensure robust, scalable, and secure data infrastructure across the organization.
Key Responsibilities
• Design, build, and optimize data ingestion and transformation pipelines using AWS-native tools (Glue,
Lambda, Step Functions, EMR, etc.).
• Develop and manage data lakes and data warehouses using Amazon S3, Redshift, and Athena.
• Implement ETL/ELT workflows for structured, semi-structured, and unstructured data at scale.
• Create and maintain data models and schemas that support analytics and ML workflows.
• Work with AWS Glue Catalog, Lake Formation, and AWS DataBrew for data cataloging, cleaning,
and governance.
• Collaborate with data scientists to prepare feature-ready datasets for model training and inference.
• Implement data quality, validation, and monitoring mechanisms to ensure accuracy and reliability.
• Optimize data performance and costs through partitioning, compression, and indexing strategies.
• Automate deployment and orchestration of data workflows using AWS CodePipeline, Docker, and CI/CD
best practices.
• Ensure compliance, data security, and access control in alignment with AWS IAM and governance
standards.
Required Skills
• 3+ years of hands-on experience in Data Engineering, with at least 3 years in AWS
environments.
• Proficiency in Python (pandas, PySpark, boto3) and SQL for data transformation and orchestration.
• Expert-level knowledge of AWS data services:
o AWS Glue, AWS Lambda, Amazon S3, Amazon Redshift, Athena, Kinesis, and Step Functions.
• Solid understanding of data lake and
📌 Data Engineer Bengaluru
🏢 McBird Technologies
📍 Bengaluru