20 Nov
|
Infosys
|
Chennai
Role Overview:We are looking for a highly skilled Data Engineer with expertise in designing, developing, and optimizing enterprise-scale data platforms and pipelines. The ideal candidate should possess strong experience across data engineering, cloud-native architectures, modern data platforms, streaming technologies, and AI-ready data ecosystems.Candidates with exposure to AI, Generative AI, DataOps, MLOps, Lakehouse architectures, and Real-Time Analytics will be highly preferred.Key Responsibilities:Data Platform Engineering:- Design, develop, and maintain scalable data pipelines for batch and real-time processing.- Build enterprise-grade data platforms supporting analytics, AI, and business intelligence workloads.- Develop reusable ingestion, transformation, and orchestration frameworks.- Implement scalable ETL/ELT solutions across structured and unstructured data sources.Data Architecture & Modeling:- Design contemporary Lakehouse and Data Mesh architectures.- Build optimized data models supporting reporting, analytics, and machine learning.- Establish enterprise data standards, governance, and lineage frameworks.- Ensure data quality, consistency, and regulatory compliance.Cloud Data Engineering:- Build cloud-native data platforms on AWS, Azure, or GCP.- Design scalable solutions leveraging managed data services.- Optimize performance, reliability, and cost efficiency of cloud data workloads.- Implement Infrastructure as Code and automated deployment pipelines.Real-Time Data Engineering:- Build event-driven architectures and streaming data solutions.- Develop real-time ingestion and processing pipelines.- Implement high-throughput and low-latency data processing systems.AI & GenAI Data Foundation:- Build AI-ready data ecosystems supporting machine learning and Generative AI workloads.- Engineer pipelines for feature stores, vector databases,
and RAG architectures.- Enable enterprise-scale data preparation for AI/ML models.- Collaborate with AI Engineers and Data Scientists for model operationalization.Engineering Excellence:- Drive best practices for DataOps, DevOps, CI/CD, testing, and observability.- Participate in architecture reviews and technical design discussions.- Contribute to reusable assets, accelerators, and intellectual property development.- Mentor junior engineers and support technical leadership initiatives.Required Technical Skills:Data Engineering:- Python, PySpark, SQL (Advanced), Data Modeling, ETL/ELT Development, Data Warehousing ConceptsBig Data Technologies:- Apache Spark, Databricks, Hadoop Ecosystem, Delta Lake, Apache Iceberg, Apache HudiData Integration & Streaming:- Apache Kafka, Apache Flink, Apache Airflow, Azure Data Factory, AWS Glue, Stream Processing FrameworksCloud Platforms:- Azure (Data Factory, Synapse, Data Lake, Fabric, Databricks), AWS (S3, Glue, EMR, Redshift, Kinesis, Lambda), GCP (BigQuery, Dataflow, Dataproc, Pub/Sub)Databases:- PostgreSQL, SQL Server, Oracle, MongoDB, Cassandra, SnowflakeDataOps & DevOps:- Git, CI/CD Pipelines, Jenkins, GitHub Actions, Terraform, Docker, KubernetesPreferred Skills:- Generative AI, LLMs, RAG, Vector Databases (Pinecone, Milvus, Weaviate), MLOps, Data Governance, MDM, Data Quality Frameworks, Knowledge Graphs, Agentic AI Data PlatformsEducational Qualification:- Bachelor's or Master's degree in Computer Science, Engineering, Data Science, Information Technology, or related disciplines.Candidate Profile:- Strong coding and problem-solving skills.- Deep understanding of distributed data systems.- Passionate about building large-scale data platforms.- Ability to work independently and thrive in ambiguous environments.- Experience working in Agile and cloud-native engineering teams.- Demonstrate innovation, ownership, and engineering excellence.Success Metrics:- Scalable and reliable data platform delivery.- Data quality and governance compliance.- Performance optimization and cost efficiency improvements.- Contribution to reusable frameworks, accelerators, and IP.- Enablement of Analytics, AI, and GenAI use cases.- Technical leadership and mentorship impact. (ref:hirist.tech)
📌 Infosys - Senior Data Engineer - Python/Big Data (Chennai)
🏢 Infosys
📍 Chennai