06 Sep
|
TekIT Software Solutions (India & USA
|
Bengaluru
06 Sep
TekIT Software Solutions (India & USA
Bengaluru
Job DescriptionData Engineer – Data Platform Location: Bangalore Experience: 5+ Years Role Overview We are looking for a hands-on Data Engineer to design, develop, and optimize scalable data platforms and data pipelines . The ideal candidate will have solid expertise in Python, SQL, PySpark/Spark, Databricks , and modern data engineering practices. Candidates should have hands-on experience with at least one cloud platform – AWS, Azure, or GCP . Experience with BigQuery, Lakehouse architecture, Delta Lake, Airflow, Kafka, and CDC will be highly valued. Key Responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines using Python, SQL, PySpark, and Apache Spark . Build and optimize data processing solutions using Databricks and cloud-native data services. Develop batch and real-time data ingestion pipelines using technologies such as Airflow, Kafka, CDC , and other streaming/integration frameworks. Implement modern Lakehouse and Medallion Architecture using Delta Lake or similar technologies.
Develop reliable data pipelines for structured and semi-structured data from multiple sources. Implement data quality, validation, reconciliation, monitoring, logging, and error-handling mechanisms. Optimize Spark jobs and data pipelines for performance, scalability, reliability, and cloud cost efficiency . Work with cloud storage, compute, databases, and data services across AWS, Azure, or GCP . Implement engineering best practices including Git, CI/CD, unit testing, integration testing, and code reviews . Troubleshoot data pipeline failures and resolve performance and data-quality issues. Collaborate with Data Architects, Solution Architects, Analysts, Developers, and Business Stakeholders to understand requirements and deliver scalable data solutions. Contribute to technical design, documentation, coding standards, and continuous improvement of the data platform. Must-Have Skills 5+ years of hands-on experience in Data Engineering . Strong programming skill
📌 Sr. Data Engineer (Bengaluru)
🏢 TekIT Software Solutions (India & USA
📍 Bengaluru