Job Title: Hadoop data engineer:

Desired Competencies (Technical/Behavioral Competency)

Must-Have:

- Robust experience in programming languages such as Python, Java, or Scala for data manipulation and engineering tasks.
- Expertise in SQL and NoSQL databases
- Hands-on experience with big data technologies like Hadoop, Spark, Kafka, and Hive to handle large-scale data processing and real-time data streams.
- In-depth knowledge of data warehousing solutions such as Amazon Redshift, Google BigQuery, and Snowflake for building and managing data warehouses.
- Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using tools like Apache NiFi, Talend, or Informatica.

Good-to-Have:

- Familiarity with cloud platforms like AWS, Azure, or Google Cloud for deploying, managing, and scaling data infrastructure and services.
- Strong understanding of data modeling concepts and techniques to create efficient and scalable data models.
- Experience with version control systems such as Git for code management and collaboration.




- Knowledge of data governance, data quality standards, and data security practices to ensure compliance and protection of sensitive information

Responsibility of / Expectations from the Role 1 Design, develop, and maintain scalable data pipelines for extracting, transforming, and loading data from various sources to ensure seamless data flow and accessibility.

2 Collaborate with cross-functional teams to integrate data from multiple disparate sources, ensuring consistency, accuracy, and reliability of data.

3 Optimize data processing workflows and storage solutions for performance, scalability, and cost-efficiency

4 Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data throughout the data lifecycle.

5 Monitor the performance of data pipelines and infrastructure, identifying and resolving issues to maintain system stability and reliability.

📌 Hadoop Spark Scala- Data Engineer (Bengaluru)
🏢 Tata Consultancy Services
📍 Bengaluru