- Design, develop, and implement scalable data pipelines to process large datasets from diverse sources, while handling data in large-scale architectures
- Develop and implement automated data validation and testing frameworks to ensure data accuracy and consistency
- Implement best practices for security, automation, and error handling using tools like Apache Kafka and data warehousing or lake technologies
- Research and implement new tools, technologies, and methodologies and integrate these into production systems, ensuring scalability and reliability
- Apply creative problem-solving techniques to design creative tools and optimized workflows and independently apply data-driven approaches when appropriate
- Independently manage and optimize data storage solutions such as data warehouses, data lakes, and cloud-based systems
- Understand technical tools and frameworks used by the team, including programming languages, libraries, and platforms and actively support debugging or refining code in projects
- Contribute to the design and documentation of data engineering solutions, clearly detailing methodologies, assumptions, and findings for future reference and cross-team collaboration
- Collaborate across teams to develop and implement high-quality, scalable engineering solutions that align with business goals, address user needs, and improve performance

Data Integration and ETL (Extract, Transform, Load) practices: Combining data from different sources into a single, unified view, and the process of extracting data from sources, transforming it into a suitable format, and loading it into a destination system.

Proficiency Level: Proficient

Data Analysis: The process of inspecting, cleansing, transforming, and modeling data to discover useful information, draw conclusions, and support decision-making

Proficiency Level: Proficient





Data Processing Frameworks: Tools and libraries used to process large data sets efficiently, such as Apache Hadoop and Apache Spark.

Proficiency Level: Foundational

Programming: Writing code to manipulate, analyze, and visualize data, often using languages like Python, R, and SQL.

Proficiency Level: Proficient

Data Modelling: The process of creating a data model for the data to be stored in a database, representing the data structures and their relationships.

Proficiency Level: Proficient

Data Quality Management: Ensuring that data is accurate, complete, reliable, and relevant for its intended use.

Proficiency Level: Proficient

SPECIALIZED SKILLS

Data Management: The development and execution of architectures, policies, practices, and procedures to manage the data lifecycle needs of an enterprise.

Big Data Technologies: Tools and techniques used to process and analyze large and complex data sets that traditional data-processing software cannot handle

Database Design and Management: The process of designing, implementing, and maintaining a database system to ensure data is stored efficiently and securely.

Data Visualization: The graphical representation of data to help people understand complex data sets and derive insights.

Data Security: Protecting data from unauthorized access, use, disclosure, disruption, modification, or destruction.

Data Storage solutions: Systems and technologies used to store data,



such as databases, data warehouses, and cloud storage.

Data Governance and Compliance: The management of data availability, usability, integrity, and security in an organization, based on internal data standards and policies. Ensuring that an organization adheres to external regulations and internal policies, managing risk, and maintaining ethical standards

Technical Documentation: Creating and maintaining documentation that explains the functionality, use, and maintenance of software or systems.

Data Architecture: The design and structure of data systems, ensuring that data is stored, managed, and utilized efficiently

CI/CD & Observability: The use of continuous integration and continuous delivery (CI/CD) pipelines to automate the process of software development, including building, testing, and deploying code

Definition of Proficiency Levels

Foundational: This is the entry level of the skill, typically expected when starting a new role or working with the skill for the first time. You rely on strong manager support, coaching, and training as you build the capability to progress to higher proficiency levels.

Proficient: This is the level at which you are considered effective in the skill. You demonstrate more than just functional competenceyou begin to have a noticeable impact in your role by applying the skill consistently and meaningfully. You require only minimal support, coaching, or training to apply the skill successfully.

Advanced: This is the level where you move beyond meeting expectations to actively leading, influencing, and delivering considerable impact across the wider business. You are seen as a role model, demonstrate the skill independently, and require little to no manager support.

📌 Data Engineer (Bengaluru)
🏢 A.P. Moller - Maersk
📍 Bengaluru