Role & responsibilities:
- The Data Engineer is responsible for designing, building, and optimizing our data ecosystem. With a focus on delivering high-quality and trustworthy data, the Data Engineer's responsibilities include:
- Data Ingestion and Transformation: Ingesting data from various sources, transforming, cleaning, and enriching it for use in Palantir Foundry.
- Data Pipeline Development: Designing and implementing robust and scalable data pipelines to ensure data flow within Foundry.
- Ontology Development: Contributing to the development and maintenance of the data ontology within Foundry.
- Data Integration: Integrating diverse data sources into Palantir Foundry using its data connectors.
- Collaboration: Working with cross-functional teams to understand business requirements and develop solutions using Palantir Foundry.
- Problem Solving: Identifying and resolving issues related to data quality, access, and performance.
- Application Development:
Potentially contributing to the development of applications within the Foundry ecosystem.
- Data Analytics: Leveraging data analytics and machine learning for insights and solutions.
- Knowledge Sharing: Stay at the forefront of Big Data technology evolution and ensure this knowledge is disseminated across the organization. Promote best practices, stablished standards, governed processes, and the adoption of the latest technologies.
Must have skills:
- Expertise and experience in Python and Pyspark (at least 4 years of experience)
- Experience with BI tools, SQL queries, and organizing and analyzing data
- Should have working knowledge and hands-on experience on the following Palantir Tools - Data Connection, Ontology, Code Workbook/Code Repository, Contour, Quiver, Workshop and Slate