16 Aug
|
Cortex Consultants
|
India
16 Aug
Cortex Consultants
India
Location: Remote
Exp: 5 to 8 years
BDatabricks Data Engineer-
We are seeking a skilled and experienced SQL to Databricks Migration Specialist to join our team. The primary responsibility of this role is to migrate SQL-based Crystal Reports to Databricks, ensuring seamless integration and optimal performance. The ideal candidate should have a strong background in SQL, Databricks, and Crystal Reports, along with excellent problem-solving skills and attention to detail.
Responsibilities: As a SQL to Databricks Migration Specialist for Crystal Reports, your key responsibilities will include:
1. Analyzing Existing Crystal Reports: - Reviewing existing Crystal Reports that utilize SQL queries. - Identifying the underlying data sources and dependencies. - Assessing the complexity of each report and its compatibility with Databricks.
2. Designing Data Models on Databricks: - Collaborating with stakeholders (business analysts, developers) to understand reporting requirements. - Translating existing SQL queries into optimized SparkSQL or Pyspark code for execution on Databricks.
- Designing efficient data models using Delta Lake or other appropriate storage formats.
3. Migrating Crystal Reports: - Developing migration strategies based on the analysis of existing reports. - Rewriting or modifying existing reports using Apache Spark APIs within the Databricks workplace. - Ensuring data integrity during the migration process.
4. Performance Optimization: - Profiling and optimizing migrated reports for improved performance on Databricks clusters. - Implementing best practices for query optimization, caching mechanisms, partitioning techniques, etc.
5. Documentation and Training: - Documenting the migration process including steps taken, challenges faced, and solutions implemented. Providing training sessions or documentation for end-users on how to use migrated reports in Databricks.
Qualifications: -
1. Bachelor's degree in Computer Science or related field (or equivalent wor
📌 Data Engineer (India)
🏢 Cortex Consultants
📍 India