Data Engineer - Data Bricks

# Data Engineer - Data Bricks ## Company Overview Our client, a leading Insurance company, is at the forefront of providing innovative solutions in the insurance industry. They are dedicated to harnessing the power of data to enhance operations and deliver exceptional service to their clients. ## Job Summary The Data Engineer will play a crucial role in designing and implementing data acquisition and integration processes. This position aims to support the organization’s goals by ensuring efficient data processing and transformation, critical for providing insights and improving decision-making in the P&C Insurance sector. ## Responsibilities - Define **data acquisition and integration designs** for the atomic layer in data architecture. - Configure **Pyspark data pipelines** to move and transform data across various data layers. - Create and maintain **HBase physical tables** and **Phoenix SQL tables** to support data storage needs. - Collaborate with relevant teams to ensure performance meets **Service Level Agreements (SLAs)**. - Confirm readiness of data for feeds and conversions based on target system and business process requirements. - Apply **Data Vault methodology** with prior experience in insurance data model on Azure with **Pyspark coding**. ## Qualifications - **8+ years** of experience in data engineering or related roles. - Proficiency in **Data Vault**, **Pyspark**, **Azure Data Factory (ADF)**, and **Python**. - Experience with **Databricks** as part of the data engineering process. - Strong understanding of the **P&C Insurance** industry and its data requirements. - Familiarity with **HBase** and **Phoenix SQL** for data storage solutions. - Ability to work independently and in a team-oriented environment. - Excellent analytical and problem-solving skills. - Degree in Computer Science, Information Technology, or a related field. ## Preferred Skills - Knowledge of additional programming languages or data processing tools. - Experience with **cloud computing platforms** and scalable data architecture. - Familiarity with data governance and compliance practices. ## Experience - Minimum of **8 years** of experience in data engineering, particularly in environments involving large datasets and complex integrations, especially within the insurance sector. ## Environment - Work will be conducted primarily during **day shifts**. - Locations available: **Pune** and **Ahmedabad**. - The position may be offered as a subcontractor or a permanent role depending on preferences. ## Salary Salary will be determined based on market standards and the candidate's experience. ## Growth Opportunities Potential career advancement opportunities within the company, including senior data engineering roles and leadership positions. ## Benefits - Comprehensive benefits package including health insurance, paid leave, and work-life balance initiatives.