Big Data Engineer (Databricks)
Posted Updated
Description
- Develop and maintain data pipelines using Databricks.
- Support the design and optimization of Delta Lake tables.
- Work with Unity Catalog to support data governance and access management.
- Participate in CI/CD processes using Azure DevOps and Databricks Asset Bundles (DAB).
- Implement incremental data processing and support schema evolution requirements.
- Collaborate with team members to troubleshoot data quality and performance issues.
- Assist with data modeling activities and platform optimization efforts.
- Document solutions and follow engineering best practices.
- Ideally 5 YOE
- Strong expertise in Apache Spark, including delivery on Databricks
- Experience with Databricks Asset Bundles (DAB).
- Experience using Azure DevOps.
- Knowledge of incremental processing and schema evolution.
- Healthcare or Insurance industry experience.
- Experience with cloud-based data platforms.
- Contrato a largo plazo.
- 100% Remoto.
- Vacaciones y PTOs
- Posibilidad de recibir 2 bonos al año.
- 2 revisiones salariales al año.
- Clases de inglés.
- Equipamiento Apple.
- Plataforma de cursos en linea
- Budget para compra de libros.
- Budget para compra de materiales de trabajo
- mucho mas..