Senior Data Engineer (Databricks)
Posted Updated
Description
We're currently recruiting on behalf of one of our clients a leading global technology company specializing in digital transformation, agile software development, and IT services. With offices across Romania and a strong international presence, our client partners with some of the worlds top brands to deliver innovative, high-quality digital solutions.
Job Description
- Work together with data scientists and analysts to understand the needs for data and create effective data pipelines
- Solution design, implement, and maintain data pipelines for data ingestion, processing, and transformation using could services and other technologies
- Create and maintain data solutions IN Azure, using any of the following: Data Factory, Synapse, Databricks/Spark, Polars/Pandas, Fabric
- Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.
- Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
- Monitoring and resolving data pipeline problems, ensuring consistency and availability of the data.
Qualifications
- Proficient in Python and SQL
- Knowledge of good data design principles (concepts such as Kimball and Star Schema), data warehousing concepts and data modelling.
- Strong experience with Relational and ideally NoSQL databases.
- Strong understanding of ETL/ELT processes.
- Exposure to data monitoring and observability tools.
- Relevant experience with Event-Driven Architecture
- Good understanding of Agile ways of working