Senior Data Engineer

# Senior Data Engineer ## Company Overview *Company details are not specified.* ## Job Summary The Senior Data Engineer will design and implement high-performance, scalable, and optimized data solutions for large enterprise-wide data mining and processing. This role focuses on the development of data flows and the deployment and implementation of Big Data Platforms, contributing significantly to the organization’s data strategy. ## Responsibilities - Design and propose **end-to-end data pipelines** for data projects, covering ingestion, storage, and consumption, as well as databases to support web-based applications. - Develop and implement **data warehouses and data marts** to serve data consumers efficiently. - Execute implementations from design stages to operationalization, ensuring effective maintenance post-implementation. - Perform **database design and modeling** to organize data at macro and micro levels and provide logical data models for consumers. - Conduct **database performance tuning** and manage data lifecycles to enhance efficiency. - Assist in **support and enhancements** of existing data pipelines and databases. ## Qualifications - 6-10 years of total experience working with data integration teams. - 3+ years of experience developing **data pipelines** within an **Apache Spark** environment, preferably using **Databricks**. - 2+ years of active work specifically within **Databricks**, showcasing in-depth experience. - 2+ years of experience in working with **data warehouses** and familiarity with data warehouse modeling techniques. - Strong knowledge of **PySpark**, **Python**, and **SQL**, along with distributed computing principles. - Proficient in **data modeling**, database technologies, and data warehousing concepts. - Experience with **ETL/ELT processes**, including design and implementation via tools like SSIS or similar. - Familiar with cloud platforms such as **AWS** or **Azure** and big data technologies like **Hadoop** and **Spark**. - Expertise in **complex SQL query performance tuning** and overall database performance tuning. ## Preferred Skills - Experience developing data solutions utilizing **native IaaS** and **PaaS solutions** on AWS (e.g., Redshift, RDS, S3) is considered advantageous. ## Experience - Required years of experience: **6-10 years** of relevant experience in data engineering and integration. ## Environment *Work setting, location (remote, in-office, hybrid), and physical conditions are not specified.* ## Salary *Salary details are not specified.* ## Growth Opportunities *Opportunities for career advancement within the company are not specified.* ## Benefits *Details regarding offered benefits are not specified.*