Data Engineer

# Data Engineer ## Company Overview *Company overview is not specified.* ## Job Summary The Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines. This role is crucial for processing large volumes of data efficiently and developing enterprise-grade data solutions on the Azure cloud platform, contributing to the organization's data-driven decision-making processes. ## Responsibilities - Design and implement complex data pipelines using Azure Data Factory and Databricks. - Write and optimize complex SQL queries to manage and manipulate data effectively. - Architect and build enterprise-grade data platforms leveraging Azure Data Services, including ADLS Gen2 and Synapse Serverless. - Gather requirements and translate them into technical architectures while assessing the pros and cons of different architectural choices. - Employ streaming technologies such as Kafka and Spark Structured Streaming for real-time data processing. - Implement Infrastructure as Code (IaC), Continuous Integration/Continuous Deployment (CI/CD), and DevOps practices in data analytics projects. ## Qualifications - Bachelor’s degree in Computer Science, Information Technology, or a related field. - In-depth knowledge of Data Engineering principles and practices, particularly on Azure Databricks. - Strong programming skills with proficiency in Python. - Extensive hands-on experience in building complex data pipelines within Azure Data Factory. - Well-versed in Azure Data Services such as ADLS Gen2 and Synapse. - Multi-year experience (over 6 years) in a data engineering role, including expertise in data processing and transformation. - Familiarity with cloud computing concepts, particularly those relevant to the Azure platform, including networking and security practices. - Experience with IaC tools and practices, preferably using Azure DevOps and Terraform. ## Preferred Skills - Knowledge of Microsoft Fabric would be considered an advantage. - Familiarity with Big Data solution architectures and best practices in cloud services selection (e.g., ADF, Databricks, Synapse). ## Experience - A minimum of 6 years in a data engineering role with a strong track record of designing and maintaining large-scale data solutions. ## Environment *Typical work setting and location are not specified.* ## Salary *Salary details are not provided.* ## Growth Opportunities *Opportunities for career advancement within the organization are not specified.* ## Benefits *Details on offered benefits are not provided.*