Senior Data Engineer- Shradha

# Senior Data Engineer ## ## Job Summary We are looking for a highly experienced **Senior Data Engineer** to design and implement robust data pipelines, manage large-scale data processing, and ensure smooth orchestration using modern cloud and data technologies. The ideal candidate will have extensive hands-on experience with **Python**, **Spark**, **SQL**, and modern cloud data services. ## Responsibilities - Develop and maintain scalable **ETL/ELT** pipelines for batch and streaming data. - Design, implement, and optimize big data processing using **Spark** and **Pandas**. - Work with cloud-based tools like **Azure Data Factory** or similar for data orchestration. - Collaborate with data scientists, analysts, and other engineers to ensure clean, reliable, and timely data delivery. - Monitor and troubleshoot pipeline performance, leveraging tools like **Airflow** and cloud monitoring solutions. - Manage, transform, and query data across structured and unstructured databases. ## Qualifications - Extensive experience in data engineering and pipeline development. - Proficiency in **Python**, **Spark**, and **SQL**. - Experience with cloud data technologies (e.g., **Azure Data Services**). - Strong understanding of ETL/ELT processes. - Familiarity with big data frameworks and tools. - Ability to troubleshoot and optimize data processing workflows. - Excellent collaboration and communication skills. - Bachelor's degree in Computer Science, Engineering, or a related field. ## Preferred Skills - Experience with additional cloud platforms (e.g., AWS, Google Cloud). - Knowledge of data warehousing solutions. - Familiarity with data governance and security practices. ## Experience - Extensive experience in data engineering roles, typically 5+ years preferred. ##