Lead Software Engineer - Data

# Data Engineer ## Company Overview *Company details are not provided, please include if available.* ## Job Summary As a member of the Data team, the Data Engineer will tackle cutting-edge data engineering and distributed computing challenges. This role involves improving throughput, reducing latency, and maintaining uptime of data pipelines and web services. The engineer will focus on writing test-driven code to process terabytes of data in multi-region distributed systems. ## Responsibilities - Scale the current data pipeline to handle 5X the present capacity within one year. - Transition from batch-oriented processing to a near-real-time processing engine. - Build performance monitoring systems for databases, web services, and processing engines. - Think big and execute with a milestone-based approach rather than a big-bang strategy. - Design and code with considerations for scale, high availability, and cost efficiency. - Mentor and review the code of fellow team members. - Lead a micro team within the Data team and implement effective tech processes and tools. - Own problem statements and devise solutions to address them. - Be open to working with a polyglot tech stack. ## Qualifications - 5-7 years of proven experience in developing scalable REST/gRPC services or streaming pipelines and data-intensive applications. - Expertise in Java programming language and frameworks. - Hands-on experience with data modeling, database design, and performance optimization. - Familiarity with web frameworks such as Vert.X, Spring Boot, and Quarkus is a plus. - Experience with data processing technologies such as Kafka, Flink, and Spark is a plus. - Knowledge of containerization tools such as Docker and Kubernetes is a plus. ## Preferred Skills - Understanding of cloud services and architectures. - Experience in automated testing frameworks and CI/CD practices. - Knowledge of big data technologies and frameworks. ## Experience - 5-7 years relevant experience in data engineering or related fields. ## Environment *Typical work setting and location details are not provided, please include if available.* ## Salary *Salary details are not provided, please include if available.* ## Growth Opportunities *Opportunities for career advancement are not specified, please include if available.* ## Benefits *Benefits details are not provided, please include if available.*