Senior Machine Learning Engineer (Active Learning)
Posted Updated
Description
- We are looking for an experienced Senior Machine Learning Engineer with expertise in active learning systems to help us scale our data operations so we can continue to improve our core production models and build the foundation model of electrical devices and energy
- This Senior Machine Learning Engineer will own the loop from data to labels to improved models
- You will use active learning and weak supervision methods to generate labels, decide what to label next, and turn errors we find in the field into new labels
- You will work with data scientists, annotators, product, and engineers to get labels and improve models. You will direct the development of annotation tools and the work of our annotators to speed up and scale labeling
- Success in this role is measured by the reduction in cost per annotation (i.e., time per annotation), while keeping label quality at or above our targets
- Design and run active learning to choose which homes, devices, and time periods to label next, based on where models are uncertain or wrong
- Turn bad detections, field accuracy results, and user corrections into new ground truth, and track whether retrained models improve
- Find rare events and hard cases in home data, such as uncommon or mislabeled devices, and add them to labeling and evaluation sets
- Own the ground truth plan: which data, how much, and in what order
- Direct a team of annotators: set priorities, track throughput, and provide guidance. Set data quality standards
- Direct development of our annotation tool. Add features that enhance our ability to collect new and better data, including features that rely on models
- Measure and improve label quality through metrics, outlier detection, human review, etc
- Work with data engineers and ML operations to version labels and track which labels trained which model. Keep evaluation sets separate from training data so offline results predict performance in the field
- Be a good teammate: review code, provide peer feedback on research and experiments, and contribute to shared documentation. Help mentor/coach less experienced team members
Benefits
- Generous healthcare benefits
- Flexible Spending Account (FSA)
- Life and disability insurance
- Dependent Care Flexible Spending Account (DCA)
- Dental and vision benefits
- Health Savings Account (HSA)
- Health Reimbursement Account (HRA)
- Accident Insurance
- Hospital Indemnity Insurance
- Flexible paid time off
- 401K with company match
- Paid parental leave
- Unconscious bias training
- Hiring practices that promote diversity
- Fitness subsidies
- Paid family and medical leave
- Hybrid and remote work opportunities
- Remote work stipend
- Commuter benefits
- MS or PhD in Computer Science, Electrical Engineering, Physics, Statistics, or a related field, or equivalent experience
- A track record of working with other teams, i.e., product, software, or hardware
- Experience measuring label quality and fixing label errors
- Hands-on experience applying active learning to real data, including choosing a sampling strategy and measuring whether it helped
- Experience training deep learning models, and with weak supervision or model-assisted labeling
- Experience finding rare or hard examples in large datasets, i.e., from model errors, uncertainty, or similarity search
- Experience with Python, SQL, and data pipelines
- 5+ years of experience in data science or machine learning, including 2+ years building or managing labeled datasets for ML
- Experience leading an annotation team or directing development of annotation tools
- Experience using embeddings from large or foundation models for data analysis
- Experience with time series, audio, sensor, or other signal data
Skills
Sign in to read and join the discussion.