Sr. Data Engineer/Architect (Databricks, DBT, Snowflake, PySpark, Unity Catalog) - 100% Remote - AS
Posted
C
cbsinfosys.catsone.com Sr. Data Engineer/Architect (Databricks, DBT, Snowflake, PySpark, Unity Catalog) - 100% Remote - AS
Description
Responsibilities
1. Ingest and process data from Salesforce (Investments), Dynamics 365 & Coupa (Finance), Workday (HR), Concur (Operations), and external program/partner data (e.g., Health, Education).
2. Build and maintain pipelines in Databricks, DBT, and Snowflake to deliver curated datasets for AI/ML.
3. Design simple, reusable data models to support training and inference.
4. Implement data quality tests, documentation, and lineage in DBT
5. Ensure performance optimization, cost efficiency, and compliance with governance standards.
6. Collaborate with Knowledge Management (AI/Data Science) team to provide feature ready datasets.
Requirements
1. Hands-on experience with Databricks (PySpark/SQL, Unity Catalog), DBT (models, tests, docs), and Snowflake (SQL, performance tuning), Delta Live Tables.
2. Strong background in ETL/ELT pipeline development.
3. Experience with enterprise SaaS systems (Salesforce, D365, Coupa, Workday, Concur).
4. Solid SQL and Python skills; knowledge of data quality frameworks.
1. Ingest and process data from Salesforce (Investments), Dynamics 365 & Coupa (Finance), Workday (HR), Concur (Operations), and external program/partner data (e.g., Health, Education).
2. Build and maintain pipelines in Databricks, DBT, and Snowflake to deliver curated datasets for AI/ML.
3. Design simple, reusable data models to support training and inference.
4. Implement data quality tests, documentation, and lineage in DBT
5. Ensure performance optimization, cost efficiency, and compliance with governance standards.
6. Collaborate with Knowledge Management (AI/Data Science) team to provide feature ready datasets.
Requirements
1. Hands-on experience with Databricks (PySpark/SQL, Unity Catalog), DBT (models, tests, docs), and Snowflake (SQL, performance tuning), Delta Live Tables.
2. Strong background in ETL/ELT pipeline development.
3. Experience with enterprise SaaS systems (Salesforce, D365, Coupa, Workday, Concur).
4. Solid SQL and Python skills; knowledge of data quality frameworks.