Data Engineer

Data Engineer

Data Strategy

Full-time, Exempt Regular, Pay Grade 4.2

Location: Hybrid

A skilled, hands-on Data Engineer responsible for designing, building, deploying, and operating enterprise data ingestion, transformation, and delivery pipelines within a Databricks Lakehouse environment. The role focuses on data integration and interoperability, enabling secure and reliable access to curated datasets for analytics, applications, and machine learning use cases.

Duties and Responsibilities:

• Design, build, test, and operate scalable batch and streaming data ingestion and transformation pipelines using Databricks, Apache Spark, Delta Lake, Python, and SQL.
• Implement data integration workflows that ingest and process data from diverse internal and external sources, including databases, files, REST APIs, and streaming platforms, in alignment with established architectural standards.
• Develop, maintain, and document curated datasets and data products that support analytics, reporting, and downstream application and machine learning use cases.
• Package, version, and deploy data pipelines using Databricks Asset Bundles, adhering to established CI/CD processes and environment promotion practices (development, test, production).
• Apply data quality validation, monitoring, and observability controls to ensure data accuracy, pipeline reliability, and compliance with defined service-level objectives.
• Implement and maintain access controls, authentication, and authorization using Unity Catalog in accordance with enterprise data governance and security requirements.
• Monitor pipeline performance and resource utilization, perform routine tuning, and support incident resolution and root-cause analysis for production data workflows.
• Collaborate with platform, analytics, and governance teams to clarify data requirements, resolve integration issues, and support data consumers.

Competencies:

• Technical execution and problem solving
• Attention to detail and data quality
• Collaboration within cross-functional teams
• Accountability for operational stability

Skills:

• Databricks, Apache Spark, Delta Lake
• SQL and Python
• Batch and streaming ingestion patterns
• Federated data access and APIs
• Unity Catalog and governance

Education & Experience Requirements

Education:
Bachelor's degree in Computer Science, Information Systems, or a related field.

Experience:
5+ years of hands-on data engineering experience supporting production data pipelines.

Preferred Experience Requirements

Education:
Master's degree preferred.

Experience:
Large-scale governed data environments.

Certifications:
Databricks certifications preferred.

All submissions should include a cover letter and resume.

Hiring Range:

$112,000.00 - $127,000.00

Apply Now