Senior Data Engineer

Plano, TX | On-site (Fridays from home after 90 days)

We are working with a client that develops and delivers technology solutions used by businesses across the U.S. With a focus on scalable, data-driven products, the company continues to invest in its technology and engineering capabilities.

They are looking for an experienced

Senior Data Engineer

to join their team in Plano, Texas. You’ll play a key role in developing and optimizing a modern Azure data platform, working with high-volume datasets and building scalable solutions that support analytics and wider business needs.

This is an

on-site position

, requiring you to commute to Plano, TX daily.

Responsibilities

Design and develop scalable ETL/ELT pipelines using

Azure Databricks, PySpark, and Spark SQL
Build data ingestion workflows across sources including

Event Hub, APIs, Azure SQL, and Blob Storage
Develop and maintain

Bronze, Silver, and Gold layers

within a medallion architecture
Implement best practices across

Delta Lake and Unity Catalog

, including schema evolution, versioning, and governance
Optimize large-scale data pipelines for performance and cost through partitioning, caching, clustering, and Spark configuration tuning
Integrate Databricks with

ADLS Gen2, Azure Data Factory, Azure Synapse, Event Hub, Key Vault, and Azure DevOps
Implement secure data access using managed identities, RBAC, and data masking
Work with data analysts, data scientists, business stakeholders, architects, and cloud teams to deliver trusted datasets and KPIs
Provide technical guidance and mentorship to junior engineers, including code reviews and architectural input
Partner with enterprise architecture and information security teams to support data security and privacy requirements
Implement data quality checks, alerts, and lineage tracking using tools such as

Great Expectations, Unity Catalog, and data observability platforms
Build monitoring, alerting, and recovery processes for mission-critical data pipelines

Qualifications

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field
5+ years of data engineering experience
3+ years of hands-on experience with

Azure Databricks and Spark/PySpark
Strong expertise in

Delta Lake, Unity Catalog, Databricks Notebooks, and SQL Warehousing
Hands-on experience with

ADLS Gen2, Event Hub, Azure Data Factory, Key Vault, and Azure Synapse
Strong programming skills in

Python and SQL

, with familiarity with Scala
Solid understanding of

OLTP/OLAP data modelling, star and snowflake schemas, and performance tuning
Experience implementing CI/CD for Databricks using

Databricks CLI, Terraform, or Azure DevOps

Additional experience that would be beneficial includes

Databricks certification, such as Databricks Certified Data Engineer Professional
Real-time streaming technologies including Structured Streaming, Event Hub, or Kafka
MLflow or supporting machine learning pipelines within Databricks
Power BI, Azure Purview, or Microsoft Fabric
Data governance, lineage, and compliance standards including GDPR, HIPAA, and SOC 2

Job description