Lead Data Engineer at Brown & Brown in Plano, TX

Job description

Built on meritocracy, our unique company culture rewards self-starters and those who are committed to doing what is best for our customers. Brown & Brown is seeking a Lead Data Engineer to join our growing team in Plano, TX ! As a Lead Data Engineer, you will play a key role in advancing our Data & AI platform capabilities by designing, building, and operating modern data solutions that power Artificial Intelligence, Machine Learning, analytics, business intelligence, and enterprise insights. This role combines hands-on engineering with technical leadership. You will partner with product owners, architects, business stakeholders, and engineering teams to translate business needs into pragmatic, production-ready data solutions. You will own Data solutions across the engineering lifecycle, from architecture and design through development, deployment, and steady-state operations. You will establish engineering patterns and standards, improve platform capabilities, mentor engineers, resolve complex technical challenges, and help the team deliver reliable, governed, scalable, and cost-effective data products. What Will You Do? Lead the design, development, deployment, and operational support of scalable batch and streaming data solutions . Design and build Databricks-native data engineering solutions using Delta Lake, Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader, Structured Streaming, and Unity Catalog. Develop production-grade ingestion, transformation, and data-processing pipelines using Python, PySpark, Apache Spark, and SQL . Define technical designs, engineering standards, and reusable patterns that improve scalability, reliability, maintainability, and developer productivity. Build and maintain governed data solutions that improve the accuracy, consistency, accessibility, and trustworthiness of enterprise data. Develop reusable frameworks, libraries, templates, and automation that accelerate data ingestion, transformation, feature engineering, and consumption. Implement data-quality validation, monitoring, observability, and remediation capabilities for critical data assets. Integrate data solutions with enterprise governance capabilities, including metadata, lineage, classification, certification, ownership, and access controls . Develop governed data assets supporting AI, Machine Learning, advanced analytics, semantic models, APIs, Genie Spaces, and AI agents . Troubleshoot and optimize production workloads using Spark diagnostics, job logs, query plans, partitioning strategies, join optimization, and Adaptive Query Execution. Operate and continuously improve data solutions through performance monitoring, incident resolution, migrations, upgrades, observability, and cost optimization. Implement automated infrastructure and environment-aware deployments using Infrastructure as Code, Git, and CI/CD practices . Partner with product owners, architects, operations teams, and engineering teams to prioritize capabilities, address technical dependencies, and expand platform adoption. Lead technical design discussions, perform code and design reviews, drive architectural alignment, and proactively identify and remove technical blockers. Mentor engineers and foster a culture of engineering excellence, collaboration, continuous learning, and accountability . Required Qualifications Bachelor’s degree in computer science, Engineering, Mathematics, Information Systems, or a related STEM field, or equivalent education and professional experience. 8+ years of experience in data engineering or related software/data engineering disciplines , including significant hands-on experience building and supporting production data solutions. Strong hands-on experience with Databricks, Apache Spark/PySpark, Delta Lake, Python, and SQL . Experience designing and operating production-grade batch and streaming data pipelines . Experience with Databricks capabilities such as Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader, Str