Data Engineer at Brown & Brown in Plano, TX

Job description

Built on meritocracy, our unique company culture rewards self-starters and those who are committed to doing what is best for our customers. Brown & Brown is seeking a Data Engineer to join our growing team in Plano, TX ! As a Data Engineer, you will play an important role in advancing our Data & AI Platform capabilities by designing, building, and operating modern data solutions that support AI/ML, analytics, business intelligence, and enterprise insights. You will work within a collaborative engineering team to develop reliable, scalable, and governed data solutions using modern cloud and lakehouse technologies. You will partner with engineers, product owners, architects, and business stakeholders to understand requirements and translate them into production-ready data pipelines and data products. This is a hands-on engineering role with responsibility across the development lifecycle, including design, development, testing, deployment, monitoring, troubleshooting, and ongoing optimization of data solutions. What Will You Do? Design, develop, test, deploy, and support scalable batch and streaming data solutions . Build Databricks-native data engineering solutions using Delta Lake, Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader, Structured Streaming, and Unity Catalog. Develop production-grade ingestion, transformation, and data-processing pipelines using Python, PySpark, Apache Spark, and SQL . Build reliable and governed data pipelines that improve the accuracy, consistency, accessibility, and trustworthiness of enterprise data. Develop reusable components, libraries, templates, and automation to improve engineering productivity and solution consistency. Implement data-quality validation, monitoring, observability, and remediation processes for critical data assets. Integrate data solutions with governance capabilities, including metadata, lineage, classification, certification, ownership, and access controls . Develop and maintain governed data assets supporting AI, Machine Learning, analytics, semantic models, APIs, Genie Spaces, and AI agents . Build and maintain workflow orchestration, including scheduling, dependencies, parameters, retries, notifications, and operational monitoring. Troubleshoot and optimize data workloads using Spark diagnostics, job logs, query plans, partitioning strategies, join optimization, and Adaptive Query Execution. Support production data solutions through monitoring, incident resolution, performance tuning, migrations, upgrades, and cost optimization. Participate in technical design discussions, code reviews, testing, and continuous improvement of engineering practices. Collaborate with product owners, architects, operations teams, and other engineers to resolve technical dependencies and deliver solutions. Maintain technical documentation for data pipelines, data models, operational procedures, and engineering solutions. Required Qualifications Bachelor’s degree in computer science, Engineering, Mathematics, Information Systems, or a related STEM field, or equivalent education and professional experience. 4+ years of experience in data engineering, software engineering, or a related technical discipline , including experience developing and supporting production data solutions. Hands-on experience with Apache Spark/PySpark, Delta Lake, Python, and SQL . Experience building and supporting production batch and/or streaming data pipelines . Experience with Databricks capabilities such as Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader, Structured Streaming, Delta Lake, and Unity Catalog . Experience developing ETL/ELT pipelines for ingesting, transforming, validating, and publishing enterprise data. Experience with workflow orchestration, including scheduling, dependencies, parameters, retries, notifications, and monitoring. Understanding of Unity Catalog and data governance concepts, including catalogs, schemas, tables, views, lineage, access controls, and permissio