(USA) Senior, Data Engineer at Walmart

Job description

Position Summary... What you'll do... Role summary: As a Senior Data Engineer, you will design, develop, and maintain scalable data pipelines and architectures that support critical business insights and decision-making. This role requires expertise in data integration, modeling, and governance within cloud and big data environments. You will collaborate across functions to translate complex business requirements into effective data solutions, ensuring data quality and accessibility. Leveraging advanced programming and analytical skills, you will drive innovation in data processing and support the implementation of data strategies that enhance operational efficiency and deliver measurable value to the organization. The Senior Data Engineer will be responsible for not only understanding data pipelines but, event streaming applications, and how to build systems that handle massive amounts of data while making it consumable by other application teams, users and data scientists. You will also be assisting in the design and architecture of highly scalable, fault tolerant infrastructure capable of processing millions of operations per minute coming from millions of TVs, efficiently store petabytes of data and provide fast insights from the data. You will also be working with teams across the Walmart/VIZIO enterprise to bring their data into our Big Data ecosystem, monitor data quality for cleanliness and fix discrepancies. Ensure data accuracy through validation tasks, perform solutions for root cause analysis and implement data prep and cleanliness, review data at all granular/aggregate levels, and versioning. About the team: The Data Lake Platform team at Walmart/VIZIO drives enterprise analytics by delivering reliable datasets that support key business decisions and new monetization opportunities. Operating at the intersection of Data and AI, the team emphasizes innovation to address complex challenges. Members collaborate to design and maintain scalable data architectures, pipelines, and governance frameworks, ensuring data quality and accessibility. This group values a methodical approach to problem-solving and continuous improvement, leveraging advanced data engineering and machine learning techniques to empower the organization with actionable insights and strategic data solutions. What you'll do: Design, develop, and maintain scalable data pipelines and architectures to support business analytics and machine learning initiatives. Extract, transform, and integrate data from various internal and external sources ensuring data quality and relevance. Collaborate with stakeholders to translate business requirements into effective data solutions aligned with enterprise strategy. Develop and optimize data models, including logical and physical designs, to enhance data accessibility and usability. Implement data governance practices to ensure compliance with regulatory and organizational standards. Utilize programming languages and tools such as SQL, Python, Spark and Apache Airflow for coding, testing, and deployment. Monitor and improve data processes to support continuous innovation and operational excellence. What you'll bring: Extensive experience in data engineering, including data integration, modeling, and pipeline development including designing and implementing ETL processes. High Proficiency in Pyspark, SparkSQL and/or Scala. Proficiency in SQL and programming languages such as Python or Java for code development and testing. Extensive knowledge of AWS/GCP Cloud Services. 5+ years of experience within Data Engineering and prior experience working with Big Data, Data Lakes and Delta Lakes. Strong knowledge of data governance, data quality standards, and regulatory compliance. Expertise in cloud and big data platforms, data lakes, and data architecture. Ability to translate business requirements into scalable data solutions aligned with strategic objectives. Prior experience with Infrastructure as code tools like Terraform