Staff Software Engineer, AI Platform & Infrastructure at Okta in San Francisco, CA

Job description

Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building the trusted, neutral infrastructure that enables organizations to safely embrace this new era. This work requires a relentless drive to solve complex challenges with real-world stakes. We are looking for builders and owners who operate with speed and urgency and execute with excellence. This is an opportunity to do career-defining work. We're all in on this mission. If you are too, let's talk. The Software Engineer Opportunity The team that built Jeeva AI is building Digital Workers, a new suite of autonomous agents for identity operations and security inside Okta. The team works directly with Todd McKinnon, Okta's co-founder and CEO, with the opportunity to bring this product to Okta's 20,000+ customers. You will help shape the product with the ownership and pace of a startup and the resources of a public company. You will build the backend and AWS platform that keep these workers running around the clock. The team owns services, connectors, workflow execution, data and production reliability. You will write Python services and infrastructure code that let workers make approved changes and preserve evidence. About You Reliable builder: You design for failure and prove that recovery works. Hands-on engineer: You write production services and improve the tools engineers use every day. Strong owner: You follow changes through deployment, monitoring and production support. What you will be doing Build Python FastAPI services, data models and reusable connectors for Okta, AWS, ServiceNow and enterprise applications. Implement durable execution with Step Functions, SQS and EventBridge. Handle approval waits, rate limits, duplicate events and safe retries. Manage PostgreSQL schemas, indexes, query performance and pgvector retrieval. Enforce tenant isolation across APIs, storage and credentials. Build model API access for OpenAI, Anthropic and supported AWS paths, including credentials, quotas, timeouts and cost controls. Add an analytics pipeline for worker events, traces, tokens, findings and verified outcomes. Evaluate ClickHouse and comparable systems such as BigQuery. Create reliable usage records and reconciliation. Prevent retries, duplicate findings and incomplete work from being counted as new outcomes. Build deployment pipelines, infrastructure as code, observability and safe release controls across development, staging and production. Implement scoped write services, kill switches, audit retention and backup recovery. Test failures and verify that sensitive changes actually took effect. What you will bring to the role Experience building and operating production backend or distributed systems, with strong coding skills in Python or a comparable language. Hands-on AWS experience with APIs, queues, containers, networking, IAM and databases. Strong SQL and data modeling skills. Experience diagnosing database performance and asynchronous system failures. Experience with infrastructure as code, deployment automation, observability and production incident response. Sound judgment about customer data, tenant isolation, credentials and safe changes to external systems. Scope by level Senior: Own a backend or platform component, including deployment, tests and production operation. Staff: Lead a domain such as execution, tenant security, data infrastructure or reliability across teams. Principal: Set shared backend and platform contracts, resolve system tradeoffs and implement foundational services. Our stack Python FastAPI, AWS Step Functions, Amazon Bedrock AgentCore, Lambda, SQS, EventBridge, API Gateway and containers. Data and security use RDS PostgreSQL, pgvector, S3 Object Lock, KMS, Secrets Manager, CloudWatch and CloudTrail. The frontend uses React and AWS Amplify. OpenAI and Anthropic model integrations are planned. We are also adding an agent harness and dedicated analytics. ClickHouse is a hiring foc