Sr. Inference Service SRE & Automation Engineer at Bitdeer Technologies Group in Singapore; Penang

Job description

About Bitdeer: Bitdeer is a world-leading technology company for Bitcoin mining and AI cloud. Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry-leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This includes equipment procurement, transport logistics, datacenter design and construction, equipment management, and network and facility operations. Bitdeer also offers advanced cloud capabilities to customers with a high demand for artificial intelligence. Headquartered in Singapore, Bitdeer operates globally with a diversified 3 GW energy portfolio, and deploys Bitcoin mining and HPC datacenters in the United States, Bhutan, Norway, Canada, Malaysia, and Ethiopia. About the role NeoCloud runs Infermesh, our managed Inference / Model-as-a-Service platform: customers call model endpoints over a public API and pay per token. For a service like this, reliability is the product — a failed request is not an internal incident, it is a customer-visible SLA breach. In this role you own the reliability of the inference serving path end to end: public API gateway, inference router, request scheduler and queue, prefill/decode and KV cache, inference engines, and the GPU model replicas behind them — plus the control plane and shared dependencies they rely on. Your mandate is to aim a 99.95% SLA, by closing the architecture, automation, and operational gaps that stand in the way. You are the SRE who stands between a fleet of H200/B300/GB300/VR NVL72 serving nodes and the customer's experience of a simple, dependable model endpoint. What you will be responsible for: End-to-end availability of customer inference endpoints — API availability, request success rate, TTFT/TPOT latency, and the published SLA. The serving path: API gateway, inference router, health-based routing, request scheduler, queueing and admission control, prefill/decode serving, and the KV cache service. Inference runtime operations: engine health (vLLM / SGLang / TensorRT-LLM class runtimes), model load and readiness validation, model revision rollout and rollback. Model replica topology and failure isolation: tensor-parallel group containment, GPU / NVLink / NCCL fault isolation, replica anti-affinity across nodes, racks, and failure domains. Multi-cluster serving across MY / SG / JP: cross-cluster traffic steering, tested cluster failover, and regional recovery or evacuation scope. GPU serving fleet health (H200 / B300 / GB300 / VR NVL72 bare metal): DCGM / Xid / ECC monitoring, automated drain and cordon, replica replacement with no sustained customer impact. The Kubernetes platform beneath the serving path: HA control plane, etcd, CNI, CoreDNS, public / management / east-west edge tiers, and egress NAT. Shared control-plane dependencies: PostgreSQL (including Temporal), Redis, Vault secrets and certificates, object storage and model registry, ClickHouse — HA, tested failover, and decoupling live serving from control-plane outages. Capacity and saturation engineering: GPU utilization, KV cache occupancy, queue depth and concurrency; N+1 headroom, admission control, load shedding, and graceful degradation instead of collapse. The SLI/SLO/SLA system: define and instrument the SLIs, publish the SLOs, run error budgets and burn-rate alerting, and produce customer-facing SLA reporting. Observability stack — VictoriaMetrics, VictoriaLogs, Tempo, Pyroscope, Grafana, ClickHouse — plus external synthetic inference probes that exercise the real customer API path. Incident management: severity model, incident commander rotation, primary / secondary / cross-team / vendor escalation, status-page and customer communication, and postmortems with tracked actions. Change safety: staged and canary rollout for application, model, and configuration changes; automatic rollback on regression; production change review and release risk classifica