This website is for sale

Member of Technical Staff (Site Reliability Engineer, Crawler)

Perplexity London

Posted 4 days ago · via Ashby

Apply at Perplexity

About this role

Keeping a crawler running is not the same as keeping it productive. Queues can grow while servers look healthy, and a small change can become a fleet-wide bottleneck. On Perplexity's Crawler team, you'll improve throughput, plan capacity, and build operational reliability through safer deployments, better observability, and durable fixes.

Responsibilities Operate and optimize Kubernetes workloads and supporting infrastructure for crawler services, including fetching and rendering fleets.

Build and maintain infrastructure as code and CI/CD pipelines, with safe rollouts, validation, and fast rollback.

Define SLOs and build monitoring, alerting, and observability for availability, latency, throughput, and resource saturation.

Own capacity planning and resource efficiency; diagnose queue backlogs, network failures, and resource bottlenecks with service owners.

Participate in the team's on-call rotation, lead incident response, and turn blameless postmortems into durable reliability improvements.

Write production software to automate diagnosis and routine remediation, with validation and approval gates for high-impact actions.

Qualifications 5+ years of experience in SRE, production engineering, or infrastructure-heavy backend roles, with strong programming skills in Python, C++, Rust, or Go.

Hands-on experience with cloud infrastructure, production Kubernetes, infrastructure as code such as Terraform, and automated deployment workflows.

Strong Linux and networking fundamentals, including TCP/IP, DNS, TLS, HTTP, and load balancing.

Experience building observability with tools such as Datadog, Prometheus, or Grafana, and using SLOs to guide reliability improvements.

Proven experience diagnosing distributed-system failures, leading production incidents, and improving system performance or resource efficiency.

Nice to have Experience operating web crawlers, search infrastructure, or browser fleets at scale.

Experience with distributed queues, backpressure, workload isolation, or multi-region recovery.

This listing is collected from the employer's public hiring board via Ashby and may be abbreviated. See the full posting and apply on the employer's site.

Apply at Perplexity

Related jobs

New Research Engineer, AI Monitoring

anthropic London via Arbeitnow

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working to…

Posted Today

New Senior Engineer II - EngEx AI Engineer

Kaluza London, UK via Arbeitnow

Senior Engineer II - EngEx & AI Engineer ● Location: UK Hybrid ● Level: TL1 ● Team: Engineering Experience & AI Foundations This role is based in the UK and requires an existing right to work in the UK. At this time, we are not able to offer visa sponsorship for this role. We are committed to buildi…

Posted Today

New Senior Engineer I - EngEx AI Engineer

Kaluza London, UK via Arbeitnow

Senior Engineer I - EngEx AI Engineer ● Location: UK Hybrid (London, Bristol, Edinburgh) ● Level: C3 ● Team: Engineering Experience & AI Foundations This role is based in the UK and requires an existing right to work in the UK. At this time, we are not able to offer visa sponsorship for this role. W…

Posted Today

New AI Engineer

Apollo London via Arbeitnow

Who we are Apollo, a Skyward Group company, is an innovation-inspired insurance platform operating at Lloyd’s of London, delivering data-driven and creative solutions across a broad range of risks. Our name reflects our origins. Inspired by one of our founding investors, Neil Armstrong, the first…

Posted Today

New Retention and Growth Project & AI Coordinator (f/m/d)

Awin London via Arbeitnow

Purpose of Position Our Retention & Growth Team is responsible for two keys areas; Customer Retention – ensuring long-term continuity and sustainability of our customer base.  Portfolio Growth – expanding revenue in line with Gross Profit (GP) and Annual Recurring Revenue (ARR) targets. This tea…

Posted Today