The vacancy is well-structured and informative, offering clear expectations and compensation details.
Job description
We are hiring a Senior LLM Systems Engineer to own and improve the LLM-driven components of our oracle automation stack. This person will focus on the accuracy, performance, resilience, and operational quality of the systems that use models to reason about wide ranging prediction market rules, evidence, and oracle outcomes.
Responsibilities
### What You'll Own:
- **LLM Accuracy:** improve prompts, model selection, tool usage, structured outputs, retrieval, and evaluation coverage so the system gets more decisions right over time.
- **System Performance:** reduce latency, token usage, and cost while preserving decision quality and operational reliability.
- **Resilience:** design validation, retries, fallbacks, uncertainty handling, and human review paths for ambiguous, adversarial, incomplete, or conflicting inputs.
- **Evaluation and Monitoring:** build datasets, regression tests, dashboards, traces, and review loops that make model quality visible and prevent repeated failures.
- **Agent and Tooling Architecture:** Improve agent orchestration and tool use across internal services, APIs, search workflows, databases, and external data sources.
- **Production Operations:** help debug live issues, investigate regressions, improve runbooks, and reduce repeated operator friction.
Requirements
### Skills & Experience
#### Required
- 3+ years of professional software engineering experience in Python, TypeScript, or similar production languages.
- Hands-on experience building production systems that use LLMs, agents, retrieval, structured outputs, or model-powered workflows.
- Experience designing evaluations, test datasets, regression checks, quality metrics, or manual review loops for AI systems.
- Strong debugging ability across APIs, databases, queues, logs, model outputs, and external data sources.
- Practical understanding of prompt engineering, tool calling, structured output validation, retrieval, and common LLM failure modes.
- Ability to reason carefully about correctness in uncertain or adversarial environments.
- High agency, strong ownership, and clear written communication.
#### Nice to Have
- Experience with oracle systems, prediction markets, DeFi protocols, or other crypto infrastructure.
- Experience with UMA, optimistic oracle mechanisms, Polymarket, or similar systems.
- Experience building agentic systems that use tools, search, browser automation, APIs, or database queries.
- Experience with LLM tracing, model monitoring, evaluation frameworks, or AI observability tools.
- Experience optimizing model cost and latency at scale.
- Experience with Postgres, data pipelines, queue-based systems, background jobs, or event-driven architectures.
- Familiarity with blockchain operational constraints, especially RPC limits, indexing, event logs, finality, and chain-specific behavior.
- Experience with GCP, Cloud Run, GitHub Actions, Terraform, or similar infrastructure.
Conditions
### Compensation and Benefits
- Pay packages include competitive salaries & meaningful long term equity participation.
- Salaries for this role range from $100-200k (USD).
- Will pay in stablecoins or fiat.
- Philosophies for a culture that show we care: Take vacation when you need it, family care, training and development (just to name a few).
- 100% remote, which means we encourage you to create the work environment that you thrive in.
- At least two team wide offsites a year.
About Risk Labs
Risk Labs Foundation is a Web3 infrastructure company building the Across protocol, a secure and capital-efficient bridging solution for DeFi. The team, composed of experienced Web2 financial professionals, enables trustless cross-chain operations and seamless asset transfers across blockchain networks.