Software Engineer AI Gateway

You will join the engineering team that builds the Kong AI gateway, empowering companies and developers around the world to govern every step of their AI & LLM based innovation. As a Software Engineer in the AI Gateway Team, you will be working on core features such as multi-LLM routing and balancing, semantic caching and routing, MCP, and more.

Responsibilities

Responsible for building and maintaining API management software for AI and LLMs and its underlying stack; Implement new innovative features including proxying capabilities, load balancing, databases support that extend Kong AI Gateway’s offering; Bugs troubleshooting, performance tuning and low-level optimizations; Fill the gaps between Kong and third-party deployment platforms and tooling; Any additional tasks required by manager

Requirements

3+ years of programming in server-side applications and components; Hands on experience with either Golang or Rust; Experience building and debugging applications that are highly scalable, distributed, reliable and resilient; Solid hands-on low-level technical expertise in Linux systems troubleshooting, and profiling; Experience in distributed systems, Linux networking stack and computer systems programming, Kong works with APIs on both L7 and L4 transports and protocols, so this expertise and knowledge are required; Comfortable working with a remotely distributed team; Strong sense of ownership and excellent verbal and written communication skills -- engineers at Kong write the documentation for their products; Familiarity with AI LLM concepts and applications development

Nice to have

Hands on experience with NGINX/OpenResty/Kong/LuaJIT; Hands on experience with Envoy or Istio