Principal Software Engineer (Washington DC-Baltimore Area)

What you’ll be doing: Architecture and strategy. Define the AI/ML and generative AI architecture and engineering strategy. Design scalable, reliable systems for machine learning, deep learning, LLMs, RAG, multimodal AI and AI agents, from backend architecture and system design through to some frontend capability. Set standards for development, inference, evaluation, observability and governance. Assess emerging AI platforms and make build-versus-buy calls. Create reusable frameworks the wider team can build on. Agentic AI and automation. Architect single-agent and multi-agent solutions that automate real operational work. Design event-based architectures for high-volume document processing pipelines, covering triggering, orchestration, summarisation, structured data extraction and database persistence. Put clear guardrails, authorisation and human-in-the-loop controls around everything. Generative AI and ML engineering. Lead the delivery of production-grade LLM and ML applications, including LLM-driven applications and RAG solutions. Own model selection, prompt engineering and optimisation strategy. Apply rigorous approaches to LLM output evaluation, hallucination reduction, accuracy measurement and LLM-as-judge patterns, and tackle the hard problems: bias, data leakage and prompt injection. Production delivery and MLOps. Define requirements such as throughput, SLAs, traffic patterns and scalability before you build, whether workloads arrive in batches or continuously. Establish MLOps and LLMOps practices, design cloud deployment architectures, and put monitoring, observability, versioning and rollback in place. Technical leadership. Mentor engineers, run design, architecture and code reviews, and define the AI roadmap and proofs of concept.

What you’ll need: 10+ years of software engineering or AI/ML experience. Deep full-stack engineering experience: strong backend architecture, system design, scalability and reliability, with some frontend capability. Meaningful hands-on experience with AI/LLM systems, agentic AI, LLM-driven applications, document processing pipelines and production-grade AI solutions are all highly relevant. Candidates should demonstrate depth through the systems they have designed. The ability to design event-based architectures for high-volume, real-time or batch document processing: triggering, orchestration, summarisation, structured extraction and persistence. Strong problem-solving approach: able to clarify and challenge requirements around throughput, SLAs, traffic patterns and scalability before committing to an architecture. Solid understanding of LLM evaluation: hallucination reduction, accuracy measurement, and patterns such as LLM-as-judge or multi-LLM consensus. Strong hands-on Python, plus experience across machine learning, LLMs, cloud platforms, MLOps and security. Experience with MS SQL Server, including writing queries, designing schemas and tuning performance. A bachelor’s or master’s degree in a related field, such as data science, machine learning or computer science. US citizenship.