AI Lead - Platform Intelligence & Applied AI
AI Lead - Platform Intelligence & Applied AI
L7
Occupations
Computer and Information Systems ManagersComputer and Information Research ScientistsComputer Systems Engineers/Architects- Continuously track and evaluate: LLM and foundation model advancements Agent frameworks and orchestration patterns Retrieval, memory, and context management techniquesAI evaluation, safety, and governance approaches
- Translate emerging AI trends into: Platform design principles Proofs of concept and experiments Scalable, production-ready capabilities
- Advise leadership on when and how new AI capabilities should be adopted. Model & Intelligence Management
- Own the strategy for LLM and model usage across the platform, including: Model selection and benchmarking Versioning and lifecycle management Cost, performance, and latency trade-offs Fallback and redundancy strategies
- Establish best practices for: Prompt and instruction design Skill and Tool calling Structured outputs and determinism Semantic Routing & Orchestration
- Design and evolve the platform's semantic routing layer, including: Intent detection and task classification Routing to appropriate models, agents, or workflows Context-aware decisioning based on workspace state
- Define orchestration patterns for: Multi-step and parallel execution Long-running and asynchronous tasks Human-in-the-loop controls
- Ensure routing logic is transparent, testable, and tunable. Agent Architecture & Execution
- Consult on the firm's agent strategy, including: When to use agents vs. workflows vs. direct LLM calls Agent composition, memory, and tool access Guardrails and behavioral constraints
- Partner with engineering to implement: Agent frameworks and runtime infrastructure Monitoring and debugging capabilities
- Ensure agents are: Predictable and auditable Aligned to service methods and delivery workflows Safe for enterprise and client-facing use Workspace Context & RAG Architecture
- Own the design of contextual intelligence within workspaces, including: Document ingestion, chunking, and enrichment strategies Vector, keyword, and hybrid retrieval approaches Context assembly across client data, firm IP, and engagement artifacts
- Define standards for: Source attribution and transparency Data isolation and compliance Relevance, freshness, and performance
- Continuously evaluate new approaches to memory, retrieval, and grounding. AI Evaluation, Testing & Trust
- Establish the platform's AI evaluation and testing framework, including: Task-based and scenario-driven evaluations Regression testing for prompts, agents, and routing logic Comparative benchmarking across models and configurations
- Define metrics for: Accuracy, relevance, and consistency Cost efficiency and latency User trust and explainability
- Partner with engineering and risk teams to ensure: Observability into AI behavior Safe deployment and controlled experimentation Continuous improvement loops based on real usage Platform Enablement & Collaboration
- Work closely with: Platform engineering teams Product and design partners Consulting and delivery leaders
- Provide technical guidance on: How AI capabilities should be embedded into platform features Where AI adds leverage vs. complexity
- Support enablement through: Technical documentation and reference architectures Internal education and design reviews Advisory support for high-impact use cases Governance & Responsible AI
- Define technical guardrails that support: Security, privacy, and data residency Responsible AI principles Regulatory and client requirements
- Ensure AI systems are: Explainable where required Observable and auditable Designed for controlled evolution over time
- The platform consistently adopts relevant AI innovations without destabilizing delivery.
- AI behavior is predictable, testable, and trusted by consultants and leadership.
- New models, agents, and techniques can be introduced rapidly through well defined abstractions.
- AI capabilities directly improve delivery quality, speed, and consistency across.
Level
ManagerL7
Location
Chicago, IL
Occupation
Computer and Information Systems Managers
Industry
Computer Systems Design Services
Posted
4 days ago
To get sharper similar jobs, create your profile using the link below.