Application Architect - DARPA SciFy Project - Penn Engineering

Job Description Summary

We are seeking an Application Architect with expertise in NLP and large language models to lead development on the DARPA SciFy project. The University of Pennsylvania leads a multi-university team building AI systems for automated scientific claim assessment and technology forecasting, in partnership with Two Six Technologies. The architect will design and build the core infrastructure for ClaimSpy, our flagship claim feasibility assessment system, extending it from verifiable materials science claims to speculative AI technology forecasting - supporting DARPA's "create and avoid surprise" mission.

This is a 12-month grant funded position with the possibility of an extension dependent on continued funding.

Job Responsibilities

• Design and build ClaimSpy's agent architecture: multi-LLM pipelines with tool-calling chains for literature retrieval, computational analysis, and claim assessment
• Develop an interactive Claim Card interface for analysts to construct, refine, and explore structured speculative claims, inspect reasoning traces, and review evidence bundles
• Extend the Panel of Virtual Experts (PoVE) system to support Tech Council in a Box (TCBX), enabling multi-perspective deliberation over speculative AI forecasts
• Integrate Two Six Technologies' speculative claims component into the ClaimSpy pipeline, including ablation studies to assess integration quality
• Instrument systems with logging, observability, and error tracing across tool-calling chains to support operational pilot deployments
• Collaborate with researchers on AutoRubric integration for automated LLM-as-judge evaluation of claim assessments
• Conform to DARPA's testing and evaluation framework
• Collaborate with postdocs and graduate students to translate research prototypes into production-ready systems
• Assist with technical documentation, system testing, and monthly DARPA reporting
• Participate in program meetings, technical demonstrations, and pilot exercises with transition partners

Qualifications

• Bachelor's in Computer Science with four to six years of substantial software development experience or an equivalent combination of education and experience.
• Strong Python skills and hands-on experience with LLM APIs (OpenAI, Anthropic, Google) and agent frameworks
• Experience building agentic systems with tool use, multi-step reasoning, and orchestration
• Experience developing APIs that expose LLM capabilities as services (REST/GraphQL endpoints, streaming, authentication, rate limiting)
• Experience with API development, system integration, and full-stack web development
• Ability to rapidly prototype and iterate in an agile research environment
• Strong communication skills for coordinating across a distributed team and with government stakeholders

Preferred

• Experience with RAG systems, vector databases, and retrieval pipelines
• Experience building interactive data exploration or sensemaking interfaces
• Familiarity with scientific literature processing and structured knowledge extraction
• Containerization and deployment (Docker, Kubernetes)
• Open source contributions
• Experience with evaluation frameworks, benchmarking, or LLM-as-judge pipelines
• Prior experience on government-funded research projects

Job Location - City, State

Philadelphia, Pennsylvania

Department / School

School of Engineering and Applied Science

Pay Range

$100,000.00 - $150,000.00 Annual Rate