Case study
Cohort: An AI Agent Framework for Unity (C#)
I built a reusable Unity agent framework with typed requests and responses, configurable agent assets, conversation state, and a sample model-provider adapter.
Creator · C# framework design, Unity development, and AI integration
I built Cohort to give Unity developers a reusable foundation for conversational AI features. The package separates the agent's configuration, conversation state, and request lifecycle from the code that communicates with a model provider.
- Ownership: Runtime interfaces, generic agent implementation, configuration assets, and the integration sample.
- Technical scope: Typed asynchronous requests and responses, in-memory conversation history, and model/endpoint abstractions.
- Delivered artifact: A public Unity package with prefabs, ScriptableObject assets, and a sample scene demonstrating a single-agent request/response flow.
The Challenge
A conversational feature inside a Unity scene needs more than a model call. It also needs an agent identity, instructions, conversation history, and a way for scene code to consume the response. Putting those responsibilities directly into individual scene scripts makes the integration harder to reuse.
I structured Cohort around a common agent contract so those concerns could be assembled separately. The resulting package combines familiar Unity components and assets with typed C# interfaces for the request path.
Design goal: Let scene code work with an agent while configuration, conversation state, and provider-specific request handling remain separate responsibilities.
From Configuration to a Response
A ConversableAgentDefinition ScriptableObject holds the agent prefab, identity, and system prompt. The sample factory instantiates the prefab and initializes it with a definition and model factory. This keeps configuration in Unity assets instead of embedding every agent's instructions in scene logic.
BasicConversableAgent specializes the generic agent for text requests and responses. Its asynchronous Execute method adds a request to the conversation, awaits a completion through ILanguageModel, records the reply, and returns a typed response to the caller.
Simplified architecture of the runtime and included sample. The concrete provider adapter is implemented in the sample.
| Architecture decision | Rationale |
|---|---|
Generic request and response content constrained by IContent | Keep the agent contract explicit while allowing different content types. |
| ScriptableObject definitions and prompt assets | Make identity, prefab, and instructions configurable in the Unity editor. |
Separate ILanguageModel and ICompletionEndpoint interfaces | Keep provider-specific message mapping outside the core agent implementation. |
| Maintain conversation fragments on the agent | Give each agent a clear owner for its current conversation state. |
Conversation State and Adapter Boundaries
The agent's Cortex list stores role-tagged, timestamped conversation fragments in memory. Requests and replies are added to that history, and Reboot clears conversation turns while retaining the system instruction. Disabling RememberConversation resets the history after a response.
The included OpenAI adapter maps Cohort's content and roles into provider messages. It handles text and texture-backed image inputs, and contains callbacks for partial, complete, and structured responses. These callbacks belong to the endpoint layer; the sample's scene code consumes the awaited response.
The architecture exposes an extension boundary for other providers. The public sample supplies one concrete provider implementation.
A Small Scene-Facing API
With an initialized agent, scene code can issue a typed request and await its result:
using Cohort.Agents.Conversable; using System.Threading.Tasks; async Task<string> Ask(BasicConversableAgent agent, string prompt) { var response = await agent.Execute( new ConversableRequest<Text>( IRole.Type.User, new Text { Message = prompt })); return response.Content.Message; }
What the Project Demonstrates
Cohort is a reusable framework and integration sample: an example of turning a model interaction into a configurable Unity development pattern. The engineering work connects interface design, asynchronous execution, state management, and the editor workflow around an agent.