-
Notifications
You must be signed in to change notification settings - Fork 175
Add GenUI Chat docs #502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add GenUI Chat docs #502
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
94244e8
Add GenUI Chat docs
pinkeshmars 4c5c126
move genui chat to app-events folder
pinkeshmars 4bc2136
move gen ui to agentic-ai
pinkeshmars c6c21f3
Address review comments part 1
pinkeshmars 6e37624
Address review comments part 2
pinkeshmars 0ea7000
address review comment part 3
pinkeshmars 186a0a4
address review comment part 4
pinkeshmars f0a09cb
address review comment part 5
pinkeshmars File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| { | ||
| "label": "Advanced", | ||
| "position": 11 | ||
| "position": 12 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "label": "Agentic AI", | ||
| "position": 11 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "label": "GenUI Chat", | ||
| "position": 1 | ||
| } |
83 changes: 83 additions & 0 deletions
83
docs/ff-concepts/agentic-ai/genui/app-event-integrations.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| --- | ||
| slug: /concepts/app-event-integration | ||
| title: App Event Integration | ||
| description: Feed local app events into GenUI so the conversation can react to live app state and time-sensitive signals. | ||
| tags: [AI, Chat, App Events] | ||
| sidebar_position: 4 | ||
| keywords: [FlutterFlow, App Events, GenUI, Conversational AI, Chat widget, AI agent, A2UI protocol, Component rendering, Tool calling] | ||
| --- | ||
|
|
||
| # App Events Integrations | ||
|
|
||
| App Event Integration lets GenUI listen to FlutterFlow **LOCAL** app events and turn them into conversation context. | ||
|
|
||
| This is how GenUI becomes aware of things the user did not explicitly type: | ||
|
|
||
| - Cart changes | ||
| - Workflow completion | ||
| - Alerts | ||
| - Navigation context | ||
| - Device or sensor updates | ||
|
|
||
| GenUI automatically listens for matching local events and converts them into hidden context messages for the conversation. | ||
|
|
||
| ## Two Integration Modes | ||
|
|
||
| - **Context Injection**: Use `auto_respond: false` when the event should enrich future replies without interrupting the user immediately. In this mode, the event message is added to a pending queue, which is then flushed before the next user message is sent, allowing the model to use these queued messages as hidden context during the next inference. | ||
|
|
||
| - **Proactive Response**: Use `auto_respond: true` when the event should trigger an immediate GenUI response. In this mode, the event message is sent directly into the conversation as an InternalMessage, inference starts right away, and the model may respond with text, UI, both, or nothing visible depending on the prompt and context. | ||
|
|
||
| ## Message Construction | ||
|
|
||
| You can either enter a custom message directly in the **Message Template** field or bind it to a variable for dynamic content. | ||
|
|
||
| If the event includes payload data, GenUI automatically appends it. For example, entering “Your order status is:” and triggering the event which includes event data such as `pending` or `in transit` will result in messages such as “Your order status is pending.” | ||
|
|
||
|
|
||
| ## Pending Context Queue | ||
| For `auto_respond: false`, GenUI stores pending event messages in memory until the user sends the next message. The queue has a maximum size of 50, and if it overflows, the oldest messages are dropped first. Before the next user request is sent, these messages are injected directly into the conversation history as InternalMessages, allowing the model to use them as context without triggering additional model calls. | ||
|
|
||
| ## Best Practices | ||
|
|
||
| #### Use context injection for ambient state | ||
|
|
||
| For example: | ||
|
|
||
| - Updated cart contents | ||
| - Current page context | ||
| - Background sync results | ||
|
|
||
| These make future replies smarter without causing unsolicited responses. | ||
|
|
||
| #### Use proactive response for time-sensitive events | ||
|
|
||
| For example: | ||
|
|
||
| - Threshold alerts | ||
| - Task completion | ||
| - Failed jobs | ||
| - Incoming high-priority updates | ||
|
|
||
| These are the moments where an immediate assistant response is justified. | ||
|
|
||
| #### Keep event data structured | ||
|
|
||
| If an event carries payload data, use a stable, well-designed data type. The generated message ends up calling `toMap()`, so clearer payload structure produces clearer AI context. | ||
|
|
||
| #### Do not flood the queue | ||
|
|
||
| If a background signal can fire rapidly, consider batching it before triggering the event. The queue has a hard cap of 50 messages. | ||
|
|
||
| ## Examples | ||
|
|
||
| #### Cart awareness without interruption | ||
|
|
||
| Use a `CartUpdated` local event with `auto_respond: false`. | ||
|
|
||
| Each cart update quietly enriches the pending context so the next time the user asks, "What's in my cart?" the model already has the latest state. | ||
|
|
||
| #### Immediate alerting | ||
|
|
||
| Use a `TemperatureAlert` local event with `auto_respond: true`. | ||
|
|
||
| When the event fires, GenUI immediately triggers inference and the model can warn the user and render a supporting UI component if the catalog contains one. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| --- | ||
| slug: /concepts/component-catalog | ||
| title: Component Catalog | ||
| description: Configure the FlutterFlow components that GenUI is allowed to render inside the chat surface. | ||
| tags: [AI, Chat, Components, Widgets] | ||
| sidebar_position: 2 | ||
| keywords: [FlutterFlow, Components, GenUI, Conversational AI, Chat widget, AI agent, A2UI protocol, Component rendering, Tool calling] | ||
| --- | ||
|
|
||
| # Component Catalog | ||
|
|
||
| The **Component Catalog** is the list of FlutterFlow components that GenUI can render inline in the conversation. Without a catalog, GenUI can still chat and call tools, but it has no specific UI to render. | ||
|
|
||
| Internally, GenUI creates documentation for each catalog component. That documentation includes: | ||
|
|
||
| - Component name | ||
| - Component description | ||
| - Parameter names | ||
| - Parameter types | ||
| - Required or optional status | ||
| - Parameter descriptions | ||
|
|
||
| The model's render decisions are only as good as the naming and descriptions you provide. | ||
|
|
||
| ## Component Requirements | ||
|
|
||
| #### The component must be serializable at the API boundary | ||
|
|
||
| Catalog components cannot expose **action parameters**. GenUI only knows how to pass structured data into the component, not callbacks or arbitrary closures. | ||
|
|
||
| #### Parameters should use supported types | ||
|
|
||
| Supported parameter categories in the generated catalog pipeline include: | ||
|
|
||
| - `String` | ||
| - `int` | ||
| - `double` | ||
| - `bool` | ||
| - `Color` | ||
| - `DateTime` | ||
| - `TimestampRange` | ||
| - `LatLng` | ||
| - `GooglePlace` | ||
| - `JSON` | ||
| - `DataStruct` | ||
| - `Enum` | ||
| - media-path string types such as `ImagePath`, `VideoPath`, `AudioPath`, and `MediaPath` | ||
| - `List<T>` of supported item types | ||
|
|
||
| #### Required complex parameters need explicit defaults | ||
|
|
||
| If a catalog parameter is non-nullable and uses one of these complex types: | ||
|
|
||
| - `Color` | ||
| - `DateTime` | ||
| - `TimestampRange` | ||
| - `LatLng` | ||
| - `GooglePlace` | ||
| - `DataStruct` | ||
| - `JSON` | ||
|
|
||
| then you should either: | ||
|
|
||
| - set an explicit default value, or | ||
| - make the parameter optional | ||
|
|
||
| For instance, if your **EventCard** component has a required `eventDate: DateTime` parameter, you must either set a default value in the component editor or make the parameter optional. Without this, GenUI validation will reject the component. | ||
|
|
||
| GenUI validation enforces this because those types do not have a safe implicit fallback in generated constructor code. | ||
|
|
||
| ## Runtime Rules | ||
|
|
||
| #### One root component per surface | ||
|
|
||
| Each GenUI surface renders exactly one catalog component as its root. That root component can be a rich component tree internally, but the model cannot compose arbitrary parent wrappers like `Column`, `Container`, or other widgets that are not in the catalog. | ||
|
|
||
| #### The model can only use listed catalog components | ||
|
|
||
| If a component is not in the catalog, it does not exist from the model's perspective. | ||
|
|
||
| ## Best Practices | ||
|
|
||
| #### Use list-friendly components | ||
|
|
||
| Because a surface has one root component, a component that accepts `List<T>` is often the right shape for result sets: | ||
|
|
||
| - `TransactionList` | ||
| - `SearchResultsGrid` | ||
| - `CartItemsSummary` | ||
|
|
||
| #### Prefer focused components over screen-sized composites | ||
|
|
||
| Good catalog components are reusable units, such as: | ||
|
|
||
| - `ProductCard` | ||
| - `OrderSummary` | ||
| - `InvoicePreview` | ||
| - `ReviewSummary` | ||
| - `AppointmentConfirmation` | ||
|
|
||
| These give the model flexible building blocks. A large page-like component is harder to reuse and usually harder for the model to choose well. | ||
|
|
||
| #### Use consistent `DataStruct` across tools and components | ||
|
|
||
| If a tool returns `ProductStruct`, prefer catalog components that also accept `ProductStruct` or `List<ProductStruct>`. That keeps tool output and rendering input aligned and makes the tool-to-UI handoff more reliable. | ||
|
|
||
| #### Describe parameters like you are documenting an API | ||
|
|
||
| Good: | ||
|
|
||
| - `estimatedDeliveryDate`: "Expected arrival date in ISO 8601 format." | ||
| - `inventoryStatus`: "Availability state shown to the user, such as inStock or backOrdered." | ||
|
|
||
| Weak: | ||
|
|
||
| - `date` | ||
| - `status` | ||
|
|
||
| #### Keep component names specific | ||
|
|
||
| Use clear, descriptive names that reflect the component’s purpose. | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Typo:
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done! |
||
| Good: | ||
|
|
||
| - `OrderStatusCard` | ||
| - `SensorAlertSummary` | ||
| - `QuoteBreakdown` | ||
|
|
||
| Weak: | ||
|
|
||
| - `Card1` | ||
| - `Summary` | ||
| - `Details` | ||
|
|
||
| #### Avoid ambiguous overlap | ||
|
|
||
| If two components do roughly the same thing, the model has to guess. Either merge them, rename them more clearly, or narrow their intended use. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: This section would benefit from a concrete example to make the requirement tangible:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done!