Add mTLS app-to-app routing support (RFC draft)#4910
Draft
Conversation
- Add app_to_app_mtls_routing feature flag (default: false) - Add allowed_sources to RouteOptionsMessage with validation - Validate allowed_sources structure (apps/spaces/orgs arrays, any boolean) - Validate that app/space/org GUIDs exist in database - Enforce mutual exclusivity of 'any' with apps/spaces/orgs lists
Tests cover: - Feature flag disabled: allowed_sources rejected as unknown field - Structure validation: object type, valid keys, array types, boolean any - any exclusivity: cannot combine any:true with apps/spaces/orgs lists - GUID existence validation: apps, spaces, orgs must exist in database - Combined options: allowed_sources works with loadbalancing
Rails parses JSON with symbol keys, but validation was comparing against string keys. Add normalized_allowed_sources helper to transform keys to strings for consistent comparison.
Rename the route options field from allowed_sources to mtls_allowed_sources for better clarity about its purpose in mTLS app-to-app routing. Updates RouteOptionsMessage to use the new field name in: - Allowed keys registration - Feature flag gating - Validation methods - All related tests
Change from nested mtls_allowed_sources object to flat options: - mtls_allowed_apps: comma-separated app GUIDs (string) - mtls_allowed_spaces: comma-separated space GUIDs (string) - mtls_allowed_orgs: comma-separated org GUIDs (string) - mtls_allow_any: boolean (true/false) This complies with RFC-0027 which requires route options to only use numbers, strings, and boolean values (no nested objects or arrays).
Replace POC route-options-based mTLS implementation with RFC-compliant architecture: Domain model changes: - Add enforce_access_rules (boolean) and access_rules_scope (any/org/space) to domains - Fields are immutable after domain creation - Update DomainCreateMessage, DomainPresenter, and DomainCreate action Access Rules resource: - New /v3/access_rules API with full CRUD operations - RouteAccessRule model with guid, name, selector, route_id - Selector format: cf:app:<uuid>, cf:space:<uuid>, cf:org:<uuid>, or cf:any - Enforce cf:any exclusivity and per-route name/selector uniqueness - Space Developer can manage rules for routes in their space Diego sync path: - Inject access_scope and access_rules into route options for GoRouter - Filter internal mTLS keys (access_scope, access_rules) from public /v3/routes API - Add access_rules to eager load to avoid N+1 queries Tests: - Unit tests for AccessRuleCreateMessage (selector validation, cf:any rules) - Request specs for /v3/access_rules CRUD (create, show, list, delete, metadata update) - Updated domain_create_message_spec for enforce_access_rules validation - Updated routing_info_spec to verify mTLS options injection - Updated route_presenter_spec to verify internal keys are filtered Remove POC artifacts: - Remove app_to_app_mtls_routing feature flag - Remove mtls_allowed_* keys from route_options_message
- Replace non-existent readable_space_scoped_space_guids_query with proper subquery - Use readable_space_scoped_spaces_query for non-global readers - Handle global readers separately with all routes
- Add after_create and after_destroy callbacks to touch associated processes - Updates process.updated_at to trigger Diego ProcessesSync immediately - Eliminates 30-second wait for access rule changes to propagate to GoRouter - Add comprehensive unit tests for callbacks and validations - Ensure RouteAccessRule model is loaded in app/models.rb This enables automatic synchronization of access rules to Diego/GoRouter within seconds instead of requiring manual app restarts or waiting for the next sync cycle.
- Add include parameter support to AccessRulesListMessage - Refactor IncludeAccessRuleSelectorResourceDecorator to match RFC format: - Return separate arrays for apps, spaces, organizations instead of selector_resources - Include full resource details using appropriate presenters - Batch resource fetching by type with eager loading - Auto-deduplicate resources - Gracefully handle stale/missing resources - Wire up decorator to AccessRulesController - Add comprehensive request specs for include=selector_resource Fixes: uninitialized constant error by adding proper require statement
Implement space-based filtering for access rules endpoint to enable querying all access rules within a given space using ?space_guids=<guid> query parameter. Changes: - Add space_guids to AccessRulesListMessage with array validation - Implement space filtering in AccessRulesController#build_dataset - Add comprehensive unit tests for AccessRulesListMessage - Add request specs for single/multiple space filtering and combinations - Follow existing CAPI patterns for space_guids filtering The filter joins through the routes table to filter access rules by the space_id of their associated routes.
Add support for including route resources when listing access rules via the ?include=route query parameter. Changes: - Create IncludeAccessRuleRouteDecorator to handle route inclusion - Wire up decorator in AccessRulesController - Add comprehensive request specs for include=route - Test single/multiple routes, uniqueness, and combining with selector_resource - Follow existing CAPI decorator patterns for resource inclusion The decorator fetches and presents Route resources referenced by the access rules, adding them to the 'included' section of the response.
…RFC updates Update /v3/access_rules API to align with latest RFC changes: - Remove 'name' field from RouteAccessRule model and API - Add database migration to drop name column and unique index - Use labels/annotations for metadata instead of name field - Add read-only relationships (app, space, organization) to responses extracted from selector (cf:app:X, cf:space:X, cf:org:X, cf:any) - Replace 'names' filter with 'guids' filter - Add 'selector_resource_guids' filter for text-match against selectors - Update include support: add individual app, space, organization (in addition to existing selector_resource and route) - Remove name-based uniqueness validation (keep selector uniqueness) - Update all tests to remove name references Breaking changes: - POST /v3/access_rules no longer accepts 'name' field - GET /v3/access_rules responses no longer include 'name' field - Filter parameter 'names' removed, use 'guids' instead - Access rule responses now include app/space/organization relationships
- Create RouteAccessRuleLabelModel and RouteAccessRuleAnnotationModel - Add one_to_many relationships for labels and annotations to RouteAccessRule - Add database migrations for route_access_rule_labels and route_access_rule_annotations tables - Fixes: undefined method 'labels' error in AccessRulePresenter This enables metadata (labels/annotations) support for access rules, required by the RFC changes that removed the 'name' field in favor of using labels/annotations for metadata storage.
Add require statements for RouteAccessRuleLabelModel and RouteAccessRuleAnnotationModel to app/models.rb. Rails autoloading is disabled for app/** so all models must be explicitly required. This fixes the error: uninitialized constant VCAP::CloudController::RouteAccessRuleLabelModel
Per RFC requirement (line 246-247): Access rules cannot be created for routes on internal domains (domains created with --internal). Internal routes use container-to-container networking and bypass GoRouter entirely, so GoRouter cannot enforce access rules. Changes: - Add validation in AccessRulesController#create to reject access rules on internal domains with 422 status - Add test coverage for internal domain validation - Error message explains why: internal domains bypass GoRouter
…up tests - Collapse 4 migrations into 1 consolidated migration (20260407100001) that creates route_access_rules, route_access_rule_labels, and route_access_rule_annotations tables - Remove name field from access rules per RFC updates - Fix all RuboCop offenses: Style/CollectionQuerying (.count > 0 -> .any?), Migration/AddConstraintName (primary_key :id, name: :id), Metrics/BlockLength, Metrics/CyclomaticComplexity, and others - Add stale resource detection in presenter (null data for deleted resources) - Extract controller methods to reduce complexity - Use relative class names within VCAP::CloudController module - Fix test shadowing of rack-test app method (let(:app) -> let(:frontend_app)) - Fix Sequel validation assertion style (.include(:presence) not strings)
…ain API surface in access rules - Wrap create action in transaction with FOR UPDATE lock to prevent concurrent inserts from violating cf:any exclusivity constraints - Rescue Sequel::UniqueConstraintViolation to return 422 instead of 500 - Join routes table at most once when both route_guids and space_guids filters are requested, preventing ambiguous column references - Escape LIKE metacharacters (% and _) in selector_resource_guids filter - Replace deprecated routes__column syntax with Sequel[:routes][:column] - Remove per-row DB existence checks in AccessRulePresenter to eliminate N+1 queries; relationship GUIDs are now included directly from selector - Only include enforce_access_rules and access_rules_scope in domain responses when enforce_access_rules is true
…tization) Escape backslash characters before % and _ in selector_resource_guids LIKE filtering to prevent backslash-based injection.
Expand selector_resource_guids filtering tests to cover all three LIKE metacharacters: %, _, and backslash.
Annotations table used key VARCHAR(1000) with a unique index on (resource_guid, key), totaling 5020 bytes in utf8mb4 — exceeding MySQL's 3072-byte max key length. Align with codebase convention established in migration 20240102150000: use key_name VARCHAR(63) with a three-column unique index on (resource_guid, key_prefix, key_name). Also add NOT NULL default '' to key_prefix on both labels and annotations tables.
…Rule RouteAccessRule does not have a name column. The test was passing name: to create(), triggering Sequel::MassAssignmentRestriction.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements Part 2 (CF Identity & Authorization) of the RFC for Identity-Aware Routing for GoRouter. This adds a new Access Rules API that allows developers to control which Cloud Foundry apps can access routes on identity-aware domains.
Note: This PR is a draft because the RFC has not been approved yet.
What This Enables
Developers can now secure routes on identity-aware domains with platform-enforced access control:
GoRouter enforces these rules automatically - no code changes needed in the backend app.
Implementation Summary
1. Access Rules API (RFC-Compliant)
New
/v3/access_rulesendpoints for managing route-level access control:GET/v3/access_rulesGET/v3/access_rules/:guidPOST/v3/access_rulesPATCH/v3/access_rules/:guidDELETE/v3/access_rules/:guidQuery Parameters:
guids,route_guids,space_guids- Filter by resource GUIDsselectors- Filter by exact selector stringsselector_resource_guids- Text-match against GUIDs in selectors (for stale rule detection)include- Sideload related resources:route,app,space,organization,selector_resourceExample Request:
Response includes:
cf:app:<guid>,cf:space:<guid>,cf:org:<guid>,cf:any)2. Domain-Level Enforcement Configuration
Domains can now be configured with enforcement settings at creation time (immutable):
New Domain Fields:
enforce_access_rules(boolean) - When true, GoRouter enforces access rules for routes on this domainaccess_rules_scope(string:any,org,space) - Operator-level boundary for allowed callersThese fields are set via
POST /v3/domainsand are not present onPATCH /v3/domains(immutable by design).3. Validation Rules
Access rule creation enforces all RFC-specified validations:
enforce_access_rules: truecf:anycannot be combined with other selectors on the same route4. Metadata Support
Access rules support labels and annotations for auditability:
{ "metadata": { "labels": { "team": "frontend", "env": "prod" }, "annotations": { "contact": "frontend-team@example.com", "slack": "#frontend-support" } } }Implemented via:
RouteAccessRuleLabelModelandRouteAccessRuleAnnotationModelroute_access_rule_labels,route_access_rule_annotations5. Read-Only Relationships
Access rules expose read-only relationships extracted from the selector:
cf:app:<guid>app: { data: { guid } }space,organization→nullcf:space:<guid>space: { data: { guid } }app,organization→nullcf:org:<guid>organization: { data: { guid } }app,space→nullcf:anynullWhen the referenced resource no longer exists, the relationship data is
nullbut the selector string is preserved (for metadata like contact info).6. Diego Sync Integration
Access rules are automatically translated into GoRouter-compatible route options during Diego sync:
Process timestamps are updated on access rule create/delete to trigger
ProcessesSync.Database Migrations
Three new migrations:
20260415000001_remove_name_from_route_access_rules.rb- Removenamefield per RFC update20260415000002_create_route_access_rule_labels.rb- Labels table for metadata20260415000003_create_route_access_rule_annotations.rb- Annotations table for metadataPrevious migrations (already applied):
20260407100000_add_enforce_access_rules_to_domains.rb- Domain enforcement fields20260407100001_create_route_access_rules.rb- Access rules tableFiles Changed
Controllers:
app/controllers/v3/access_rules_controller.rb- Access Rules API implementationModels:
app/models/runtime/route_access_rule.rb- Core model with Diego sync callbacksapp/models/runtime/route_access_rule_label_model.rb- Labels metadataapp/models/runtime/route_access_rule_annotation_model.rb- Annotations metadataapp/models/runtime/domain.rb- Add enforcement fieldsapp/models/runtime/route.rb- Add access_rules relationshipapp/models.rb- Explicit requires for new models (Rails autoloading disabled)Messages:
app/messages/access_rule_create_message.rb- Create validationapp/messages/access_rule_update_message.rb- Update validation (metadata only)app/messages/access_rules_list_message.rb- List filteringapp/messages/domain_create_message.rb- Add enforcement fieldsPresenters:
app/presenters/v3/access_rule_presenter.rb- API response formatapp/presenters/v3/domain_presenter.rb- Add enforcement fieldsDecorators:
app/decorators/include_access_rule_selector_resource_decorator.rb- Sideload selector resourcesapp/decorators/include_access_rule_route_decorator.rb- Sideload routesActions:
app/actions/domain_create.rb- Set enforcement fields (immutable after creation)Diego Integration:
lib/cloud_controller/diego/protocol/routing_info.rb- Generate route optionsTests:
spec/request/access_rules_spec.rb- Full API integration testsspec/unit/models/runtime/route_access_rule_spec.rb- Model unit testsspec/unit/messages/access_rule*_spec.rb- Message validation testsRelated PRs
cf add-access-rule,cf access-rules, etc.): TBDTesting
All RFC requirements have test coverage:
Run tests:
Deployment Notes
rake db:migrateenforce_access_rules)Breaking Changes
None - this is additive functionality. Existing routes are unaffected.