Skip to content

[Draft] OpenTelemetry out-of-process forwarder integration#9135

Draft
rajkumar-rangaraj wants to merge 11 commits intomainfrom
rajrang/otel-forwarder-integration
Draft

[Draft] OpenTelemetry out-of-process forwarder integration#9135
rajkumar-rangaraj wants to merge 11 commits intomainfrom
rajrang/otel-forwarder-integration

Conversation

@rajkumar-rangaraj
Copy link
Copy Markdown

@rajkumar-rangaraj rajkumar-rangaraj commented Mar 27, 2026

This is a draft PR containing the full OTel forwarder integration. It will be split into smaller PRs for review. Posting as a draft to give reviewers an overview of the complete feature.

What this does

Integrates OpenTelemetry.OutOfProcess.Forwarder.Configuration package into dotnet-monitor to forward logs, metrics, and traces from target .NET applications via OTLP — without requiring any changes to the target app.

Planned PR breakdown

Note: This draft PR is for visibility only and will not be merged. The changes will be submitted as separate, smaller PRs in the order below for easier review. Each PR will target main and can be reviewed and merged independently.

  1. InfrastructureENABLE_OTEL conditional compilation, package reference, generate-dev-sln.ps1 update
    • dotnet-monitor.csproj, Directory.Packages.props, generate-dev-sln.ps1
  2. MetricsStore changesSnapshotMetrics overload, MetricsSnapshot types, instance processStartTimeUtc
    • IMetricsStore.cs, MetricsStore.cs
    • Required by signal implementations (PR 3)
  3. Core pipelineOpenTelemetryService, EndpointManager, EndpointListener, EndpointInfoSourceCallbacks, DI registration
    • OpenTelemetryService.cs, OpenTelemetryEndpointManager.cs, OpenTelemetryEndpointListener.cs, OpenTelemetryEndpointInfoSourceCallbacks.cs, OpenTelemetryServiceCollectionExtensions.cs, CollectCommandHandler.cs
    • Depends on PR 1
  4. Signal implementationsOpenTelemetryLogRecordLogger, OpenTelemetryCountersLogger + MetricProducer, OpenTelemetryActivityLogger
    • OpenTelemetryLogRecordLogger.cs, OpenTelemetryCountersLogger.cs, OpenTelemetryMetricProducer.cs, OpenTelemetryMetricProducerFactory.cs, OpenTelemetryActivityLogger.cs
    • Depends on PRs 2 and 3
  5. Documentationopentelemetry-configuration.md, config README update
    • opentelemetry-configuration.md, README.md
  6. Test configtest-otel-config.json

Files changed (18 files, +2055 / -31)

File Change
Directory.Packages.props Added OpenTelemetry.OutOfProcess.Forwarder.Configuration package version
documentation/configuration/README.md Added link to OpenTelemetry configuration doc
documentation/configuration/opentelemetry-configuration.md New — full configuration documentation
generate-dev-sln.ps1 Removed stale OTel project refs, removed $OtelRepoRoot param, added ENABLE_OTEL
src/.../Metrics/IMetricsStore.cs Added MetricsSnapshot, MetricsSnapshotMeter, MetricsSnapshotInstrument types
src/.../Metrics/MetricsStore.cs Added SnapshotMetrics(out MetricsSnapshot) overload, instance processStartTimeUtc
src/.../Commands/CollectCommandHandler.cs Wired services.AddOpenTelemetry() under #if BUILDING_OTEL
src/.../Extensions/OpenTelemetryServiceCollectionExtensions.cs New — DI registration (internal)
src/.../Logging/OpenTelemetryLogRecordLogger.cs New — converts EventPipe logs → OTel log records
src/.../Metrics/OpenTelemetryCountersLogger.cs New — bridges counter data to OTel metrics
src/.../Metrics/OpenTelemetryMetricProducer.cs New — custom MetricProducer for OTel SDK
src/.../Metrics/OpenTelemetryMetricProducerFactory.cs New — factory for metric producers
src/.../OpenTelemetryEndpointInfoSourceCallbacks.cs New — lifecycle callbacks for process connect/disconnect
src/.../OpenTelemetryEndpointListener.cs New — per-process listener managing log/metric/trace pipelines
src/.../OpenTelemetryEndpointManager.cs New — manages listeners, IAsyncDisposable
src/.../OpenTelemetryService.cs New — BackgroundService for process discovery
src/.../Tracing/OpenTelemetryActivityLogger.cs New — converts activities → OTel spans
src/.../dotnet-monitor.csproj Conditional package ref + BUILDING_OTEL define
test-otel-config.json New — example test configuration

Testing

Verified end-to-end with a sample ASP.NET app (WebApplication2):

  • ✅ Metrics exported to http://localhost:4318/v1/metrics
  • ✅ Logs exported to http://localhost:4318/v1/logs
  • ✅ Traces exported to http://localhost:4318/v1/traces
  • ✅ Config reload (options change triggers pipeline restart)
  • ✅ Process discovery via DefaultProcess filter

CodeBlanch and others added 9 commits April 24, 2025 09:41
- Replace project reference with NuGet package reference
- Rename OTEL_REPO_ROOT to ENABLE_OTEL; make OTel package conditional
- Fix async void StopListening -> async Task StopListeningAsync
- Add thread safety (lock) to EndpointListener start/stop
- Add IAsyncDisposable to EndpointManager
- Fix exception handling in service retry loop
- Improve process discovery log message (info instead of warn+stacktrace)
- Add per-pipeline error handling with RunPipelineAsync wrapper
- Promote pipeline lifecycle logs from Trace to Information level
- Fix static processStartTimeUtc -> instance field in MetricsStore
- Change visibility: public -> internal on ServiceCollectionExtensions
- Bound maxMetricCount to 50k instead of int.MaxValue
- Fix documentation to match actual JSON config format
- Fix test-otel-config.json (Batch -> PeriodicExporting for metrics)
- Remove stale project refs from generate-dev-sln.ps1
- Perf: Cache InstrumentationScope objects in log/trace loggers
- Perf: Reuse attribute list in MetricProducer.ParseAttributes
- Perf: Reuse DiagnosticsClient per listener
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants