Invantive Data Hub biedt standaard een integratie met Elastic APM.
De integratie van Invantive Data Hub met Elastic APM maakt het gemakkelijker om inzage en grip te houden op grote complexe omgevingen met veel verwerkingen.
Elastic APM is een apart bij derden af te nemen product van Elastic voor de bewaking en sturing van complexe IT-applicaties.
Elastic APM Werking
De Elastic APM-integratie van Data Hub registreert de volgende elementen:
- Invantive Trace: op diep detailniveau inzicht welke activiteiten uitgevoerd worden door wie, waar en wanneer;
- SQL-transacties: per uitvoering van een (P)SQL-statement wordt het executiepad gemeten en geregistreerd;
- Opgetreden fouten: per foutmelding wordt een registratie uitgevoerd.
- Relaties met andere services voor de Elastic APM service map.
Elastic APM Voorbeelden
Het is bijvoorbeeld mogelijk om te zien welke transacties wanneer uitgevoerd worden:
waarbij tot op individuele transacties van een SQL-statement doorgezoomd kan worden. In onderstaand voorbeeld is de uitvoering van een uitgebreid PSQL-statement met veel statements, elk met joins te zien:
Over alle uitvoeringen van een statement in een tijdsperiode heen is eenvoudig de distributie van de uitvoeringstijden beschikbaar:
Ook de achterliggende Invantive Trace van een statement is maar 1 klik weg in Elastic APM:
Ook een beeld over alle activiteiten heen van een service of server zijn eenvoudig en snel op te vragen, en te combineren met alerts of dashboards:
Appsettings.Production.json
Voor Invantive Data Hub is voor 24.0-releases sinds juli 2024 de aanwezigheid van een applicatieconfiguratiebestand appsettings.Production.json in de installatiemap vereist (verplaatst naar %USERPROFILE%/invantive vanaf 24.0.314).
Met dit configuratiebestand is het mogelijk om de transacties te registreren binnen een service op Elastic APM.
Data Hub zonder Elastic APM
Bij geen gebruik van Elastic APM is de inhoud van het configuratiebestand:
Inhoud geen gebruik
{ "Logging":
{ "LogLevel":
{ "Default": "Warning"
}
}
//
// Group to configure Invantive Data Hub.
//
, "DataHub":
{ //
// Elastic APM
//
"ElasticApm":
{ //
// Whether enabled.
//
"Enabled": false
//
// The API key used to send data to the APM server.
// Ensures that only your agents can send data to your APM server.
//
, "ApiKey": null
//
// When defined, all namespaces not starting with one of the values of this collection are ignored when determining
// Exception culprit.
//
, "ApplicationNamespaces": []
//
// Controls which baggage values are automatically attached to the given event (transactions, spans, errors). Baggage values are derived from the "baggage" header defined in
// https://www.w3.org/TR/baggage/.
// You can programmatically write and read baggage values via the Activity API.
//
, "BaggageToAttach": null
//
// For transactions that are HTTP requests, the agent can optionally capture the request body, e.g., POST variables.
// If the request has a body and this setting is disabled, the body will be shown as [REDACTED].
// Valid values are off, errors, transactions and all.
//
, "CaptureBody": "off"
//
// Configures for which content types the body should be captured.
//
, "CaptureBodyContentTypes": ["application/x-www-form-urlencoded*, text/*", "application/json*", "application/xml*"]
//
// Capture request and response headers, including cookies.
//
, "CaptureHeaders": true
//
// Whether the agent is configured to make periodic requests to the APM server to fetch and use the latest
// APM agent central configuration.
//
, "CentralConfig": false
//
// Specify which cloud provider should be assumed for metadata collection. By default, the agent will attempt to detect
// the cloud provider or, if that fails, will use trial and error to collect the metadata. Valid options are
// "aws", "gcp", and "azure".
// If this config value is set to "False", no cloud metadata will be collected.
//
, "CloudProvider": "aws"
//
// Disables the collection of certain metrics. If the name of a metric matches any of the wildcard expressions, it will
// not be collected
//
, "DisableMetrics": []
//
// Enables OpenTelemetry bridging. If this is set to true, the agent will automatically capture every
// <see cref="System.Diagnostics.Activity" /> as part of a trace captured in Elastic APM.
//
, "OpenTelemetryBridgeEnabled": true
//
// The name of the environment this service is deployed in.
//
, "Environment": "Production"
//
// A list of namespaces to exclude when reading an exception's StackTrace to determine the culprit.
// Namespaces are checked with string.StartsWith() so "System." matches all System namespaces
//
, "ExcludedNamespaces": [ "System.", "Microsoft.", "MS.", "FSharp.", "Newtonsoft.Json", "Serilog", "NLog", "Giraffe." ]
//
// Sets the minimum duration of exit spans in milliseconds. Exit spans with a duration lesser than this threshold are attempted to be
// discarded.
// If the exit span is equal or greater the threshold, it should be kept.
// In some cases exit spans cannot be discarded. For example, spans that propagate the trace context to downstream
// services, such as outgoing HTTP requests, can't be discarded.
// However, external calls that don't propagate context, such as calls to a database, can be discarded using this
// threshold. Additionally, spans that lead to an error can't be discarded.
//
, "ExitSpanMinDuration": 100.0
//
// The maximal amount of time (in seconds) events are held in queue until there is enough to send a batch.
// It's possible for a batch to contain less then <seealso cref="MaxBatchEventCount" /> events
// if there are events that need to be sent out because they were held for too long.
// A lower value will increase the load on your APM server,
// while a higher value can increase the memory pressure on your app.
// A higher value also impacts the time until transactions are indexed and searchable in Elasticsearch.
//
// * Positive number - The maximal amount of time to hold events in queue.
// * 0 - Events are not held in queue but are sent immediately.
// * Negative - Invalid and the default value is used instead.
//
, "FlushInterval": "00:00:10"
//
// Unknown.
//
, "GlobalLabels": {}
//
// Allows for the reported hostname to be manually specified. If unset, the hostname will be detected.
//
, "HostName": "ENTER_YOUR_HOST_NAME"
//
// Disables the tracing of messages from certain queues, topics exchanges.
// If the name of a queue, topic or exchange matches any of the wildcard expressions, it will
// not be traced
//
, "IgnoreMessageQueues": []
//
// The logging level for the agent.
//
, "LogLevel": 3
//
// The maximal number of events to send in a batch.
// It's possible for a batch contain less then the maximum events
// if there are events that need to be sent out because they were held for too long.
//
// * Positive number - The maximal number of of events to send in a batch.
// * 0 and negative - Invalid and the default value is used instead.
//
, "MaxBatchEventCount": 10
//
// The maximal number of events to hold in queue as candidates to be sent.
// If the queue is at its maximum capacity then the agent discards the new events
// until the queue has free space.
// * Positive number - The maximal number of of events to send in a batch.
// If MaxQueueEventCount is less than MaxBatchEventCount then
// MaxBatchEventCount is used as MaxQueueEventCount.
// * 0 and negative - Invalid and the default value is used instead.
//
, "MaxQueueEventCount": 1000
//
// Unknown.
//
, "MetricsIntervalInMilliseconds": 30000.0
//
// Whether the agent is recording.
// When set to true. the agent instruments and capture requests, tracks errors, and
// collects and sends metrics.
// When set to false, the agent does not collect data or communicate with the APM server, except to
// fetch central configuration.
// Recording can be changed during the lifetime of the application.
//
, "Recording": true
//
// Sometimes it is necessary to sanitize the data sent to Elastic APM, e.g. remove sensitive data.
// Configure a list of wildcard patterns of field names which should be sanitized.
// These apply for example to HTTP headers and application/x-www-form-urlencoded data.
//
, "SanitizeFieldNames": []
//
// A secret token to ensure that only your agents can send data to your APM server.
// Both agents and APM server have to be configured with the same secret token.
//
, "SecretToken": "ENTER_YOUR_SECRET_TOKEN"
//
// The path to the PEM-encoded certificate used by APM server. This can be used when using a certificate
// signed by a Certificate Authority (CA) that is not in the trust store, such as a self-signed certificate,
// to perform validation through certificate pinning.
//
, "ServerCert": null
//
// The URL for APM server.
//
, "ServerUrl": "https://ENTER_YOUR_APM_SERVER_NAME.apm.eu-west-1.aws.cloud.es.io:443"
//
// The URLs for APM server.
//
, "ServerUrls": null
//
// If set to true, Agent will use windows default credentials that the service running the agent has to send requests if it is needed for proxy.
//
, "UseWindowsCredentials": false
//
// The name of service instrumented by the APM agent. This is used to group all the errors and transactions
// of the service together, and is the primary filter in the Elastic APM user interface.
//
, "ServiceName": "Invantive Data Hub"
//
// A name used to differentiate between nodes in a service. If not set, data aggregations will be done
// based on a container ID (where valid) or on the reported hostname (automatically discovered).
//
, "ServiceNodeName": null
//
// The version of the service.
// If deployments are not versioned, it is recommended to set this to the commit identifier of the deployed revision,
// e.g. the output of git rev-parse HEAD.
//
, "ServiceVersion": "24.0.291"
//
// Setting this option to true will enable span compression feature.
// Span compression reduces the collection, processing, and storage overhead, and removes clutter from the UI.
// The tradeoff is that some information such as DB statements of all the compressed spans will not be collected.
//
, "SpanCompressionEnabled": true
//
// Consecutive spans that are exact match and that are under this threshold will be compressed into a single composite
// span. Measurement is in milliseconds.
// This option does not apply to composite spans. This reduces the collection, processing, and storage overhead, and
// removes clutter
// from the UI.
// The tradeoff is that the DB statements of all the compressed spans will not be collected.
//
, "SpanCompressionExactMatchMaxDuration": 100.0
//
// Consecutive spans to the same destination that are under this threshold will be compressed into a single composite
// span.
// This option does not apply to composite spans.
// This reduces the collection, processing, and storage overhead, and removes clutter from the UI. The tradeoff is that
// the DB statements of
// all the compressed spans will not be collected.
//
, "SpanCompressionSameKindMaxDuration": 100.0
//
// The agent limits stack trace collection to spans with durations equal or longer than the given value
// negative value: Disables stack trace collection for spans completely
// 0: stacktrace will be collected for all spans
// positive value n: stacktrace will be captured for spans with a duration equal or longer than n ms.
//
, "SpanStackTraceMinDurationInMilliseconds": 100.0
//
// The agent limits stack trace collection to spans with durations equal or longer than the given value
// 0: Disables stack trace collection for spans completely
// negative value: stacktrace will be collected for all spans
// positive value n: stacktrace will be captured for spans with a duration equal or longer than n ms.
//
, "SpanFramesMinDurationInMilliseconds": 100.0
//
// The number of stack frames the agent collects.
// 0: no stacktrace is collected - This also applies to spans no matter what is the value of
// SpanFramesMinDurationInMilliseconds.
// negative: all frames must be collected
// positive number n: top n frames must be collected
//
, "StackTraceLimit": -1
//
// By setting this to true the agent will ignore the sampled part of the W3C TraceContext traceparent header
// when it's false and when the upstream transaction is not coming from an Elastic APM agent.
// In practice this means that in case a caller service calls another service where this value is true,
// the agent will ignore the sampling decision of the upstream service, and it will make a new sampling decision.
// This can be useful when a caller service always sets the sampled flag to false and the agent would have no chance to
// create any sampled transaction.
//
, "TraceContextIgnoreSampledFalse": false
//
// The traceparent header of requests that are traced with our agents might have been added by a 3rd party component.
// This situation becomes more and more common as the w3c trace context gets adopted. In such cases we can end up with
// traces where part of the trace is outside of our system.
// In order to handle this properly, the agent SHOULD offer several trace continuation strategies.
// The agent SHOULD offer a configuration called trace_continuation_strategy with the following values and behavior:
// - continue: The agent takes the traceparent header as it is and applies it to the new transaction.
// - restart: The agent always creates a new trace with a new trace id. In this case the agent MUST create a span link in
// the new transaction pointing to the original traceparent.
// - restart_external: The agent first checks the tracestate header. If the header contains the es vendor flag, it's
// treated as internal, otherwise (including the case when the tracestate header is not present) it's treated as external.
// In case of external calls the agent MUST create a new trace with a new trace id and MUST create a link in the new
// transaction pointing to the original trace.
//
, "TraceContinuationStrategy": "continue"
//
// A list of patterns to match HTTP requests to ignore. An incoming HTTP request whose request line matches any of the
// patterns will not be reported as a transaction.
//
, "TransactionIgnoreUrls": []
//
// A list of patterns to be used to group incoming HTTP server transactions by matching names contain dynamic parts
// to a more suitable route name.
//
, "TransactionNameGroups": []
//
// The number of spans that are recorded per transaction.
// * 0: no spans will be collected.
// * -1: all spans will be collected.
//
, "TransactionMaxSpans": 500
//
// The sample rate for transactions.
// By default, the agent will sample every transaction (e.g. a request to your service). To reduce overhead and storage
// requirements,
// the sample rate can be set to a value between 0.0 and 1.0. The agent still records the overall time and result for
// unsampled
// transactions, but no context information, labels, or spans are recorded.
//
, "TransactionSampleRate": 1.0
//
// If true, for all outgoing HTTP requests the agent stores the traceparent in a "elastic-apm-traceparent" header
// name.
// Otherwise, it'll use the official w3c "traceparent" header name.
//
, "UseElasticTraceparentHeader": false
//
// If true, the default, the agent will use the path of the incoming HTTP request as the transaction name in situations
// when a more accurate route name cannot be determined from route data or request headers.
//
, "UsePathAsTransactionName": true
//
// The agent verifies the server's certificate if an HTTPS connection to the APM server is used.
// Verification can be disabled by setting to false.
//
, "VerifyServerCert": true
//
// A description for the configuration.
//
, "Description": "Invantive Data Hub"
}
}
}
Data Hub Met Elastic APM
Bij gebruik van Elastic APM dient het bestand in de basis alleen op de volgende onderdelen aangepast te worden:
Enabled:falsewordttrue.HostName: invullen met gewenste naam device in APM of leeglaten voor default.SecretToken: token uit Elastic APM.ServerUrl: server URL uit Elastic APM.
Deze gegevens zijn te vinden in het menu bij “Management” > “Integration” en dan “APM”:
onder:
