#Grafana 12 is here with major upgrades!
Now Generally Available: Git Sync, dynamic dashboards, improvements to Drilldown which gives code-free point-and-click insights into data, and a Cloud Migration assistant.
Explore what's new: https://bit.ly/4dyu3cB
#InfoQ #DevOps #Monitoring #Logging #DistributedTracing #OpenTelemetry
Fastrace: A Modern Approach to #DistributedTracing in #Rust
https://fast.github.io/blog/fastrace-a-modern-approach-to-distributed-tracing-in-rust/
🚀 Dive into Distributed Tracing with Helidon, Coherence, and Oracle Autonomous Database using OpenTelemetry!
In this article, Ali Mukadam explains how to enhance observability in modern applications. A must-read for developers and architects!
📖 Read it here: https://medium.com/helidon/distributed-tracing-in-helidon-and-coherence-with-opentelemetry-9f2062ba75b4
#DistributedTracing #OpenTelemetry #Helidon #Coherence #OracleDatabase #Tech #Observability
📰 [News] Distributed Tracing Tool Jaeger Releases Version 2 with OpenTelemetry at the Core by Matt Saunders
#Jaeger v2 is out! This release contains a significant architectural transformation, bringing Jaeger and its components into the OpenTelemetry framework.
Trace Context: #MQTT protocol
#KubeCon news rail: #Jaeger V2 is heading to GA at #KubeConNA 🎉
Hear all about the long-awaited major release of the #DistributedTracing tool, powered by #OpenTelemetry, on our recent episode with #JaegerTracing creator
@yurishkuro
https://horovits.medium.com/be612dbee774
Have you heard about the #Jaeger V2 major release?
It presents Distributed Tracing Powered by OpenTelemetry.
Here are insights from #JaegerTracing creator and core maintainer Yuri Shkuro:
http://medium.com/p/be612dbee774
Want to learn more? join #kubecon NA next week to follow the launch and for more updates.
#distributedTracing #opentelemetry #opensource #cloudnative #observability @opentelemetry @linuxfoundation
📢 October episode of OpenObservability Talks is out!
It's time to unveil #Jaeger V2 🍾
I sat down with Jaeger creator Yuri Shkuro to unpack the highly anticipated V2 of the popular #opensource #DistributedTracing tool, re-architecture with #OpenTelemetry Collector and more.
Check out the episode on YouTube or wherever you get your podcasts:
🎧 https://podcasters.spotify.com/pod/show/openobservability/episodes/Jaeger-V2-Unveiled-Powered-by-OpenTelemetry---OpenObservability-Talks-S5E05-e2q7ri0
📺 https://www.youtube.com/watch?v=lICivVwm-F8&list=PLd57eY2edRXz4djMETYTm-2p8WGTdoX3D
Or read the recap post: https://horovits.medium.com/jaeger-v2-unveiled-distributed-tracing-powered-by-opentelemetry-be612dbee774
#Jaeger V2 is here!🎉
The popular open source #DistributedTracing tool under the #CNCF, is introducing a new architecture with deep #OpenTelemetry integration, which promises more flexibility, performance, extensibility and ease of use.
Next week I'll hold a live stream of 𝗢𝗽𝗲𝗻𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗧𝗮𝗹𝗸𝘀 🎙 with Jaeger's creator and core maintainer Yuri Shkuro.
Join us October 14th at 11 AM PT / 20:00 CET:
👉
https://www.youtube.com/watch?v=lICivVwm-F8
📢 The RC1 of @JaegerTracing V2 is out.
This release contains an official pre-release candidate of Jaeger v2, as binary and Docker image, so you can test it out
https://github.com/jaegertracing/jaeger/releases/tag/v1.61.0?s=09
#distributedtracing #observability #opensource #microservices #cloudnative #DevOps #jaeger #jaegertracing
Is it possible to do #DistributedTracing without the distributed part?
I want a CLI/desktop app (#Rust, if it matters) to collect a trace, like I would with #OpenTelemetry or similar.
No sending it over the network, just dump it into a file on exit. One process, one execution, one trace dump. Then I want to load that file into some GUI and explore it like in Honeycomb or Jaeger (or browser's dev console).
Any ideas how to do that? I'll add some more info in 🧵
📢 New, small one, up on the interwebs!
"Transactional outbox pattern meets distributed tracing and OpenTelemetry"
I continue to work on my #OpenTelemetry demo. Its main idea is to showcase traces across various technology stacks, including #asynchronous communication via an #MQTT queue. This week, I added a couple of components and changed the #architecture. Here are some noteworthy learnings; note that some of them might not be entirely connected to @opentelemetry
https://blog.frankel.ch/even-more-opentelemetry/
#Observability #DistributedTracing #ApacheAPISIX @graalvm #Golang #Ruby
Going on stage at #DevoxxUK
to talk about distributed tracing and how you can use #opentelemetry in #Java for that.
This weekend I was frustrated with my debugging, and just not up to digging in and carefully, meticulously analyzing what was happening. So … I took a left turn (at Alburquerque) and decided to explore an older idea to see if it was interesting and/or useful. My challenging debugging was all about network code, for a collaborative, peer to peer sharing thing; more about that effort some other time.
A bit of back story
A number of years ago when I was working with a solar energy manufacturer, I was living and breathing events, APIs, and running very distributed, sometimes over crap network connections, systems. One of the experiments I did (that worked out extremely well) was to enable distributed tracing across the all the software components, collecting and analyzing traces to support integration testing. Distributed tracing, and the now-popular CNCF OpenTelemetry project weren’t a big thing, but they were around – kind of getting started. The folks (Yuri Shkuro, at least) at Uber had released Jaeger, an open-source trace collector with web-based visualization, which was enough to get started. I wrote about that work back in 2019 (that post still gets some recurring traffic from search engines, although it’s pretty dated now and not entirely useful).
We spun up our services, enabled tracing, and ran integration tests on the whole system. After which, we had the traces available for visual review. It was useful enough that we ended up evolving it so that a single developer could stand up most of their pieces locally (with a sufficiently beefy machine), and capture and view the traces locally. That provided a great feedback loop as they could see performance and flows in the system while they were developing fixes, updates and features. I wanted to see, this time with an iOS/macOS focused library, how far I could get trying to replicate that idea (time boxed to the weekend).
The Experiment!
I’ve been loosely following the server-side swift distributed tracing efforts since it started, and it looked pretty clear that I could use it directly. Moritz Lang publishes swift-otel, which is a Swift native, concurrency supported library. With his examples, it was super quick to hack into my test setup. The library is set up to run with service-lifecycle pieces over SwiftNIO, so there’s a pile of dependencies that come in with it. To add to my library, I’d be a little hesitant, but an integration test thing, I’m totally good with that. There were some quirks to using it with XCTest, most of which I hacked around by shoving the tracer setup into a global actor and exposing an idempotent bootstrap call. With that in place, I added explicit traces into my tests, and then started adding more and more, including into my library, and could see the results in a locally running instance of Jaeger (running Jaeger using Docker).
Some Results
The following image is an overview of the traces generated by a single test (testCreate):
The code I’m working with is all pushing events over web sockets, so inside of the individual spans (which are async closures in my test) I’ve dropped in some span events, one of which is shown in detail below:
In a lot of respects, this is akin to dropping in os_signposts that you might view in Instruments, but it’s external to Xcode infrastructure. Don’t get me wrong, I love Instruments and what it does – it’s been amazing and really the gold standard in tooling for me for years – but I was curious how far this approach would get me.
Choices and Challenges
Using something like this in production – with live-running iOS or macOS apps – would be another great end-to-end scenario. More so if the infrastructure your app was working from also used tracing. There’s a separate tracing project at CNCF – OpenTelemetry Swift – that looks oriented towards doing just that. I seriously considered using it, but I didn’t see a way to use that package to instrument my library and not bring in the whole pile of dependencies. With the swift-distributed-tracing library, it’s an easy (and small) dependency add – and you only need to take the hit of the extra dependencies when you want to use the tracing.
And I’ll just “casually” mention that if you pair this with server-side swift efforts, the Hummingbird project has support for distributed tracing currently built in. I expect Vapor support isn’t too far off, and it’s a continued focus to add more distributed tracing support for a number of prevalent server-side swift libraries over this coming summer.
See for Yourself (under construction/YMMV/etc)
I’ve tossed up my hack-job of a wrapper for tracing during testing with iOS and macOS – DistributedTracer, if you want to experiment with this kind of thing yourself. Feel free to use it, although if you’re amazed with the results – ALL credit should go to Moritz, the contributors to his package and the contributors to swift-distributed-tracing, since they did the heavy lifting. The swift-otel library itself is undergoing some major API surface changes – so if you go looking, I worked from the current main branch rather than the latest release. Moritz shared with me that while the API was not completely solid yet, this is more of the pattern he wants to expose for an upcoming 1.0 release.
Onward from here
I might push the DistributedTracer package further in the future. I think there’s real potential there, but it is not without pitfalls. Some of the challenges stem from constantly exporting data from an iOS app, so there’s a privacy (and privacy manifest) bit that needs to be seriously considered. There are also challenges with collecting enough data (but not too much), related choices in sampling so that it aligns with traces generated from infrastructure, as well as how to reliably transfer it from device to an endpoint. Nothing that can’t be overcome, but it’s not a small amount of work either.
Weekend hacking complete, I’m calling this a successful experiment. Okay, now back to actually debugging my library…
https://rhonabwy.com/2024/04/02/distributed-tracing-with-testing-on-ios-and-macos/
Looking forward to speaking next month at #Devoxx UK again, this time in the #Java track, on how to implement Distributed Tracing using #OpenTelemetry.
It's a great way to gain #observability into your #microservices, understand the dependency graph and where performance bottlenecks are.
So if you're a Java programmer, don't miss this one out.
See you 8th May in London 🇬🇧
https://www.devoxx.co.uk/talk/?id=10633
#distributedTracing #opensource #programming
📢 Jaeger 1.55.0 is out, including:
▪️Support for uploading traces to UI in OpenTelemetry format (OTLP/JSON)
▪️Add Elasticsearch storage support for adaptive sampling
And more, with progress towards the next major release, i.e. 𝗝𝗮𝗲𝗴𝗲𝗿 𝘃𝟮.
Check out the updates on the Distributed Tracing graduated project of the Cloud Native Computing Foundation #CNCF on the release notes:
https://github.com/jaegertracing/jaeger/releases/tag/v1.55.0
#cloudnative #devops #opensource #observability #distributedtracing
I've been following #Tracetest since it started in May 2022 and it's always such a treat to speak with my Tracetest friends. This week on @geekingout_pod, I speak with Adnan Rahić of Tracetest about all things trace-based testing, including how trace-based testing à la Tracetest was added to the OTel Demo.
We've been asked whether this was achievable at all using #eBPF, and now we have finally shipped it!
This is an insane super-power: Viewing all CPU-time of a request all in one flamegraph/iciclegraph! No other profiling tool can do this infrastructure-wide!
#opensource #opentelemetry #profiling #distributedtracing #tracing
https://www.polarsignals.com/blog/posts/2024/03/05/correlating-tracing-with-profiling-using-ebpf
