The OpenTelemetry Bootcamp: Sampling and dealing with high volumes

Learn how to use OpenTelemetry to get full-stack observability of your distributed system.
The OpenTelemetry Bootcamp brings you a weekly video series of everything you need to get started with OpenTelemetry. From zero to hero, this is where you get your OpenTelemetry questions answered.
🤓 This is a live Q&A format, so feel free to ask anything in the chat!
🔗 Join the Bootcamp's channel on the CNCF Slack to keep the conversation going: app.slack.com/client/T08PSQ7B...
--
🎉 The OpenTelemetry Bootcamp Episode 4! 🎉
💻 In this episode:
✅ Managing sample rate
✅ Tradeoffs to consider
✅ Cost
--
📆 Upcoming Bootcamp video schedule: www.aspecto.io/opentelemetry-...
--
#opentelemetry #opentelemetrybasics #cncf #opentelemetryarchitecture #cloudnative #distributedtracing #microservices #observability
--
0:00 - Intro
2:06 - What to expect
5:12 - How much data should you collect?
8:52 - Why traces are expensive
10:30 - How much tracing data should you collect?
16:05 - How does sampling work?
18:37 - Tail sampling
29:41 - Head sampling
34:50 - Sampling use cases
48:53 - Collector performance optimization
50:53 - Tail sampling performance benchmarks
51:45 - Auto scaling the collector
53:05 - Auto scaling your database
54:52 - Cost & how to calculate it
59:28 - Tips for production
1:00:52 - Summary
1:02:15 - Deploying to prod cheat sheet
1:03:14 - Q: Managing tracing infrastructure
1:05:24 - Outro

[@heyitspranav]

I was struggling to learn about opentelemetry by referring to the official website. Fortunately, landed to these bootcamp sessions, now everyone has started to make sense. It would be great if you could do a video on OTEL Java SDK. Thanks,

[@aspecto_io]

Thanks Pranav! happy to hear that. Just 2 weeks ago we released a getting started video for Python. Will follow with NodeJS and Java soon! [later this week we have a special episode on OTel + Async messaging].

[@amolfuke]

Thanks for Great session. I have few questions.
What is the benefit of using opentelemetry over normal spring boot default metrics? We have a batch auto scaling application and We are doing analysis if We can use otel

[@youngbae7170]

Thank you for the great contents. I have one question - I'm not cleear on what the purpose of the wait time is when using the tail sampling? Once the tracing is completed, how does the wait time help making a better decision on whether or not to send the trace to the backend system?

[@aspecto_io]

Hi! If the collector is ingesting span by span, then it should wait to aggregate the whole trace, this is where the wait time comes into play - how much the collector should wait before assuming to trace is complete and taking a decision