# Music streaming DJ This is an example to illustrate the concepts of the [Langfuse Academy](/content/academy/index.html). ## Context A music streaming app adds a DJ feature in beta. The DJ keeps a queue going based on what the user has played before and what the algorithm thinks the user will like. Every few songs it says a short word about what's coming up and why it chose it. Listeners can also optionally interact with the DJ to steer it, by clicking a microphone button and talking to it. --- ### Good evening **Alex** **Your DJ** **Mixing songs for your evening** It picks the tracks, talks you through them, and listens when you talk back. ### Now playing DJ · why this song A deep cut after that album you ran on repeat last week — easing you in before I pick the pace up. **Slow Tide** **Marlowe** 0:48 3:36 ### Talk to DJ Listening… Play something with a bit more energy Try saying Who is this? Skip this one More like this Tap the orb to stop --- Two characteristics drive how to approach the AI engineering setup: 1. The **risk of a bad output is low**. The worst case is a skipped song or an awkward DJ comment, and the feature is labeled beta. 2. This is a new feature, so **there is no historical data to start from**, and the team will need to learn from live listener behavior. Because of this, it makes sense to go live as soon as possible and [iterate based on live feedback from the beginning](/content/academy/ai-engineering-loop#you-dont-have-to-close-the-full-loop-on-day-one/index.html). ## Tracing the DJ There are two different kinds of [traces](/content/academy/tracing#anatomy-of-a-trace/index.html) that will together [form a session](/content/academy/tracing#traces-vs-sessions/index.html): | Trace name | Details | |----------------------|------------------------------------------------------------------------------------------------------------------------------------------| | `plan-next-set` | Plans the next set of tracks and writes the commentary, triggered by the DJ itself every few songs or by a DJ request. | **input:** | listening context and any instructions | | **output:** | next tracks queued, plus the commentary line | | `handle-dj-request` | Handles a voice request from the listener, triggered by the microphone button. | **input:** | voice clip and the current queue | | **output:** | a short reply, plus instructions for the `plan-next-set` run it triggers | A listening session and both trace types up close: Example session Example plan-next-set trace Example handle-dj-request trace Session: listen_7f3e · user: u_8841 - **plan-next-set**: "Kicking off with two favorites from your week." 2.8s - **plan-next-set**: "Staying in this lane with some mellow electronica." 2.4s - **handle-dj-request**: "Play something calmer." 1.6s - **plan-next-set**: "Calming it down, here is some ambient piano." triggered by: dj-request, 1.5s ## Capturing user behavior In order to learn from users, we will log some [interesting behavior](/content/academy/monitoring#user-feedback/index.html) on the applicable traces. - **track_skipped**: The listener skips a DJ-picked track shortly after it starts - **dj_replaced**: The listener turns the DJ off but keeps listening - **message_type**: Classifies each voice request as `steering`, `correction`, `repeated_instruction`, `positive_reaction`, or `negative_reaction` In principle, the team could already iterate with only this in place. Tracing and [monitoring](/content/academy/monitoring#metrics-and-signals/index.html) form a small loop of their own. In the beginning, this is probably enough for the team to quickly improve the DJ feature. ## Structured testing With only the live signals, testing a change means shipping it and watching the scores. The team can add two more deliberate ways of testing: [experiments on datasets](/content/academy/examples/music-streaming-dj#experiments-on-datasets/index.html), and A/B tests on live users. ### Experiments on datasets In this use case, testing end to end is very hard offline: whether a session was good only shows in live listening behavior, and taste differs per user, so no expected output holds for everyone. [A single step](/content/academy/datasets#what-makes-a-good-dataset/index.html) like `select-tracks` can be tested, with the expectation describing the direction of the set rather than exact tracks: - **selection-directions**: Listening contexts paired with good and bad directions for the next set. - **direction_match**: Does the selected set go in one of the good directions and avoid the bad ones? ### A/B tests on live users Some changes are hard to grade offline, like the tone of voice of the commentary. Since the risk is low, the team can give a small group of listeners the new version and compare the signal scores between the groups, like the skip rate and the `message_type` distribution. If the new version does better, they can roll it out to everyone. With this in place, the full loop is running: Deploy Trace every set and every request Monitor skips, dj_replaced, message_type, A/B comparisons Build datasets from what we see while monitoring production Experiment selection algorithm, DJ prompt, ... Evaluate grade step outputs against their expectations ## Conclusion This is an example of a feature that's low risk and there is no historical data to learn from. The best thing you can do in such cases is getting traces in as soon as possible and start monitoring them to learn from. Everything else, datasets, experiments, A/B tests, can be built on top of that over time.