Looking for a way to get source code at runtime


I’m thinking about a way I’d like to visualise genstage/flow pipelines possibly with liveview. It’s a tricky thing but quite quickly I realised I need

  1. flow stats - messages going through each stage per second, maybe some demand stats
  2. A view to the code executing in each stage.

I guess the idea is to get something like a rudimentary spark ui type view. Is anyone already working on something like this?

Flow stats are easy with telemetry. But I have no idea if you can get code that is running? Any ideas?

For example, in a flatmap which is splitting a string, i’d like to get back "&String.split(&1, " “)”

Any ideas would be appreciated.


You can try to get it via beam_lib, however whether it work or not highly depends if you strip your releases.

It might be better to have users instrument their stages with a macro. Then you can ensure appropriate information is set. Otherwise you’d need releases that aren’t stripped as mentioned above. It might also be possible using a compiler tracing plugin?


thanks for the ideas, i’ll look into them and get back if i have a solution :slight_smile: