@rm-rf-etc sorry for a late reply - I was away from a computer for a couple of days, but I’ve seen your message and have been thinking about it.
Regards abstractions and trying to squeeze in everything into a single system - partly I would agree that some things could be generalized for example “tick” prices/candles, etc for the display purposes - crypto, forex, stocks could just have price and time fields and that’s about it - so really simplified no generalized.
From everything that I’ve done over the years, I think the system needs to be really simple to succeed. People will build on top of it so it needs to be as simple as this is the data and you do whatever you want with it (write your strategy). In my course (which looks already much different than Igthorn: https://github.com/frathon/hedgehog/tree/video_07 [at this moment at the 7th episode]) I took a PubSub approach. If you think about it - the only things that consumer needs to know is the data struct that will be broadcasted and the topic name. It works for anything that you want to stream - just prefix your stuff with type of what you stream. You would have topics like “stocks:AAPL”, “forex:GBPUSD”, “crypto:ADAUSDT” - nothing would interfere with each other. Then you write your specific consumers for crypto data, forex data, or stock data. There will be multiple consumers for single topic - one storing data to Postgres (or timescaleDB - or both[would guess that would be 2 consumers though!] as this could be beneficial as well!), one making some intermediate indicators and publishing them forward to topics like “forex:GBPUSD:ATR” or something like that. Obviously one of the consumers will be your strategy.
The above idea works perfectly with backtesting as you can just open the database (or a file) and publish everything to the topic and in the end, you will have a database with all orders, transactions and anything else that would happen.
I used Postgres as I don’t have any experience with timescaleDB. I would shy away from dropping too
much tech if there’s no extremely good reason for it.
I know of one more trading project from @rupurt - https://github.com/fremantle-industries/workbench
I didn’t look into it but it’s also crypto-related and it looks like it’s more ‘abstracted’ away as it marketed as a workbench to create strategies on top of.
So in my opinion, it’s more the idea to embrace the common core that will just contain the project structure, possibly a frontend (so also it will need to contain common simplified structures), probably a bunch of behaviors that will dictate what is expected from each element of the system like “DataStreamer” - it needs to have “switchOn”, “switchOff” methods (that is just an example).
I would be interested in planning this out but I need to admit that I’m starting new work on Monday and also record my YouTube videos so I’m a little bit low on time. I need to give a good look at all of your projects (the ones that are shared on GitHub) and compare - I would love to brainstorm and come up with something as I can see that we are all doing similar things and wasting time by reinventing the wheel.
I wanted to keep this short… failed again - sorry