That is my feeling, either by side effect of it using older plugins or wp4 itself. There are open issues on the tw-jit talking about other build systems, so there might be some amount of friction in both directions.
There is a webpack5 PR/issue for Phoenix, I think the plan is to migrate eventually.
Though by looking onto this, manually migrating a project to another build system wouldn’t actually be very hard. Basically update :watchers in your dev.exs and then follow whatever your chosen build system asks you to do.
