A join is required to be able to refer to the CTE. For example:

Conversation
|> with_cte("conversation_query", as: ^conversations_query)
|> join(:inner, [c], "conversation_query", on: [id: c.id], as: :msg)
|> order_by([c], c.updated_at)
|> select([c, msg: m], %{c | last_message: m})
# The preload should be outside.
|> preload(creator: :avatar, participants: :avatar)

But this may not be the complete or right solution. The first thing to keep in mind is that distinct will pick an unpredictable row from the duplicates, unless you tell it to order by a particular column. See DISTINCT Clause according to PSQL.

Using the CTE may not even be necessary. A subquery in a join might solve it equally well. For example:

latest_message_query =
  Conversation
  |> join(:inner, [c], Message, on: [conversation_id: c.id], as: :m)
  |> distinct(:id)
  # Indicate that the row with the latest message among the duplicate rows should be picked.
  |> order_by([c, m: m], [c.id, desc: m.inserted_at])
  |> select([m: m], m)

Conversation
|> join(:inner, [c], Participation, on: [conversation_id: c.id, user_id: ^visitor_id])
|> join(:left, [c], subquery(latest_message_query), on: [conversation_id: c.id], as: :m)
|> order_by(:updated_at)
|> select([c, m: m], %{c | last_message: m})
|> preload(creator: :avatar, participants: :avatar)

I made some assumptions and tried to best understand what you’re trying to achieve. I hope this works or at least shows you a path to continue on.