Recommendations to parse Atom and RSS feeds?

Wigny · December 28, 2025, 10:27am

I was just playing with xmerl this week and sadly it parses XML generating atom keyed trees.

Yes, we have the option of using xmerl_sax_parser and handle the string parsing ourselves:

lib/xml/parser.ex

7a41eeed3


      
          
          def parse(xml) do
            case :xmerl_sax_parser.stream(xml, event_fun: &handle_parse_event/3, event_state: []) do
              {:ok, [element], _rest} ->
                element
          
              {:fatal_error, _parser_state, _reason, _location, _event_state} ->
                raise ArgumentError, "invalid XML document"
            end
          end
          
          defp handle_parse_event(
                 {:startElement, _uri, _local_name, {prefix, name}, attributes},
                 _location,
                 ancestor_elements
               ) do
            [{normalize_name(prefix, name), normalize_attributes(attributes), []} | ancestor_elements]
          end
          
          defp handle_parse_event(
                 {:endElement, _uri, _local_name, _qualified_name},

dimitarvp · December 28, 2025, 11:50am

Thank you for the grounding, I was unsure myself.

garrison · December 28, 2025, 1:27pm

Thanks for confirming. Note that there is some further guidance from the erlef for the sax parser.

derek-zhou · December 30, 2025, 7:37pm

CDATA is just string, which is handled in Floki’s built in parser here:

github.com/philss/floki

lib/floki/html/tokenizer.ex

ab1d36110


      
                 <<d, o, c, t, y, p, e, html::binary>>,
                 s
               )
               when d in [?D, ?d] and o in [?O, ?o] and c in [?C, ?c] and
                      t in [?T, ?t] and y in [?Y, ?y] and
                      p in [?P, ?p] and e in [?E, ?e] do
            doctype(html, s)
          end
          
          # TODO: fix the check for adjusted current node in HTML namespace
          defp markup_declaration_open(<<"[CDATA[", html::binary>>, s = %State{adjusted_current_node: n})
               when not is_nil(n) do
            cdata_section(html, s)
          end
          
          defp markup_declaration_open(html, s) do
            bogus_comment(html, %{
              s
              | token: %Comment{},
                errors: [{:parse_error, nil} | s.errors]
            })

Yes, it is common practice to embed HTML content in the feeds. I handle 2 most common cases:

directly insert the html DOM tree within the RSS item This is probably not 100% kosher XML but it is very easy to handle. I just take the HTML sub-node verbatim.
embed the html fragment as an encoded string. I just take the string, call Floki again on it to parse the html structure

I believe the above 2 cases handle the 99% feeds in the wild. They are embarrassingly easy to handle so I will skip posting my not very elegant code. If you see something more exotic than this, please give me a url, I’d like to play with it.