dogweather
The idea is: Decoupled IndexNow URL submissions, using a website’s XML Sitemaps as input.
Here’s a walkthrough. It works, but there’s not a ton to see yet: https://www.youtube.com/watch?v=DJSjUHUb3Rs
I started making this last week: I have thousands of legal content web pages. These get updated weekly, but only a few hundred at a time. Since I have so many pages, I have the problem of telling search engines efficiently about the updates. I need to be very careful about my “crawl budget” (I have too many pages for crawlers to re-crawl my entire site at once.)
Google has stopped allowing pings to notify them about XML Sitemap updates. But I have a ton of investment in getting sitemaps working. IndexNow is a newer alternative. It’s push, rather than pull.
I was in the middle of custom coding IndexNow support when I realized I could cut down on the work by just using the sitemaps I’m already generating. And then I realized that this could be a standalone app. A week later, here we are.
I’m looking for beta testers, too. DM me if you’re interested.
Thanks for the feedback!
Where Next?
Trending in Discussions
Other Trending Topics
Latest Phoenix Threads
Latest Oban Threads
Latest on Elixir Forum
Sponsor Spotlight
We build reliable cloud platforms for business-critical systems.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #ai
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #elixirconf-eu
- #metaprogramming
- #hex
Our Sponsors
Build Elixir applications with speed and confidence.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Practical resources that improve the lives of professional developers.
Master modern tools, write cleaner code, and tackle real-world challenges.
Producing high quality Elixir screencasts since 2017.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 4- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
cgraham
Cool! Is this open-source or will be free? I was just about to vibe code an open-source version of this but if you are doing that, happy to participate in your beta!
dogweather
Hi, here’s my plan — free for basically, 1 or two updated pages per week. Like, a small blog. I really need to earn some more income - I’m making a go at being fully self employed.
Notes
I just finished Stripe integration and we’ll see…I might get some paying customers.
That all said, I’m open to collaboration and the “Slicing Pie” method of fair equity splits if you’re interested.
cgraham
Nice!! Roll with it! I think it might have legs! In between our conversations, I rolled my own (vibe coded) for my own site(s) but I can see this having legs with a lot of people - especially new vibe coders!
Rooting for you for this to take off!
dogweather
Yeah, I realized that a pretty simple script could do this - just pull down your xml sitemaps, loop through the entries, and submit the ones with
lastmodlater than the last time the script ran. Like, literally a 5-line Ruby script.And of course, I underestimated how much more work it is to make a SaaS app that other people would want to pay for. The biggest additions I’m adding are:
lastmods correctly — or that I’m publishing on the schedule I think I am. In development I’m working on a histogram for each website showing URLs in buckets by age. It’s already helped highlight issues I didn’t know I have: I thought I’d have URLs many years old, but it shows them all as < 30 days, which is not what I expected.