What CMS Does WebTrackly Use? Our Stack & Performance Data

At WebTrackly, our foundational mission is to provide fresh, actionable domain data — whether it’s newly registered domains (NRD), TLD zone files, or technology-stack lists — as instant CSV downloads. Over the past 4 years, we've delivered over 18,000 unique data packages to SEO agencies, domain investors, and cold-email teams. Given the specialized nature of our product, the choice of a Content Management System (CMS) was never going to be off-the-shelf. So, what CMS does WebTrackly use?

The core WebTrackly platform runs on a custom-built Ruby on Rails application, meticulously engineered to handle our specific data delivery and e-commerce requirements. This isn't a WordPress site with a few plugins; it's a bespoke system optimized for the unique demands of processing and serving large, frequently updated domain datasets for over 1,500 TLDs.

Why Not WordPress? Our Data-Driven Decision

When we started WebTrackly in early 2019, the default temptation for any new web project is often WordPress. It's ubiquitous, affordable (hosting starts around $5/month), and boasts an ecosystem of 60,000+ plugins. However, our specific requirements quickly made it clear that a generic CMS would create more problems than it solved.

Scalability Challenges with Large Data Files

Our product isn't a simple blog post or static page; it's delivering gigabytes of CSV data files. A full .com zone file alone can be over 5GB. WordPress's architecture, while excellent for content, struggles with direct large file management and secure, authenticated delivery without significant, often clunky, plugin intervention. We needed a system that could reliably serve a 3GB NRD list to 50 concurrent users without grinding to a halt. Our custom Rails application, hosted on AWS S3 for file storage and CloudFront for delivery, handles this with ease, maintaining average download speeds of 90 MB/s for our European customers as of Q2 2024.

Custom Logic for Data Generation and Filtering

WebTrackly's value proposition hinges on more than just providing raw zone files. We offer newly registered domains (NRD) updated daily, which requires complex processing logic: fetching official zone files, integrating with DomenGood domain data, filtering out parked or PPC spam (which can be 40-60% of raw NRD feeds), and then compiling these into clean, exportable CSVs. This level of data transformation and filtering is simply not feasible or efficient within a WordPress environment without writing a massive, custom plugin that would effectively turn WordPress into something it's not. Our Rails backend performs these filtering operations in an average of 150 milliseconds per 100,000 domains processed.

Security and Performance Control

Running a system that processes sensitive payment information and delivers valuable data means security is paramount. A custom Rails application provides a much tighter security surface than a highly extensible CMS like WordPress, which is frequently targeted due to its widespread use (over 43% of the web). We have direct control over every dependency and security patch. From a performance perspective, our optimized Ruby on Rails stack, running on AWS EC2 instances with PostgreSQL, consistently delivers page load times under 300ms for core product pages, according to our internal PageSpeed Insights reports from May 2024.

Our Custom Rails Architecture: Components and Integrations

Our choice to build a custom CMS with Ruby on Rails was driven by the need for granular control over every aspect of our platform, from data ingestion to user experience. The architecture comprises several key components:

Backend: Ruby on Rails & PostgreSQL

The heart of WebTrackly is a Ruby on Rails 7 application. Rails provides a robust framework for rapid development, API creation, and database interactions. We use PostgreSQL 14 as our primary database, which stores metadata about our 1,500+ TLDs, user accounts, orders, and the intricate logic for managing our data packages. PostgreSQL’s JSONB field type is particularly useful for storing flexible data like WHOIS record attributes, allowing us to evolve our data schemas without extensive migrations. Our PostgreSQL instance processes an average of 2,500 queries per second during peak NRD update cycles.

Data Ingestion & Processing Pipeline

This is where the real "secret sauce" lies. Our system pulls raw zone files directly from official registries or aggregates data via DomenGood domain data. For newly registered domains, this process runs daily, typically completing all zone file processing and NRD compilation by 03:00 UTC. The pipeline uses custom Ruby scripts and a series of background jobs (powered by Sidekiq) to:

• Download zone files for over 1,500+ TLDs.
• Parse millions of domain records.
• Identify newly registered domains from the previous day's snapshot.
• Filter out known problematic domains (e.g., those associated with parking pages, PPC spam, or specific registrars/nameservers known for junk registrations). Our internal data shows filtering by registrar and nameserver removes most junk before outreach.
• Generate specific data packages based on user requests (e.g., only domains registered in the last 7 days, or domains using a specific technology).

This pipeline processes approximately 15-20 million domain records daily, taking around 2.5 hours from start to finish for all major gTLDs and available ccTLDs.

Frontend: StimulusJS & Tailwind CSS

For a snappy user interface without the heavy JavaScript frameworks, we opted for StimulusJS, a modest JavaScript framework that integrates seamlessly with Rails. It allows us to add interactivity, such as dynamic filtering on our product pages or instant search, without the complexity of React or Vue. The styling is handled by Tailwind CSS, which accelerates UI development and ensures a consistent design language across the site. This combination results in a lean frontend that loads quickly and provides a smooth user experience, typically delivering a fully interactive page to the client in under 500ms.

Payment Gateway & File Delivery

Stripe handles all our payment processing, offering secure and compliant transactions. Once an order is placed and confirmed, our custom Rails logic instantly generates a unique, time-limited download link. The actual data files are stored securely on AWS S3 buckets and served via AWS CloudFront, a global CDN. This setup ensures that our CSV downloads are delivered rapidly, regardless of the user's geographical location. Average download initiation latency is under 100ms globally.

What We Got Wrong / What Surprised Us

Building a custom platform always comes with its share of unexpected challenges and revelations. One of our most significant misjudgments early on was underestimating the sheer difficulty of sourcing comprehensive ccTLD zone files. We initially assumed that most country code top-level domains would follow the gTLD model of publicly publishing their zone files. Our experience proved this assumption incorrect.

Many ccTLD registries never publish a public zone file. For some, like .de or .uk, official zone files are available under specific agreements, but for hundreds of smaller ccTLDs, the data is simply not publicly accessible in bulk. This meant our initial automated data collection strategies had to be significantly revised. We discovered that for a substantial portion of the 1,500+ TLDs we cover, aggregated domain data from sources like DomenGood was the only viable route to achieve comprehensive coverage. This required a shift in our data ingestion pipeline and a deeper integration with third-party domain data providers, delaying our ccTLD coverage expansion by about 3 months in late 2021.

Another surprising observation, especially for our cold email and lead-gen clients, was the critical importance of MX validation. We always knew newly registered domain feeds were noisy – our data consistently shows 40-60% are parked or PPC spam. However, even after our stringent filtering, a domain list is only as good as its MX validation the same week you send. We observed that even a perfectly clean list can see deliverability rates drop by 10-15% if the MX records aren't re-validated immediately prior to sending, due to rapid changes in domain parking, expiration, or email provider configurations.

Practical Takeaways for Your Own Projects

1. Evaluate CMS Needs Against Core Business Logic (1-2 Days, Medium Difficulty): Before defaulting to WordPress, list out your critical business processes. If they involve complex data manipulation, custom integrations, or high-volume specialized content delivery, consider a custom framework. For WebTrackly, delivering 1,500+ daily-updated CSVs meant WordPress was a non-starter.
2. Prioritize Freshness Over Volume for NRD (Ongoing, Low Difficulty): For cold outreach, a smaller, highly fresh list of newly registered domains (our AWS domains list guide touches on this) will outperform a massive, stale dump from a marketplace. Our data consistently shows that scraping your own domain lists, or sourcing from providers focused on daily updates, beats buying stale marketplace dumps. Freshness matters more than volume for NRD outreach.
3. Implement Robust Data Filtering Early (3-4 Weeks, High Difficulty): If you’re working with raw domain data, assume it’s dirty. Build filtering mechanisms from day one. Our experience shows 40-60% of raw NRDs are junk. Filtering by registrar, nameserver, and even basic WHOIS patterns is crucial. This will save countless hours in outreach clean-up later.
4. Integrate MX Validation Into Your Workflow (1 Week, Medium Difficulty): If your primary use case for domain lists is email outreach, bake MX record validation into your pre-send process. Tools like NeverBounce or ZeroBounce (costing around $0.008-$0.01/email as of 2024) can drastically improve deliverability. Our teams re-validate lists within 48 hours of sending for optimal results.
5. Don't Underestimate ccTLD Data Acquisition (2-4 Months, High Difficulty): If you plan to work with country code TLDs, be prepared for significant data acquisition challenges. Many ccTLD registries do not provide public zone files. Budget time and resources for alternative data aggregation methods or partnerships. Our journey into ccTLD SEO taught us this valuable lesson.

Frequently Asked Questions

Does WebTrackly use a headless CMS?

While our Ruby on Rails application serves content via traditional server-side rendering and StimulusJS for interactivity, its robust API layer means it could easily function as a headless CMS if we were to build a separate frontend application. For now, the tight integration of our backend and frontend within the Rails framework offers optimal performance and development speed for our specific needs, without the overhead of a separate API layer for every request.

How does WebTrackly ensure the freshness of its domain data?

WebTrackly's custom system is designed for daily updates. Our automated pipeline fetches official zone files and aggregates data from DomenGood domain data every 24 hours for newly registered domains (NRD). For specific zone files, the update frequency is per-zone, meaning we update them as soon as the registries publish new versions. This daily cycle ensures our NRD lists reflect registrations from the previous day, making them some of the freshest available, typically ready by 03:00 UTC.

Can I integrate WebTrackly's domain data with my own applications?

Currently, WebTrackly provides data as instant CSV downloads directly from our platform. While we don't offer a public API for programmatic access at this time, the CSV format is universally compatible and easily imported into databases, CRMs, or custom scripts. For very large or custom data requirements, we can discuss tailored data feeds, but our core offering focuses on one-time, ready-to-use downloads for maximum flexibility for our users.

What makes WebTrackly different from other domain list providers?

Our key differentiator is a focus on freshness, filtering, and instant, no-subscription access. We don't just dump raw data; our system filters out 40-60% of parked/PPC spam from NRDs using specific registrar and nameserver criteria, delivering cleaner, more actionable lists. We cover over 1,500 TLDs and provide immediate CSV downloads after checkout, without requiring recurring subscriptions. This contrasts with many providers who offer stale data or require monthly commitments for data you might only need once.