Email Marketing Publisher Data Management

Industry: Affiliate Marketing

Challenge: The customer, a group of affiliate marketing publishers, faced the task of managing vast amounts of email marketing data, including email addresses, contact details, and the digital footprints of their targeted customers.

Acquiring, storing, and leveraging this data effectively can be costly and resource-intensive, making it challenging to generate targeted email lists, ensure compliance with email regulations, and optimize campaign performance. Moreover, publishers often struggle to accurately attribute revenue to specific email addresses and data providers, hindering their ability to optimize data acquisition.

Solution

Our robust MongoDB-based email marketing data management infrastructure has revolutionized the way publishers approach email marketing in the affiliate space. By centralizing data acquisition efforts and providing a robust, scalable solution for storing and managing email-related data, we have empowered publishers to:

  1. Generate highly targeted email lists based on multiple criteria, such as opt-in status, engagement, and verticals of interest.
  2. Ensure compliance by easily suppressing email addresses associated with bounces, unsubscribes, spam traps, and other red flags.
  3. Accurately attribute revenue to individual email addresses and data providers, enabling data-driven decisions for list acquisition and campaign optimization.
  4. Monitor the performance of various lists and campaigns through intuitive dashboards and reporting tools.

To achieve these benefits, we implemented a master-slave MongoDB cluster architecture, with a duplication on another set of Linux servers for redundancy and high availability. We store email addresses as MD5 hashes, using binary format to reduce data size and improve query performance. By organizing related documents and lists under each unique email hash, we can efficiently track all associated actions, such as opt-ins, unsubscribes, clicks, conversions, and more.

Diagram showing a MongoDB sharded cluster architecture with a backup

Our solution leverages the affiliate marketing platforms’ link parameterization capabilities, such as those in Everflow and Hitpath. By adding salted email hashes to the links, we can attribute clicks and conversions back to specific individuals while maintaining data privacy. Each publisher has a unique hashing method, ensuring a fair and competitive environment.

Diagram illustrating the email address hashing process using salt and MD5

To further enhance the value of our solution, we developed Python ETL processes that aggregate the MongoDB data into a structured MariaDB database for reporting and analysis. This structured data is then connected to a Domo BI platform, enabling the creation of interactive dashboards that display the ROI of acquired data by list, provider, and source, as well as the performance of various lists and campaigns.

Dashboard showing key metrics for data acquisition performance

We have also built a web application that empowers publishers to generate and extract targeted lists on their own, putting the power of data-driven email marketing at their fingertips.

Result

Since implementing our MongoDB-based data management solution, our client has seen remarkable improvements in their email marketing efforts and overall performance:

  1. Cost-effective data acquisition: By leveraging our ROI-driven process for evaluating and acquiring email lists, our client has successfully acquired over 400 million opt-ins while reducing their acquisition costs by more than $100,000. This has allowed them to scale their email marketing efforts while maintaining a strong focus on profitability.

  2. Rapid list segmentation and generation: Our solution has dramatically accelerated the process of segmenting and generating targeted email lists. Publishers can now quickly create lists based on multiple criteria, such as opt-in status, engagement, verticals of interest, and more. This has enabled them to deliver highly personalized and relevant content to their subscribers, resulting in improved engagement and conversion rates.

  3. Streamlined compliance management: Our infrastructure provides a centralized and efficient way to manage compliance-related data, such as opt-ins, unsubscribes, bounces, and spam complaints. This has empowered our client to quickly demonstrate proof of opt-in when required by advertisers or in response to compliance concerns. By maintaining a high level of compliance, our client has strengthened their relationships with advertisers and protected their reputation in the industry.

  4. Increased operational efficiency: Our web application for list generation and extraction has streamlined the email marketing workflow for publishers. By automating key processes and providing an intuitive interface for managing email lists, we have significantly reduced the time and effort required to execute successful email marketing campaigns. This has allowed our client to focus on strategic initiatives and growth opportunities.

Technologies and Tools

  • MongoDB
  • Linux servers
  • Python
  • MariaDB
  • Domo BI platform
  • Affiliate marketing platforms (Everflow, Hitpath)
  • Custom web application (Django)