How does Article Forge content compare to human-written content? A Case Study

This test was last run on May 11th, 2022.

In this case study, we test the quality of Article Forge content by asking humans to compare articles created by Article Forge with articles written by real freelance writers.

Given that there is a large range in the quality and price of human-written content, we created a 5-star scoring system. In this scoring system, 1-star is the lowest quality, cheapest, human-written content and 5-star content is equivalent to an article published in the New York Times, or similar.

Our goal is to progressively benchmark against each content level as Article Forge improves.

The last time we ran this test, Article Forge beat 1-star content written by freelancers. So, with our most recent update, we tested Article Forge against 1-star and 2-star human-written content.

Methodology

The Study

We paid real people to read hundreds of pairs of ~750 word articles - one written by Article Forge and one written by a freelancer. The readers were not told where the articles came from and were simply asked to choose which they thought was better based on: overall quality, readability, general flow, syntax, accuracy, usefulness, and grammar.

The Articles

The article topics were selected by randomly sampling over 2 million user keywords to create a diverse dataset across many industries and niches. The topics range from “the benefits of planting trees” to “blockchain in the automotive industry” and cover both short tail (broad) and long tail (specific) keywords.

We then generated an article for each topic with Article Forge and purchased an article about each topic from freelance writers. We paid an average of $5.90 per 1-star article ($0.78 per 100 words) and an average of $8.35 per 2-star article ($1.11 per 100 words).

Results

When comparing Article Forge articles directly to 1-star quality human-written articles, readers thought Article Forge articles were better 81.40% more often than the human-written articles. The results were statistically significant with a p-value of 0.0000000007.

With Article Forge winning handily over 1-star content, we moved on to testing against 2-star content:

When comparing Article Forge content directly to 2-star quality human-written articles, readers thought Article Forge content was better 3.09% more often. With a p-value of 0.783, this results in a null hypothesis, meaning neither side was better than the other beyond reasonable doubt.

Takeaways

Because the results indicate that readers clearly prefer Article Forge content over 1-star content, we can conclude that Article Forge creates higher quality content than 1-star level human-written content.

When looking at the Article Forge vs 2-star article comparison, we can see that readers thought Article Forge content was about the same, if not slightly better, quality than 2 star content.

While the purpose of this case study is to benchmark Article Forge’s content quality against human writers, there are two more factors to consider: price and turnaround time.

1. The 1-star articles we purchased were approximately $5.90 each ($0.78 per 100 words) and the 2-star articles were approximately $8.35 each ($1.11 per 100 words). With Article Forge, you can create 250,000 words for $57 per month, which works out to $0.02 per 100 words. For anyone who has a strong content demand, Article Forge is orders of magnitude cheaper while offering higher quality deliverables.
2. In general, turnaround time for the articles written by freelancers ranged anywhere from a few hours to over a day, while Article Forge takes about 60 seconds to generate each full article.

Therefore, when taking into account quality, price, and turnaround time, Article Forge clearly outperforms 1-star content written by humans and is a significantly better deal than purchasing 2-star content.