Screaming Frog has released version 2.30 of their SEO Spider. Big news in the SEO world, happy users, and all that jazz.
I now find the competitor's product much more appealing. It can't be any different:
let's have a look at their release main features...
1) Pixel Width Calculated For Page Titles & Meta Descriptions
Completely copied from Visual SEO Studio, which sports the feature since almost two years (the first public beta of VSS was released 30 Sept 2012, and private betas appeared much before).
The feature as available since the earliest versions of Visual SEO Studio
Excerpt from Screaming Frog web site:
Back in 2012 Google changed the way they display snippets in their search results. Historically this was simply a character limit of around 70 characters for page titles and 156 characters for meta descriptions. However, Google switched this to determine the actual pixel width...
Let me put it straight:
Char-based title length in Google SERP has been one of the biggest, widely spread myths in SEO industry since years, unconsciously perpetrated by several SEO software vendors and SEO influencers, mutually endorsing and reinforcing their belief. Bona-fida granted, mind you.
I wrote the first implementation of Visual SEO Studio SERP snippet emulator at the end of September 2011, and I assure you Google SERP back at the time already was pixel-based.
I strongly suspect SERP limit has never really been chars-based. Google never talked about a chars based limit, not even in its GWT "HTML Improvements" (previously named "HTML Suggestions") where it has a "Short title tags" report since ever.
I personally always gave for granted it was pixel-based. Not because I'm smarter then the others, just I come from a programming background, where it is well understood that if you need to fit a text in a box using a non-monospaced font you have to measure its size using low level APIs offered by the operating system GDI. Browsers in the end are GUI apps like the others, they are bound to the same rules.
It's pixel-based. Period. SEOmofo is credited as the first to blog about it in 2012, and already was a known fact for many. Nevertheless, the thing went mostly unnoticed. The big players were still discussing whether the limit was 65/70/71/pick-one.
Suddenly Search Engine Roundtable came up on Jan 2013 "discovering" it was pixel-based (and giving a wrong pixel width).
The industry started to realize they had been given the wrong tools and information.
One year later Screaming Frog downloads Visual SEO Studio free beta, realize it kicks asses and poses a threat to their monopoly, and understand they have to catch-up with VSS powerful features. Two (and not the only ones) of those are SERP snippets and pixel-based reporting of truncated title and meta-descriptions.
Nothing wrong with it, you have to protect your own business, improve your product to keep up with competitors. I firmly believe in free market.
It would have been fairer had they said their paying customers:
"We have been wrong, sorry guys for making you waste so much time pursuing a magic chars number just because [we didn't know it | it was more convenient to us | we were not able to calculate a pixel-based width | whatever other reason]".
It's probably easier pretending Google switched from chars to pixel on 2012 (btw, it's two long years anyway) when SEOmofo blogged about pixel based limits ...and let the market think you are the great innovator you are not.
2) SERP Snippet Tool
Completely copied from Visual SEO Studio, which sports the feature since almost two years.
SERP snippet emulator in Visual SEO Studio, with March 2013 SERP layout
Granted, in SF they added a nice grid which looks better than VSS's one. Too bad their algo to truncate meta-descriptions is plain wrong.
There have been web-based SERP emulator around the Internet since the SERP exists, but Visual SEO Studio has been the first on-premises SEO software not only to provide SERP snippets for every crawled page, but also to let you find the truncated titles and descriptions among thousands. This is innovation. Two years earlier, when someone was too busy selling char based length as the Saint Graal leveraging their social media power and influence.
3) Configurable Preferences
Copied from Visual SEO Studio 0.8.13, released Feb 2014. Heck, I could even have released earlier, the configurable engine was there since the beginning.
HTML and URL Suggestions criteria in Visual SEO Studio
4) Canonicals Are Now Crawled
Copied from Visual SEO Studio 0.8.11, released Dec 2013
5) More In-Depth Crawl Limits
Hey, this one I don't have. Not that it is any useful in my opinion.
Dudes, you sure you didn't misunderstand my URL Suggestions feature?
6) Microsoft Excel Support
Copied from Visual SEO Studio, it's there since the first public beta (Sept 2012).
Granted, adding support to Excel is expected from a software, but if I were a paying SF user I would annoyed for them not caring to give such a trivial help and forcing their users to pass through CVS until the new guy come to town to steal the girls.
Other Smaller Updates
I leave it as an exercise to the user: spot the ones inspired by VSS.
To summarize:
The entire SF 2.30 release is based on the work of copying Visual SEO Studio ideas.
Again: a company has the right to innovate its own software to protect its business, and also an obligation toward its paying customers.
They did't copy a source code, they "just" copied ideas and market research.
Nevertheless, shamelessly copying a competitor's product I do not call it "innovating", I do not call it "ethical".
They are a Goliath with maybe 100.000 users (free users included), many employees, money and resources, and a lot of media power. They just need to whisper they released a cool new feature crafting words to let people think they invented it, and hundreds retweet, mention, discuss, and credit them. It stinks.
They could devote all those resource to do their own strategic marketing homework.
Cards on the Table
Yes Screaming Frog, you copied my ideas shamelessly, no point denying it.
You also carefully avoided to at least credit VSS original ideas and did your best to appear as if you were such innovators.
Does the word "decency" bear any meaning in your book?
Dan, we both do know you regularly use Visual SEO Studio, you have it installed on your PC and a bunch of others.
Yes, I know since the first download, it's not rocket science segmenting Analytics data: download events, long visits to rel.notes, etc...
an example of what you can find segmenting your events
And yes, I can see you,
Shot taken on 21 March, 2014
check the End Users License Agreement you accepted upon installing the program, you'll see what I can track. You are well aware of the EULA, so much you also bothered to add one for your software as well after seeing mine.
And by the way, check also the part about "reverse engineering", you can never know...
Of course you are free to download Visual SEO Studio, for free, and use it for SEO website auditing for your agency work.
Of course that's what you are doing. I'm glad you love Visual SEO Studio so much.
A business perspective
From a business perspective I understand SF's doing: VSS poses a thread and they want to kill the beast before it grows big.
Let's face it. I am small. I'm not a company, I'm a solo programmer working on his toy-project during his spare time, a wannabe entrepreneur. I do not earn a dime from Visual SEO Studio today, I invest my pocket money on it and give the program for free - unlimited - while it's in beta stage.
But, I have managed to build, market and promote a software thousand of users use, translated already in six languages.
Not too bad. Imagine what I could do tomorrow...
I've been asked how I feel about having been copied...
I've been thinking about it quite a lot.
Screaming Frog choice to copy everything I do is not a personal attack, is a business attack. I have to keep it in mind when answering. I slept it over, and considered the good sides, so:
- Fine: with all this echo hopefully now the 60-70 chars mantra is going to die, and I've been fighting it for a long time.
- Fine: I've shaken the industry, forcing a sleeping 800 lb gorilla - an established business with at least 15-20 employees, the de-facto monopoly of desktop SEO spiders - to rush and thus making the whole SEO industry benefit. Not the first time someone takes inspiration from Visual SEO Studio, surely not the last.
Imitation is the highest form of flattery. - Fine: my ideas were good and ahead of time, and many other people would have appreciate them had they known about it (well, maybe the market wasn't ready). You don't copy a bad idea unless you are a fool.
Did the whole thing affect me personally?
Of course. I cannot deny it. I'm working on the product since four years. Two years ago I made a choice to get seriously involved with its development, and I identified myself with Visual SEO Studio. I work at my daily job, and then I return back home to work on VSS. I work on it usually until 2 a.m. and wake up at 6 a.m. (no alarm is needed) and work other two hours. Then I go to work again. I sleep 3-4 hours per night, week-ends too. I occasionally go jogging or to the gym, a walk or some social event, but force myself to work on VSS everyday at least a few hours. I limit my holidays to the strict necessary, and for the rest I work on the product. I gave up sleep, spare time, social life, fiancées, expenses, everything to work on it. I eat, breath, live VSS.
Of course it affected me on a personal level.
I'm a firm believer in free market, but it stinks the fact they copied it with no shame. This is not innovating your product, it's just copying everything Federico (ideas, marketing research, technical research, everything) does just because he is a little fish and very few know him yet. "Who knows that little Italian who can barely express himself in English? Go ahead, no one will notice."
It stinks, but I'm dealing with it. I understand business rules. And business sometimes is a war, a war the giant SF started against the little guy. Who is next in their plans?
A message to Screaming Frog users
You are the real winners of the story. I am sincerely happy you eventually had something more. You had a product improved - of course, it aims to look like VSS! - and if you'll read this message you'll also learn there is another product which is a great product too.
Better, worse? Different. Differently from SF, I strive to keep it different and innovate. Visual SEO Studio has its weaknesses - at the moment - and has its points of strength.
By the way, did you notice someone didn't care for months to update with features worth mentioning a product you are paying, until forced to do it by an outsider who posed a serious threat to their golden eggs goose?
Ever noticed someone carried on with the 60-70 chars mantra as long as their user base were ready to pay for it? And reinforced that false myth making you waste so much time?
And those all back and forth from CVS format and Excel, reformatting columns, how many hours did you spend just because of lazy programmers?
Oh, by the way, Visual SEO Studio is completely Free and Unlimited while in Beta.
It's improving day by day, releasing a new version every month. Worth checking it.
Hey, Visual SEO Studio is so good that even Screaming Frog team loves it, it uses it on a regular basis!
A message to Visual SEO Studio users
Dear all, thank you for all the appreciation you have shown me so far.
I know many of you are also Screaming Frog satisfied users. It's OK to me.
I'm certainly not going to ask to take sides on this.
I just want to let you know I will continue to improve the product with the same passion as before. It has been a great ride so far, it will be even better in the future.
Time to go ahead
"Kill the beast when it's young", they say (funny, I often self define myself as an "office beast").
Too bad this beast does not agree to be killed. Too stubborn.
Sorry, I'm not going to let a rotten frog kill my dream.
I always preferred playing David's role
Time to go ahead
Future strategic marketing goals of Visual SEO Studio:
- continue to improve the product through careful research, innovation, and listening to the users.
- promote a little better all the hidden gems the product has, and perhaps be a little less open about my future plans.
- of course, differently from someone else, I try my best to differentiate the product and innovate with my own research. At this point, with someone copying everything good I do, I'll need more time to differentiate, so I guess the Free Beta phase will have to last some little longer.
But this is good news for the users, so we all are happy about it. Isn't it?
Future strategic marketing goals of Screaming Frog:
- continue to copy everything Visual SEO Studio does.
My bet for the first one in queue: Crawl Ajax Sites
By the way, if you wish any new feature to be added to Screaming Frog SEO Spider, just ask Fred to add it to VSS, there are good chances it will be copied in SF in no time.
UPDATE - April 1, 2014 (not an April Fool's):
I've been contacted by Dan of Screaming Frog a few days after publishing this article (couldn't update is sooner). He told me how they took to heart my post and felt it unfair. He contested my points and offered a different perspective.
I'm tired of this story and feel it's not worth debating it any more. It's not something users would care: they care more about having good features, and that's what I want to offer them.
I have to add that Dan's messages have been polite and friendly. After a few e-mail bounces he offered a friendly chat over a beer or three, and I for sure never refused a friendly chat over a beer or four. Accepted.