Backlinks are essential elements in achieving website keyword rankings, increasing traffic (referrals, organic, or direct), and diversifying anchor text. It’s also the most used technique in the world of search engine optimisation (SEO). A page that has lots of diversified backlinks tend to rank higher in all major search engines including, Google, Yahoo and Bing.

What are internal links?

Internal links are hyperlinks on web pages that can be found either on your site, other pages, or other domains. Depending on the target destination, your links can either be “external” or “internal” in terms of backlink sources. The Hyperlinks on the other hand, can either be found in the main navigation, main body, site-wide links, or in the footer links.

With the help of the right tools, you can deeply crawl any website’s internal or external links. Links are essential in obtaining traffic, decreasing bounce rate, and enhancing user usability experience (depending on how it was placed). You can simply download free tools online, install (Windows, Mac, Linux) and upload a list of urls.

__________________________________________________________________________________________________________

5 most used tools and their functions

1. Screaming Frog

Screaming Frog software is one of the best SEO tools. It is basically a web crawler software where  you can crawl and view the entire links (free trial) of a particular website you want to check. Any website can be crawled except if it has no index, no follow attributes, or the site is blocked by robots.txt. But you can ignore the robots.txt by going to configuration > Spider > find and uncheck ignore robots.txt and try running it again (you will be needing the paid version for you to uncheck the feature).

The software is available to Windows, Mac, and Linux for download. It has a minimum specification to be able to run Java 8 update 66 above and at least 512MB of RAM. Your memory allocation depends on how many URLs you can crawl. For a paid version, you can check their pricing here and download the software here.

Screaming Frog untold Features

1. How to increase Screaming Frog memory for Windows Operating System

Most users experience running out of RAM when trying to crawl a huge list of sites. You will receive pop-up messages when it occurs. To avoid this error, simply upgrade your machine to 64 bit and install the 64-bit version of Java to maximise the memory allocation. To verify and check your memory allocation visit HELP > Debug and look at the memory line of the software.

2. Unlimited List Mode for free trial version

For the trial version, the list mode is unlimited. You can directly upload up to 10K URLs and crawl them.  However, if you avail their paid subscription, Screaming Frog will extend the features of the crawling rates.

3. Editing Meta Tags on SERP snippet Emulator

One of the features that a free trial offers is that you can edit the Meta tags to fit the recommended length of characters.

2. Xenu’s Link Sleuth

The software won’t run on Windows 3.11. Win32, the developer, said that he won’t make a Java, MacOS, Linux etc. but a few curious developers tried to run it faultlessly using Red Hat 8, MacOS under Crossover, and OS X via winebottler.

To learn more, you can visit their website here.

Once it’s done generating data, it will automatically generate a report on hierarchies of broken links. It provides a detailed report that you can either save via FTP in your database or HTML format based on your website statistics.

3. Google Search Console

4. Sitemap (www.domain.com/sitemap.xml)

Let’s say Page “A” is the sitemap in the above illustration. The search engine bot crawler can easily visit the internal links if you have proper sitemap set up. You can generate it through a WordPress plug-in or by using a sitemap generator tool online. The crawling rate will depend on your sitemap indexing frequency set-up. You can set daily, weekly or quarterly depending on the frequency of your content’s changes. For example, if you updated your web pages daily you can also set up a daily indexing frequency on your sitemap settings.

5. Scrapebox Bulk Backlinks Checker

Scrapebox backlinks checker can check a large number of URLs and can scrape up to 1000 backlinks for each URL. The data is provided by mozscape API (MOZ) which entry level cost starts from $250 per month. The entry level scraped 120,000 rows per month and $20 per additional 10,000 rows. It includes URL metrics, links, top pages, anchor text and has a rate limit of 200 requests per second.

_______________________________________________

How to check your competitor’s backlinks

These insights will give you a few basic ideas on how to spy on your competitor’s backlinks. Before you begin digging up, list your top 5 competitors. You just need to use 3 tools to check your competitors’ backlinks. The good thing about these tools is that they crawl all links that are indexed within your competitors’ web pages.

SEO Spy-Glass

• The Summary menu will give you a list of backlinks e.g. do follow, no follow, and C-Blocks (the number of Ip’s backlinks that were built).
• The Backlinks tab will provide you all the lists of their backlinks sources, linked page, and lists of anchor text.
• The Linking Domains tab gives you the list of IP’s, Domain Countries, number of backlinks and linking domain.
• The Anchor’s and Pages tab will give you a list of the variation keywords/anchor text they are targeting.

Once you have gathered all the things you need, save all anchor texts and the list of URL pages to your personal excel spreadsheets and start digging up more information through the next process.

Google Search

You can search numbers of internal links indexed pages of the website using Google search by just typing site:www.domain.com.

SEMRUSH

After you listed and organised everything in your spreadsheet, the next step is to conceptualise how they got that number of traffic based on the data you have collected. From that standpoint, build your ideas and marketing strategy to surpass them gradually. Record your progress and compare it on a weekly basis.

Semrush is one of the best tool I can recommend. One thing that I didn’t like though is their keyword research feature. I didn’t find it accurate, however if you’re looking for a wide variation of keywords it might be a good source. But, what I like is their backlinks checker, website statics and varieties of local Google search results.

List of free backlink checker

• Backlink Watch (Free but not so accurate)
• Google Webmaster tools
• Bing Webmaster Tools
• Yandex Webmaster
• Backlink Test
• Sheer Explorer
• Rank Signal
• OpenLinkProfiler
• MoonSearch
• Monitor Backlinks
• LinkPad
• Linkody
• Megaindex  

List of Paid/Subscribe backlink checkers

• Ahrefs
• Buzzsumo
• MOZ – Open Site Explorer
• Link Diagnosis
• SemRush Pro
• SEO Spyglass
• MOZ
• Majestic
• Link Detox
• CognitiveSEO
• Kerboo
• Link Research Tools
• Botify

_________________________________________________

Recommendations

I’ve posted and asked SEO professionals on Linkedin Search Engine Land group about what  tools they can recommend. I got lots of responses but Jim Hodson’s remark stood out among the rest. Let me know what tools you’re using and let me know your SEO stories. You can write them in the comment section below.