![]() ![]() When the exploration work is finally done, and before moving on to the final export, I strongly recommend that you briefly check the status of the pages directly through the Screaming Frog interface: Our “SEO Pro Tip” of the day is therefore to always insert the sitemap before the crawl, as shown below: Consequently, the final export may not contain all the pages if the interlinking is not done correctly, the orphan pages will certainly be missing. It is very common to crawl a site that does not have its sitemap in the robots.txt, and sometimes, the robots have trouble finding this specific file. Unchecking the less relevant of them can save time in your export and your classification work. The different options and tabs allow you to select precisely the elements that will be explored by Screaming Frog robots (ex: images, PDFs, external links, etc.). The configuration tab is full of options that will allow you to make decisive adjustments, more specifically the Spider sub-menu option: However, we suggest fine-tuning your settings to ensure a successful crawl! Once the software is open, the process can be as simple as pasting the URL to crawl in the field located right of the logo, and clicking on start: In order to please those wishing to get to the heart of the matter, here is an overview of how an exploration can be done with our favorite tool.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |