Author: Ryan Walsh
Date: 20/01/2026
Subject: Technical SEO for business in 2026
Read time is: 10 minutes

In this article we will cover:

• Website load speeds and web core vitals
• Broken links and why they need to be fixed
• U.X factors
• Redirects
• Robot.txt
• XML file
• Ensuring that a website can get crawled and indexed
• What is Googlebot?
• Why business need to conserve their crawl budget

 

Introduction

In search engine optimisation or SEO for short, we often as a leading agency get caught up in daily conversations about content marketing and link building. These two topics pretty much take up the majority of our time when liaising with clients.

Well, that’s with good reason, you might say, as these are the “pillars” or the “bedrock” of SEO.

Which is true; however, what’s often talked about far less is technical SEO.

However technical SEO and getting it right, well, that is essentials, its what wheels are to car, without them in place things are not going to go anywhere fast.

There’s a simple reason for this: talking about structured data, 301 redirects, and how to maximise your business’s crawl budget might actually make the person you’re talking to start yawning excessively.

This is why often technical SEO is not given much lime-light on an agencies blog. As it could even be termed as rather dull topic of conversation, its also something that businesses do not actively discuss with their agency neither. However, its about time that technical SEO is given the same amount of air-time if you like, in terms of analysing and discussing to make sure that you get this aspect right.

The technical part is therefore a key aspect the agency must get right, and thus, even if this is the more tedious side of an SEO consultant’s role, well, they must still roll up their sleeves and get the job done.

Before you start to jump in head first and start to “have a go” at improving your technical SEO you need a robust knowledge of the following:

 

Crawling and indexing- as well as what Googlebot is

• Redirects- what are they, why do they matter? 301 and 302 redirects
• Why you should set up Google Analytics and Google Search Console
• What is structured data, and how can schema be used to improve your local SEO
• Who are Schema.org
• What are canonical tags
• Is it still essential to keep a sitemap up to date?
• Why you might want to use Robot.txt to help preserve some of your crawl budget
• Is the speed of my company website critical?

This all sounds complex and perplexing, can you break technical SEO into more manageable parts?

Okay, so not to baffle you with science, the main parts of technical search engine optimisation can be put neatly into these broad categories- they are:

• Crawling and indexing
• User experience (U.X for short)
• Making it super clear what you are selling through structured data
• Good housekeeping in terms of sorting out aspects such as broken links.

 

Crawling and indexing (Googlebot)

So, Googlebot is not the only bot in town; Bing has its own, and so do LLM scrapers such as ChatGPT.

However, the vast majority of our clients are focused on where their business ranks in Google’s organic results and whether they appear in

Google AI Overviews. Afterall when Google gobbles up the vast majority of traffic from shoppers, well, it just makes sense to focus on winning at improving your businesses visibility on Google.

Therefore, it stands to reason that it’s essential to make sure, in particular, that Googlebot can do its job and “crawl and index” a page.

Basically, Googlebot is like a builder surveyor: it comes out, has a look, surveys what it needs, then reports back to Google HQ.

Googlebot, well that’s one busy-bee. Googlebot is constantly crawling and indexing, following links and detecting changes that have been made to a page. If you’ve asked someone to give an old blog post a good old re-vamp, well, its highly probable that Googlebot has crawled and indexed it shortly after.

 

Googlebot is therefore interested in the following:

• New pages (New URL’s)- such as when you publish a new blog post
• Changes to existing pages- improvements made, such as updating the content marketing
• Deleted pages- because these need to be cleared from Google’s index (nobody likes seeing a 404 page when you want to buy something).

 

Redirects- why you must get this bit right

301 or 302 redirects sound confusing, technical if you like. However, don’t get hung-up on the technical jargon so much. In a nutshell, you just need to remember this, if you’ve put time and energy into creating a page, don’t just delete it when it becomes, well obsolete.

Don’t therefore treat an important page as you would say a Christmas tree on the 1st of January and lob it out of the front door?

No, instead, just as you should re-plant a Christmas tree, well, you should also work on squeezing the SEO goodness from a page before you delete it. Therefore this is what makes SEO consultants globally cross-eyed, as part of there job is using redirects, to pass the link authority and page authority onto new-shinny pages, without just deleting or lobbing the page out of the door.

Redirects, as the name suggests, point one page to another, just as you would redirect traffic during roadworks. Now, with redirects (301 or 302 redirects) in SEO, you have to get this bit right, as otherwise, well, you could be losing link equity, for one.

Redirects are often used in this scenario: you have a product that sells well, yet the manufacturer has released a new, shiny, upgraded version. So out with the old product and in with the latest product that’s grabbing every shoppers attention; however, more often than not, the business deletes the page. This means that all that DA and page authority built up on that page is lost and deleted.

From an SEO standpoint, deleting quality pages without any permanent redirects in place is madness. The same as deleting pages which have a ton of good quality and powerful links, that’s like saying no to an extra glass of wine when the owner says “it’s on the house”, why would you say no?

Adding structured data

Adding structured data, well, that sounds super confusing, right? Something for the web developer to take care of?

Well, in some circumstances, you might need a web developer and the costs associated with hiring one.

However, for a small business with, say, a WordPress website, adding schema can sometimes mean just clicking a few buttons, installing a plugin, and ensuring all settings are configured correctly.

Honestly this can sometimes be done in minutes, all before your luke warm espresso turns completely cold.

Now, a schema is a form of structured language (head on over to Schema.org for more information), so you can, for example, markup the company’s NAP information with some WordPress plugins. This can make it abundantly clear to crawlers, such as Googlebot, that your business is located on that street in sunny Cardiff when you mark the NAP information up correctly.

NAP information to local SEO is what butter is to bread, the two really to need each other.
So, let us explain what we mean, for the vast majority of searches made online, people want to find a local business. Perhaps a plumber in a hurry or a local pet shop to make their pet poodle look a million dollars.

However, to do this Google needs to know where you are, where the shop is, then marry the two.

How does Google achieve this modern marvel of electronic engineering?

 

Proximity

Proximity is shared by your smartphone buzzing back and fore with Google where you are.

Are you ontop of a remote mountain, or sipping a cocktail in the city centre when you make the search.

Then NAP information- where is the business located, how close is it to Sarah, sipping her old fashioned cocktail as she ponders a local establishment to enjoy pasta dish, why not it’s the weekend.

So, NAP information needs to be made abundantly clear to Google- here’s how you do that:

• NAP information should be on the contact us page and in the footer
• Google Business NAP information should reinforce where your business is
• Then the 100 odd say business citations should then reinforce your companies NAP

How does tie into a technical SEO article we hear you scream?

Simple, if things cant be crawled and indexed, because well, the NAP information is not present on the website, this is technical SEO issue that will become a local SEO issue- now are you with us?

Site structure (why it’s essential to simplify)

We are an agency with a wealth of experience (10+ years!), so we’ve seen e-commerce websites that are just overly complicated (we see this quite often). Some are so complicated you need a flat white to try to get your head around why they have three product categories and 17 odd pages for what is essentially the same blinking product!

 

Keyword cannibalisation

From an SEO perspective, these are just pages competing with each other; “keyword cannibalisation” is therefore at work. Too much of this is going on, and from an SEO standpoint, you’re just creating a bit of a giant mess, to be honest.

Regarding site structure, keep things super streamlined and straightforward.
Sure, you might have a complicated product range, with, say, thousands of different products to choose from?

However, no matter how complex things become, make sure you hire the right freelancer or agency to advise you on how to streamline and apply white-hat SEO best practices to your company website.

Here’s our best advice for streamlining and improving site structure:

Set out each main service on its own page

Group products logically- such as all refrigeration products under one category that then splits out

Make it so, that the customer can get to right product or service with the absolute minimal amount of clicks

 

Because you have to consider the following:

Keyword cannibalisation- you don’t want multiple pages competing for the exact keywords

Crawl budget- you’re wasting your crawl budget if you have a lot of “content-thin” pages, for example. A lot of businesses have this problem, that’s either a mass of duplicated pages or just content-thin pages.

Poor U.X- another consideration is, your mass of pages, all with say duplicated text, or just poor site structure, means you’re offering a poor

U.X- that means “user experience” for those that do not know.
What does crawl budget even mean?

So, we mentioned earlier a bit about crawl budgets, that’s with good reason. Even though Google has some of the most powerful processing power on the planet, that doesn’t mean Google wants to waste its time repeatedly crawling and indexing the same duplicated, content-thin, or spammy pages! That’s just a waste of everyone’s time, Googlebots included.

Therefore, it’s always a good idea to chat with your marketing agency about adding a robots.txt file to prevent bots, such as Googlebot, from wasting time crawling non-essential business pages. For example, you don’t want your customers to use your WordPress login page so that it can be no-indexed by your agency and added to the robot.txt.

 

Do add an XML sitemap

An XML sitemap is essential.

Again, don’t think it’s super complicated or complex to implement, because, well, it’s not (if you have a WordPress website).

Again, if you have a WordPress website, well, millions of businesses use this CMS and the free Yoast SEO plugin.

There is a paid version of Yoast, but even the free version lets you generate an XML sitemap and you don’t have to even spend a penny of your hard earned dough!

Googlebot and other bots then use the XML sitemap to better navigate your website and crawl and index the pages listed in the XML list. For example, to understand all the blog posts and pages you have added since you created the website, Googlebot often follows the XML sitemap to ensure it crawls and indexes all the right pages.

Site Speed- is it still important?

So, in short, the answer is yes.

Nobody likes waiting around, whether that’s in a queue at the supermarket or waiting for an e-commerce website to load whilst your on train ride home.

Therefore, “core web vitals”, which is a rather complicated way of saying how fast the website becomes usable and able for a shopper to navigate, is essential for your SEO team to improve.

Google Core Web Vitals scores therefore need to be improved. Its all well and good having website that has all the bells and whistles, but if its bloated, to the point its slow and cumbersome this hold your SEO back.

Therefore, most businesses want the best website design they can afford. Most companies know that today they need to stand out from the crowd and opt for an impressive, memorable web design.

For example, have you every looked for local business, thought all these websites look the same!

However, the web design must balance speed with offering the customer a good user experience (UX) and deliver a good overall design. This means you need to make the website fast and intuitive, so people don’t get confused about where to find the relevant products or services on the main menu that they need.

If a shopper does start to get confused, have that expression a bit like a rabbit caught in headlights when trying to navigate your e-commerce website, well, it’s a sign you’ve made the website too complicated.

The main menu therefore may need redesigning, or could be technical SEO glitches such as for example the links not working.

Monitoring links, especially the do-follow links in particular is important.

Okay, some die-hard SEO consultants might say that link building isn’t part of technical SEO, but hear us out for a second.
Sometimes, web designers, the businesses’ copywriters and U.X people, well, sometimes they all move to their own merry tune.

 

Meaning they do what they want!

An old page might get deleted, because let’s say the content marketing team want to delete some of their older stock to make way for new products.

However, they might have deleted a page with a important backlink that was a game-changer for the business. A link that is now broken, causing a technical SEO issue, plus meaning that the all important link juice is not being sent to the linked page.

So, for example, let’s say a local business was ranked number one in their city.

Well, part of that could be due to a BBC link leading to a blog page because, say, the employee won an award.

If they obtained a link from the BBC, the link juice would be highly valuable to that company and its SEO.

Therefore, if it were deleted, well, this would majorly hamper SEO efforts if Googlebot suddenly noticed the link was removed or broken. So, this is why we would argue that part of the technical SEO responsibilities is to monitor broken links and lost backlinks.

Because if that page gets deleted, you could lose a vital backlink.

 

How important is minimising hosting downtime?

It’s essential to minimise how often your website goes offline.

For example, you will want to minimise the time pages display a “404-Page Not Found” error to the customer. This is bad from U.X point of view, but also from SEO perspective as well.

This is the same with Googlebot: if it tries to crawl and index your website too often and it’s offline, it will simply deem the website unreliable and offer a poor user experience (U.X).

If the website is deemed unreliable because it’s offline so often, it will offer a poor customer experience.

This means that Google’s algorithm is likely to rank the website much lower in its index.

 

What does “content thin” pages mean, and why do these need to be identified?

Okay, so this might not be so much of a technical SEO issue as an on-page/content marketing issue.

However, where it spills over into a technical SEO issue is simple: if you’re wasting a lot of your crawl budget getting, say over 300+ spammy blog posts indexed, it’s a waste of time.

It’s going to waste your business’s crawl budget, also, all that rubbish on-page will hold your SEO back, a bit like competing in a marathon, with 70 mph headwinds! Not fun!

 

Which CMS is best for improving a website’s technical SEO?

It truly doesn’t matter which CMS you choose; that’s a content management system, as long as you follow white hat principles.

Also, have a good read of Google’s own advice on its blog and Google Webmaster Guidelines.

It just makes sense that if you want your business to appear at the top of Google, whether that’s organic or AI Overviews you need to understand Google’s advice on how to achieve this.

However, if you’re a small local business, we would encourage you to consider having a WordPress website designed.

As WordPress has a built in Dashboard that anyone can use and make website edits- it makes using the CMS a piece of cake. Therefore, you don’t need to be on the phone with your web developers to ask for simple changes, then get struck by a massive bill!

 

Why do you strongly advocate avoiding one-page websites like the plague?

One-page websites are quite simply difficult and a bit of a nightmare to optimise if your business offers multiple different services.

Take, for example, the relatively simple business model of an estate agent; they will often have residential sales, commercial sales, and a rental team. Now, to optimise all these different terms (keywords) on one page, well, that’s going to be hard work.

Therefore, if you were to ask our SEO agency, we would say, most definitely opt for a multi-page brochure website (WordPress is a good option) rather than opting for a single-page CMS if possible.

 

Why should I monitor in Google Search Console a rise in “crawled but not indexed pages”?

It’s possible to keep writing new blog posts and pages until you are exhausted and blue in the face!

However, this doesn’t mean those pages will appear in Google’s results (Google’s Index) if the quality isn’t good enough.

That’s because, well, Google decides if the page is indexed or not.

Now, therefore, it’s perfectly possible to write, say, 100 pages and for none of them to get indexed by Google.

This is why the best SEO agencies in the world will always tell you that, when it comes to links and content marketing, it’s far better to place your efforts into producing quality work and not quantity.

 

Why does U.X matter so much?

When it comes to technical SEO, you can optimise a website to load faster, thereby improving the UX. However, web designers, UX designers, as well as the technical SEO boffins should all work together to make sure that the website looks brilliant- but also that it’s super sonic fast.

 

Why is it essential to have each leading service on its own page?

From a technical SEO perspective, it’s much better to have an XML sitemap where each leading service and product has its own page.
This means they can have stronger on-page SEO, which means you optimise each page much better.

 

Can we help:

SEO takes up a considerable amount of time- this blog post itself took most of the day!

Therefore, you’re going to want to partner with a company that can take care of all this work for you. This is why so many businesses flock to us like seagulls to a fallen tray of chips!

We know our stuff, we can take care of your business’s technical, on-page and off-page SEO.