What is a User Agent?

A user agent acts between a web server and the person using the web: it makes the request, retrieves the content, and displays it. Browsers like Mozilla Firefox, Google Chrome, and Safari are the most common examples, but crawlers, download managers, screen readers, link checkers, and the browsers built into game consoles are user agents too.

A user agent typically identifies itself with a User-Agent string, a line of text in the HTTP request header that names the application and, less precisely than it once did, its version and operating system. The header is optional and self-reported, but servers still read it to tell browsers from bots in access logs, adapt what they serve, and block misbehaving clients.

What a user-agent string looks like

Here's an example of a reduced user-agent string from Chrome 143 on Android, taken from MDN's User-Agent header reference:

Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Mobile Safari/537.36

The pattern is a product/version pair followed by comments in parentheses. Reading left to right:

• Mozilla/5.0: a general compatibility token. For historical reasons, almost every browser opens its string with it, which is why user-agent strings look so alike.
• (Linux; Android 10; K): system information. In Chrome's reduced string these are fixed values, not the visitor's real Android version or device model.
• AppleWebKit/537.36 (KHTML, like Gecko): platform tokens naming the rendering engine.
• Chrome/143.0.0.0: the product and its version. The major version is real; the minor numbers are shown as zeros.
• Mobile Safari/537.36: "Mobile" marks a phone-sized device, and the trailing Safari token is another compatibility holdover.

The limits of user-agent data

The string is self-reported: any client can send any value, a practice known as user agent spoofing, or leave the header out entirely. Never base security decisions on the User-Agent header, and never treat it as proof of identity. To confirm a visitor really is Googlebot, check that its IP address matches Google's published crawler ranges instead of trusting the string.

Browsers have also deliberately reduced what the string reveals. To limit fingerprinting, Chrome froze its minor version numbers at 0.0.0 in June 2022 (Chrome 101), trimmed desktop operating system details by February 2023 (Chrome 107), and replaced the real Android version and device model with the fixed "Android 10; K" by May 2023 (Chrome 110), per the Chromium User-Agent reduction plan. Servers that need more detail are expected to request it through User-Agent Client Hints, which the browser answers with Sec-CH-UA-* headers. Treat the string as a rough label for analytics, not a precise device profile.

User agents and SEO

Search engines identify their crawlers with user agents of their own, and that's your control surface. In a robots.txt file, the User-agent line names the crawler each group of rules applies to, and User-agent: * matches every crawler that honors the file. This rule blocks Google's main crawler from one directory:

User-agent: Googlebot
Disallow: /private/

Googlebot is the token that controls crawling for Google Search; Google's common crawler list documents the tokens and full strings for the rest, like Googlebot-Image and Storebot-Google.

Your server's access logs can show the user-agent value each request presents, which is how you see what search engines are doing on your site. Because the string can be faked, verify that requests claiming to be Googlebot come from Google's published crawler IP ranges before you attribute them to Google. If verified Googlebot visits are going to pages that don't matter, that's a crawl budget problem you can spot in the logs and fix with robots.txt rules.

One thing not to do: don't serve crawler user agents different content from what human visitors see. Google's spam policies call that cloaking, and sites that use it can rank lower or disappear from results entirely. MDN gives the same warning about serving different content to different browsers. Check your logs, control crawlers in robots.txt, and show every user agent the same page.