The User-Agent string format is currently specified by section 10.1.5 of HTTP Semantics. The User-Agent string is one of the criteria by which Web crawlers may be excluded from accessing certain parts of a website using the Robots Exclusion Standard ( robots.txt file).Īs with many other HTTP request headers, the information in the 'User-Agent' string contributes to the information that the client sends to the server, since the string can vary considerably from user to user.

The concept of content tailoring is built into the HTTP standard in RFC 1945 'for the sake of tailoring responses to avoid particular user agent limitations'. For example, the User-Agent string might be used by a web server to choose variants based on the known capabilities of a particular version of client software. In HTTP, the User-Agent string is often used for content negotiation, where the origin server selects suitable content or operating parameters for the response. Bots, such as Web crawlers, often also include a URL and/or e-mail address so that the Webmaster can contact the operator of the bot. In HTTP, SIP, and NNTP protocols, this identification is transmitted in a header field User-Agent. When a software agent operates in a network protocol, it often identifies itself, its application type, operating system, device model, software vendor, or software revision, by submitting a characteristic identification string to its operating peer. 2.1 Format for human-operated web browsers.

Firefox mobile user agent