URL Parser

Understanding URL Anatomy

Every URL follows a structured format defined by RFC 3986. The general syntax is scheme://userinfo@host:port/path?query#fragment. Each component serves a specific purpose. The scheme identifies the protocol, the authority section (userinfo, host, port) identifies the server, the path locates the resource, the query string passes parameters, and the fragment points to a specific section within the resource. Understanding this structure is essential for web development, API design, and debugging network issues.

When browsers request a web page, they parse the URL to determine which protocol to use, which server to contact, and which resource to request. Servers then use the path and query string to determine what content to return. Fragments are handled entirely by the browser and are never sent to the server, making them useful for single-page application routing and in-page navigation.

Percent-Encoding and Reserved Characters

URLs can only contain a specific set of ASCII characters. Characters outside this set, as well as reserved characters used as data rather than delimiters, must be percent-encoded. The encoding replaces each unsafe character with a percent sign followed by two hexadecimal digits representing its byte value. For example, a space becomes %20, an ampersand becomes %26, and a plus sign becomes %2B. Reserved characters include : / ? # [ ] @ ! $ & ' ( ) * + , ; =. Unreserved characters include letters, digits, hyphens, periods, underscores, and tildes, and these never need encoding.

How Query Strings Work

Query strings begin after the question mark character and consist of key-value pairs separated by ampersands. Each pair uses the format key=value. Multiple values for the same key are allowed and common in web forms with checkboxes or multi-select fields. Query strings are limited in length by browsers and servers. Most web servers accept query strings up to 2,048 characters, and some browsers enforce limits around 2,083 characters for the entire URL. For large data transfers, POST requests with a request body are preferred over long query strings.

Relative vs Absolute URLs

An absolute URL includes the full scheme and authority, such as https://example.com/page. A relative URL omits parts of the URL and is resolved against a base URL. For example, /about is a path-relative URL resolved against the current host, while ../images/logo.png is a directory-relative URL resolved against the current path. Protocol-relative URLs like //cdn.example.com/script.js inherit the current page's scheme. While once common, protocol-relative URLs are now discouraged in favor of explicit https:// references.

Common URL Schemes

The most widely used schemes are http and https for web content, with HTTPS being the secure, encrypted variant. Other common schemes include ftp for file transfers, mailto for email addresses, tel for phone numbers, ssh for secure shell connections, and data for inline data embedding. Each scheme has its own rules for the rest of the URL format. For example, mailto:user@example.com has no authority section, while data:text/plain;base64,SGVsbG8= encodes data directly in the URL.

Frequently Asked Questions

What are the components of a URL?

A URL has a scheme (protocol), optional userinfo, host, port, path, query string (key-value pairs after ?), and fragment (section after #). Not all components are required.

What is the difference between a URI and a URL?

A URI identifies a resource. A URL is a URI that also specifies how to access it via a scheme like http. All URLs are URIs, but not all URIs are URLs.

What is percent-encoding?

Percent-encoding replaces unsafe characters with a % followed by two hex digits. Spaces become %20, ampersands become %26. This prevents breaking URL parsing.

How do query strings work?

Query strings follow the ? in a URL as key=value pairs separated by &. They pass parameters to the server. Example: ?search=cats&page=2.

What are common URL schemes?

The most common are http, https, ftp, mailto, tel, ssh, and data. Each tells the client which protocol to use for accessing the resource.