```yaml
product: AlterLab
title: "Character Encoding — Web Scraping Glossary | AlterLab"
comparison_context: "AlterLab is an alternative to Firecrawl, ScrapingBee, and Bright Data."
source_url: https://alterlab.io/glossary/encoding
```

protocol

# Character Encoding

Character encoding defines how text characters are represented as bytes; mismatched encoding between server and parser causes garbled (mojibake) text extraction.

Every web page's text must be encoded as bytes for transmission. The encoding specifies which byte sequence represents each character. UTF-8 is the dominant encoding on the modern web (covering all Unicode characters) but legacy sites still use ISO-8859-1 (Latin-1), Windows-1252, or region-specific encodings (GBK for Chinese, Shift-JIS for Japanese).

The encoding can be declared in three places: the HTTP `Content-Type` header (`Content-Type: text/html; charset=utf-8`), the HTML `<meta charset='utf-8'>` tag, or the XML declaration. Scrapers must detect and apply the correct encoding before parsing text; mismatched encoding produces mojibake — garbled sequences like `é` instead of `é`.

Python's `chardet` and `charset-normalizer` libraries can detect encoding heuristically when it is not declared. `requests` library applies the `apparent_encoding` detected by chardet when the server does not specify a charset. BeautifulSoup also handles encoding detection internally when given raw bytes.

### What is Character Encoding?

Character encoding defines how text characters are represented as bytes; mismatched encoding between server and parser causes garbled (mojibake) text extraction.

### How does AlterLab handle Character Encoding?

Python's `chardet` and `charset-normalizer` libraries can detect encoding heuristically when it is not declared. `requests` library applies the `apparent_encoding` detected by chardet when the server does not specify a charset. BeautifulSoup also handles encoding detection internally when given raw bytes.

## Examples

```
import requests

response = requests.get("https://example.com")
# requests uses encoding from Content-Type header by default
# Override if detection is wrong:
response.encoding = "utf-8"
text = response.text  # correctly decoded string

# Or let chardet detect:
response.encoding = response.apparent_encoding
text = response.text
```

## Related Terms

[HTMLHTML (HyperText Markup Language) is the standard markup language for structuring web page content, defining elements like headings, paragraphs, links, images, and tables.](https://alterlab.io/glossary/html)[HTTP HeadersKey-value metadata fields sent with HTTP requests and responses that communicate authentication, content type, caching, and other directives.](https://alterlab.io/glossary/http-headers)[HTTPHTTP (HyperText Transfer Protocol) is the application-layer protocol used by web browsers and scrapers to request and receive resources from web servers.](https://alterlab.io/glossary/http)[Data ExtractionThe process of identifying and pulling specific fields from raw web content, converting unstructured HTML into typed JSON records.](https://alterlab.io/glossary/data-extraction)[Data NormalizationData normalization standardises scraped values into a consistent format — stripping currency symbols, trimming whitespace, converting dates, and unifying units.](https://alterlab.io/glossary/normalization)

## Extract Character Encoding data from any website

AlterLab returns clean, structured data from any public URL — no scraper infrastructure needed. Start free, no credit card required.

[View API docs](https://alterlab.io/docs/api/rest)

[Back to Glossary](https://alterlab.io/glossary)