ForgeLearn
|$ curl https://forge-ai.dev/api/markdown?path=docs/python/httpx-requests
$cat docs/httpx-&-requests.md
updated Today·18-24 min read·published

httpx & requests

◆Python◆HTTP◆httpx◆Intermediate🎯Free Tools
Introduction

requests is the classic sync HTTP client. httpx adds HTTP/2, timeouts by default patterns, and first-class async. For new code — especially async — prefer httpx.

requests Basics
requests_basic.py
Python
1import requests
2
3r = requests.get(
4 "https://httpbin.org/get",
5 params={"q": "python"},
6 timeout=5,
7)
8r.raise_for_status()
9print(r.status_code, r.json())
10
11r = requests.post(
12 "https://httpbin.org/post",
13 json={"name": "Ada"},
14 headers={"X-Trace": "1"},
15 timeout=(3.05, 27), # connect, read
16)
17print(r.json()["json"])
✕

danger

Never call requests without timeout= — it can hang forever.
Sessions
sessions.py
Python
1import requests
2
3with requests.Session() as s:
4 s.headers.update({"User-Agent": "forgelearn/1.0"})
5 s.get("https://httpbin.org/cookies/set/session/1", timeout=5)
6 r = s.get("https://httpbin.org/cookies", timeout=5)
7 print(r.json())
httpx Sync
httpx_sync.py
Python
1import httpx
2
3with httpx.Client(timeout=5.0, follow_redirects=True) as client:
4 r = client.get("https://httpbin.org/get", params={"q": "1"})
5 r.raise_for_status()
6 print(r.json())
7
8 r = client.post("https://httpbin.org/post", json={"ok": True})
9 print(r.status_code)
Async httpx
httpx_async.py
Python
1import asyncio
2import httpx
3
4async def fetch_many(urls: list[str]) -> list[int]:
5 async with httpx.AsyncClient(timeout=5.0) as client:
6 async def one(url: str) -> int:
7 r = await client.get(url)
8 return r.status_code
9 return await asyncio.gather(*(one(u) for u in urls))
10
11async def main():
12 codes = await fetch_many([
13 "https://httpbin.org/get",
14 "https://httpbin.org/status/204",
15 ])
16 print(codes)
17
18# asyncio.run(main())
Auth
auth.py
Python
1import httpx
2from httpx import BasicAuth
3
4with httpx.Client(timeout=5.0) as client:
5 r = client.get(
6 "https://httpbin.org/basic-auth/user/pass",
7 auth=BasicAuth("user", "pass"),
8 )
9 print(r.status_code)
10
11 r = client.get(
12 "https://httpbin.org/bearer",
13 headers={"Authorization": "Bearer TOKEN"},
14 )
Timeouts & Errors
timeouts.py
Python
1import httpx
2
3timeout = httpx.Timeout(5.0, connect=2.0)
4try:
5 with httpx.Client(timeout=timeout) as client:
6 client.get("https://httpbin.org/delay/10")
7except httpx.TimeoutException as e:
8 print("timeout", e)
9except httpx.HTTPStatusError as e:
10 print("status", e.response.status_code)
11except httpx.RequestError as e:
12 print("network", e)
LibraryTimeout defaultAsync
requestsNone (hangs!)No (use async libs)
httpx5s default on Client if setAsyncClient
Streaming
stream.py
Python
1import httpx
2from pathlib import Path
3
4with httpx.stream("GET", "https://httpbin.org/stream/3", timeout=10.0) as r:
5 r.raise_for_status()
6 for line in r.iter_lines():
7 if line:
8 print(line)
9
10# download to file
11with httpx.stream("GET", "https://httpbin.org/bytes/1024", timeout=10.0) as r:
12 with Path("blob.bin").open("wb") as f:
13 for chunk in r.iter_bytes():
14 f.write(chunk)
requests vs httpx
Featurerequestshttpx
Sync APIYesYes
Async APINoYes
HTTP/2NoOptional
WSGI transport (tests)LimitedASGI/WSGI helpers
TimeoutsMust setFirst-class Timeout
Retries (pattern)
retries.py
Python
1import httpx
2import time
3
4def get_with_retries(url: str, attempts: int = 3) -> httpx.Response:
5 last: Exception | None = None
6 for i in range(attempts):
7 try:
8 with httpx.Client(timeout=5.0) as client:
9 r = client.get(url)
10 if r.status_code in {502, 503, 504}:
11 raise httpx.HTTPStatusError("retryable", request=r.request, response=r)
12 return r
13 except (httpx.TransportError, httpx.HTTPStatusError) as e:
14 last = e
15 time.sleep(0.2 * 2**i)
16 assert last is not None
17 raise last
🔥

pro tip

For production, prefer tenacity or urllib3 Retry mounted carefully — and idempotent methods only.
Testing FastAPI with httpx
asgi_transport.py
Python
1import httpx
2from httpx import ASGITransport
3# from app import app
4
5# transport = ASGITransport(app=app)
6# async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
7# r = await client.get("/health")
8# assert r.status_code == 200
Proxies & HTTP/2
proxy.py
Python
1import httpx
2
3# proxies
4# with httpx.Client(proxy="http://localhost:8080", timeout=5.0) as client:
5# ...
6
7# HTTP/2 requires h2 package: pip install httpx[http2]
8# with httpx.Client(http2=True, timeout=5.0) as client:
9# print(client.get("https://example.com").http_version)
Cookies & Headers
headers.py
Python
1import httpx
2
3headers = {"Accept": "application/json", "X-Request-Id": "abc"}
4with httpx.Client(headers=headers, timeout=5.0) as client:
5 r = client.get("https://httpbin.org/headers")
6 print(r.json()["headers"].get("X-Request-Id"))
7 client.cookies.set("session", "1")
8 print(client.get("https://httpbin.org/cookies").json())
JSON Encoding Gotchas
  • requests/httpx encode dict bodies with json= parameter
  • Do not pass data=dict if you meant JSON — that becomes form encoding
  • Check Content-Type on responses before .json()
  • Raise for status before trusting body
json_gotcha.py
Python
1import httpx
2
3with httpx.Client(timeout=5.0) as client:
4 # JSON body
5 client.post("https://httpbin.org/post", json={"a": 1})
6 # form body
7 client.post("https://httpbin.org/post", data={"a": "1"})
Multipart Uploads
multipart.py
Python
1import httpx
2from pathlib import Path
3
4path = Path("file.txt")
5path.write_text("hello", encoding="utf-8")
6with httpx.Client(timeout=30.0) as client:
7 files = {"file": (path.name, path.read_bytes(), "text/plain")}
8 # r = client.post("https://httpbin.org/post", files=files)
9 # print(r.status_code)
Client Checklist
  • Always set timeouts
  • Reuse Client/AsyncClient (connection pooling)
  • raise_for_status() or check status
  • Prefer json= over manual dumps
  • Use secrets for tokens; never log Authorization
← PreviousDjango
Next →SQLAlchemy
🎯

35+ Free Developer Tools

AI Chat, Mermaid, Excalidraw, JSON Editor, API Tester & more

→
$Blueprint — Engineering Documentation·Section ID: PYTHON-HTTPX·Revision: 1.0

Community

Get help on Slack, Discord or VIP

Stuck on a guide? Join the community and ask.

Join SlackJoin DiscordKaundal VIP
Follow