Scramjet | Proxy

Pitfall: The Scramjet Proxy uses too much CPU. Solution: Reduce maxParallel and add a setTimeout delay of 10ms inside the map function to prevent event loop starvation.

Pitfall: Proxies die mid-stream. Solution: Implement a .filter() that checks for HTTP error codes and re-routes dead proxies to a .catch() stream that removes them from the active list.

Pitfall: Memory leak with large HTML responses. Solution: Use Scramjet’s StringStream and .split() to process the response chunk by chunk rather than storing the entire HTML string.

A proxy shouldn't just pass data; it should add value. Scramjet Proxy allows you to manipulate data streams on the fly. Because it integrates with the Scramjet Transform Hub (STH), you can inject small, executable scripts directly into the proxy layer. scramjet proxy

Imagine this scenario:

Checking Google rankings for 100,000 keywords requires SERP scraping. With a Scramjet Proxy, you can integrate with proxy APIs (like BrightData or Oxylabs) and use Scramjet’s map function to enrich the stream with rank data before saving to PostgreSQL.

We'll use eXpress Data Path (XDP) + BPF for kernel‑bypass speed. Pitfall: The Scramjet Proxy uses too much CPU

To understand the Scramjet Proxy, we must first appreciate the metaphor.

In short: A Scramjet Proxy is a lightweight, event-driven, zero-copy proxying engine optimized for hypersonic data rates and sub-millisecond latency.

Engineers and architects designing low-latency network systems, edge compute middleware, CDN/edge proxy stacks, and teams evaluating high-throughput proxy designs or implementing protocol adapters for event-driven platforms. In short: A Scramjet Proxy is a lightweight,

If you are self-hosting the Scramjet Hub and need to configure the actual Proxy server (which sits in front of the Hub), you typically use Nginx or Traefik.

Here is a sample Nginx configuration for a Scramjet Hub setup:

server 
    listen 80;
    server_name scramjet.yourdomain.com;
location / 
    # Forward all traffic to the Scramjet Hub
    proxy_pass http://localhost:9000; # Default Scramjet Hub API port
# Standard Proxy Headers
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support (crucial for streaming data)
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";

Note: The Scramjet Hub handles the internal routing to the containers; Nginx just passes the traffic to the Hub.