Why real-time personalization matters for e‑commerce
Imagine a shopper landing on a fashion site and instantly seeing a curated list of dresses that match their recent clicks, weather forecast, and even the tone of the product description they just read. According to a 2023 McKinsey report, retailers that deliver hyper‑personalized experiences see conversion rates up to 30% higher than generic sites. The challenge is delivering that relevance within milliseconds, before the user decides to abandon the page.
Core components of a modern recommendation stack
The engine combines three proven technologies: large language models (LLMs) for semantic understanding, vector search for fast similarity matching, and serverless edge functions to run the logic close to the user. LLMs translate product titles, reviews, and user queries into dense embeddings; vector databases (FAISS, Pinecone, or Vespa) index those embeddings for sub‑second nearest‑neighbor lookups; edge runtimes (Cloudflare Workers, AWS Lambda@Edge) execute the recommendation pipeline at the network edge, cutting latency to under 50 ms for most regions.
Preparing data and generating embeddings
Start with a clean product catalog: SKU, title, description, category, price, and any user‑generated content such as reviews. Export it to CSV or Parquet, then run a batch job that calls an LLM API (e.g., OpenAI’s text‑embedding‑ada‑002) to create a 1536‑dimensional vector for each textual field. Store the vectors alongside the original record ID.
import csv, os, requests, json
API_KEY = os.getenv("OPENAI_API_KEY")
ENDPOINT = "https://api.openai.com/v1/embeddings"
def embed(text):
payload = {"model": "text-embedding-ada-002", "input": text}
headers = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}
response = requests.post(ENDPOINT, headers=headers, data=json.dumps(payload))
return response.json()["data"][0]["embedding"]
with open("products.csv") as f:
reader = csv.DictReader(f)
vectors = []
for row in reader:
text = f"{row["title"]} {row["description"]}"
vec = embed(text)
vectors.append({"id": row["sku"], "vector": vec, "price": row["price"]})
# Save vectors for the next step
with open("vectors.json", "w") as out:
json.dump(vectors, out)
Running this script on a 10k‑product catalog typically takes under an hour and costs less than $10 on the pay‑as‑you‑go API.
Creating a fast vector index with FAISS
FAISS (Facebook AI Similarity Search) offers an in‑memory index that can be serialized and loaded by the edge function. For a 10k‑item catalog, an IndexFlatIP (inner product) fits in 200 MB and answers a 10‑nearest‑neighbor query in under 2 ms on a modest CPU.
import faiss, json, numpy as np
with open("vectors.json") as f:
data = json.load(f)
ids = np.array([int(item["id"]) for item in data])
embeddings = np.vstack([np.array(item["vector"], dtype=np.float32) for item in data])
index = faiss.IndexFlatIP(embeddings.shape[1])
index.add(embeddings)
faiss.write_index(index, "product.index")
# Persist mapping for later lookup
np.save("ids.npy", ids)
The resulting product.index file can be uploaded to a CDN and cached at edge locations.
Adding LLM‑driven contextual ranking
Pure similarity may surface items that match keywords but ignore the shopper’s intent. By sending the top‑k vectors to an LLM with a short prompt, you can re‑rank results based on nuance such as “eco‑friendly fabrics” or “budget‑friendly alternatives”. The LLM returns a JSON list of SKU IDs ordered by relevance.
def rerank(sku_list, user_query):
prompt = f"User is looking for: {user_query}. Rank these product IDs by suitability: {sku_list}. Return JSON array of IDs."
response = requests.post(
"https://api.openai.com/v1/chat/completions",
headers={"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"},
json={"model": "gpt-4o-mini", "messages": [{"role": "user", "content": prompt}]}
)
return json.loads(response.json()["choices"][0]["message"]["content"])
This extra step adds 10‑20 ms, still well within the edge latency budget.
Deploying the pipeline as a serverless edge function
Cloudflare Workers lets you bundle the FAISS index (converted to a binary blob) and the reranking logic into a single script. The worker receives a GET request with query and optional userId, computes the query embedding, performs a nearest‑neighbor lookup, calls the LLM for re‑ranking, and returns a JSON payload of product recommendations.
addEventListener('fetch', event => {
event.respondWith(handleRequest(event.request))
})
async function handleRequest(request) {
const url = new URL(request.url)
const query = url.searchParams.get('query') || ''
if (!query) return new Response('Missing query', {status:400})
// 1. Embed the query using OpenAI (cached token)
const embedRes = await fetch('https://api.openai.com/v1/embeddings', {
method: 'POST',
headers: { 'Authorization': `Bearer ${OPENAI_TOKEN}`, 'Content-Type': 'application/json' },
body: JSON.stringify({model: 'text-embedding-ada-002', input: query})
})
const embedData = await embedRes.json()
const queryVec = embedData.data[0].embedding
// 2. Perform vector search (FAISS compiled to WebAssembly)
const topK = await FAISS.search(queryVec, 10) // returns array of SKU IDs
// 3. Re‑rank with LLM
const rerankRes = await fetch('https://api.openai.com/v1/chat/completions', {
method: 'POST',
headers: { 'Authorization': `Bearer ${OPENAI_TOKEN}`, 'Content-Type': 'application/json' },
body: JSON.stringify({model: 'gpt-4o-mini', messages: [{role: 'user', content: `User wants: ${query}. Rank these IDs: ${JSON.stringify(topK)}.`}]})
})
const rerankJson = await rerankRes.json()
const ranked = JSON.parse(rerankJson.choices[0].message.content)
return new Response(JSON.stringify({recommendations: ranked}), {headers: {'Content-Type':'application/json'}})
}
Deploy with wrangler publish, enable KV caching for recent queries, and you have a globally distributed recommendation endpoint that answers under 50 ms for most users.
Monitoring performance and continuous improvement
Instrument the edge function with metrics: request latency, LLM token usage, and conversion uplift per recommendation slot. Tools like Cloudflare Analytics or Datadog can alert you if latency spikes above 80 ms. Periodically retrain embeddings with fresh product data and incorporate user‑behavior signals (click‑through, dwell time) into the reranking prompt to keep relevance high.
Key takeaways
By marrying LLM‑generated embeddings, a lightweight FAISS index, and serverless edge execution, you can deliver AI‑driven product recommendations that are both contextually aware and ultra‑fast. The architecture scales from a few hundred SKUs to millions with minimal operational overhead, and the pay‑as‑you‑go model keeps costs predictable.
Sources
OpenAI API Documentation; FAISS Official GitHub Repository; Cloudflare Workers Documentation
Author: Mahmut Sarıkaya — sarikayadev.com
