Self-Hosted Models
Listed inSelf-Hosted ModelsModels & Providerson
Running weights yourself: GPUs, quantization, batching, and what it really costs.
Advanced9 minDraft
#deployment
Page 2 of 11
Listed inSelf-Hosted ModelsModels & Providerson
Running weights yourself: GPUs, quantization, batching, and what it really costs.
Listed inSecurity and Privacy ConcernsSafety & Ethicson
The industry checklist: injection, insecure output handling, supply chain, data leakage, and the rest — with mitigations.
Listed inSupabase VectorVector Databaseson
pgvector on Postgres — vectors sitting next to your relational data.
Listed inLLM ObservabilityObservabilityon
Chip Huyen assembles the whole reference architecture step by step — context, guardrails, routing, caching, observability.
Listed inWhat is a Context Layer?Context Engineeringon
The service that assembles, ranks, and budgets context for every call.
Listed inPrompt CachingHow LLMs Workon
How cache breakpoints work, what invalidates a prefix, and the latency and cost numbers to expect.
Listed inTracing & LoggingObservabilityon
Spans across retrieval, tool calls, and generations in one trace.
Listed inEmbedding Models on Hugging FaceEmbeddingson
Benchmark scores across retrieval, clustering, and reranking tasks. Useful for a shortlist, dangerous as a final answer.
Listed inSampling ParametersHow LLMs Workon
How decoding settings turn one distribution into many possible responses.
LangChain's concept doc on the splitter family — what each one optimises for and when to reach for it.
Listed inHugging Face Inference SDKModels & Providerson
Calling hosted or local models through a single client.
Listed inIndexing EmbeddingsVector Databaseson
Qdrant's explainer on HNSW, payload indexes, and the quantization trade-offs behind recall and memory.
Listed inStoring in a Vector DBRAGon
Index layout, namespaces, and re-indexing without downtime.
Listed inPrompt EngineeringPrompt Engineeringon
A dense, well-cited tour of zero-shot, few-shot, chain-of-thought, and automatic prompt search.
Listed inContext EvaluationContext Engineeringon
Measuring whether the assembled context contained the answer at all.
Listed inRagasEvaluationon
Faithfulness, answer relevance, and context precision — how the metrics are computed and what they miss.
Listed inPostHogObservabilityon
Product analytics alongside LLM traces, so quality ties back to behaviour.
Listed inLangfuseObservabilityon
Open-source tracing, prompt management, and evaluation — including the data model behind a trace.
Listed inStreaming ResponsesHow LLMs Workon
Token-by-token delivery, and the UX and error handling it forces you to design for.
Listed inSentence TransformersEmbeddingson
The library most self-hosted embedding pipelines are built on: pooling, training, and inference.