Elastic (@elastic)

Binary quantization sounds like it should tank recall. BBQ in Elasticsearch doesn't due to its asymmetric nature. Vectors compress to single-bit values. Queries stay at int4 precision, so distance calculations keep the detail that matters. You trade a bit of oversampling and reranking for an approximately 95% storage reduction. Trade-off: queries cost slightly more to compute per comparison. Storage costs don't move at all.

15-minute live demos, Q+A. Every week. Starting tomorrow. Relevance Please is a new weekly livestream: demos across Search, Observability, and Security with rotating hosts and rotating topics. First up: @_jphwang on Making Video Search Easy. Join us tomorrow, 11AM ET / 8AM PT / 4PM BST. Links in the reply below.

We're introducing Elasticsearch Columnar Mode: A new index mode that stores data once, in columnar form, with no redundant copies and no indexes the workload doesn't need. Not replacing the document model. Adding a second way to organise data alongside it, for the workloads where columnar is the right shape: logs, telemetry, metrics, security events, AI retrieval. One platform for search and analytics at the same level, on the same data. Tech Preview in 9.5, GA in 9.6.

CPUs don't have vector search instructions. You borrow from neural nets and video codecs instead. Elasticsearch's simdvec engine reformulates vector math to fit whatever the CPU already runs fast. Four recent examples: - int7 quantization: fit unsigned-only multiply-accumulate by trading 1 bit of precision. ~6x faster. - int8 bias rewrite: algebraic shift + precomputed correction. ~20% bulk gain. - bf16 Euclidean distance as 3 dot products instead of a float32 conversion. Up to 2.4x. - Binary dot product via popcount: AND the bits, count the 1s. ~4x over scalar. A compiler can't make these calls. Each one requires reformulating the problem to fit an instruction the hardware was never designed to use this way.

You've got 10k slides, scans, and screenshots to search. Your first instinct is to throw a VLM at it. But a VLM reads one image at a time. Running it across your whole corpus on every query doesn’t scale. Split the pipeline instead. jina-clip embeds every image into a vector once. At query time, the same model embeds your question and does a similarity lookup. Jina-VLM only touches the matched slides: a handful of images, not thousands. Build the index once, then retrieval is instant. The VLM only reasons where it matters.

Most search benchmarks only tell half the story. You test relevance. You ship it. Then p99 latency tanks under real concurrency and users start filing tickets. Or you optimize for speed, and your top-k results are fast garbage. The fix: measure both sides every time. 10 metrics that matter, mapped out.

3 patterns for multimodal RAG. Here's how they differ and when each one breaks down. Most RAG systems add multimodal support by converting everything to text first. Is your system natively multimodal, or just a conversion pipeline? The architecture choice shapes what you can query and what you lose. Shared vector space - Cross-modal search without format conversion - Requires large multimodal training datasets - Semantic drift is a real risk if training data is narrow Single grounded modality - Works with any existing text search setup - Spatial relationships in images don't survive conversion - Retrieval quality depends on captioning/transcription accuracy Separate retrieval pipelines - Best per-modality retrieval accuracy - Most complex to rank across modalities - Highest compute cost, independent search per modality Pick your pattern, clone the repo, and build it.

0.89 recall at k=10 and zero cross-tenant leaks on a persistent agent memory layer. Built on Elasticsearch with 3 indices mapped to cognitive science: episodic events, semantic facts, procedural playbooks. Each has its own write rate, aging rules, and update logic. Episodic decays. Semantic gets superseded when a user contradicts it (the old fact stays for audit, a filter hides it from recall). Procedural tracks success and failure counts across conversations. Recall is one hybrid query: BM25 + Jina v5 dense fused with RRF, then a cross-encoder reranker on the merged candidates. Time decay and use-count scoring keep fresh, frequently-recalled facts on top. DLS scopes every query to the user's API key. The cluster won't return another user's documents regardless of what the agent asks for. Open source, with an MCP endpoint for any agent runtime.

3 types of mappings in Elasticsearch Dynamic: Elasticsearch detects field types as documents arrive. Explicit: you define every field upfront. Recommended for production. Runtime: schema-on-read, no reindexing needed. Each trades setup speed for indexing control.

Learn how to cut Elasticsearch log storage by up to 76% with LogsDB: 1. Create a LogsDB index with "index.mode": "logsdb" 2. Reindex your logs into both a standard and LogsDB index 3. Force merge both indices with _forcemerge?max_num_segments=1 4. Measure the difference with the _stats API In our test: 15.37 MB (standard) vs 8.6 MB (LogsDB). 44% reduction on test data. 76% in production benchmarks.

Here are 5 distance metrics in vector search. But how do you choose the right one? • L1 (Manhattan): sum of absolute differences, exact kNN only with no HNSW support • L2 (Euclidean): straight-line distance, the safe default for most models • Cosine similarity: angle between vectors, magnitude ignored • Dot product: same ranking as cosine on normalized vectors, less compute • Max inner product: dot product without the normalization constraint Most teams default to cosine and move on. That works until your model outputs non-normalized vectors, and suddenly dot product or max inner product is the better fit. Scoring formulas and config details in the blog.

Building a dashboard mid-investigation means losing the thread you were pulling. Open the editor, pick indices, configure panels, wire up ES|QL queries. That's ten minutes of context switching before you've answered a single question. Kibana skips that now. • Describe what you need in plain language. • The agent explores your indices, generates ES|QL, picks chart types, and lays out the panels. Everything stays in your conversation until you save it. Then it becomes a first-class Kibana object your team can open and edit. Already viewing one? The agent attaches automatically. Ask why a metric spiked, add a comparison panel, break it down by region: all inline. Available as a technical preview in Elastic 9.4