Develop AI Solutions Using Azure Data Management Services

Cosmos DB for NoSQL is a common backing store for AI applications because it combines flexible, schema-free JSON documents with low, predictable latency at any scale — and, increasingly, native vector search alongside the rest of an item's data.

Connecting and querying

• The SDK (Python, .NET, Java, JS) connects via a `CosmosClient` constructed from an endpoint and key, or — the recommended production pattern — Microsoft Entra ID with a managed identity, avoiding a long-lived key in configuration.
• Every operation is scoped to a container within a database; a container's *partition key* determines how items are distributed across physical partitions and is the single most consequential design decision for both cost and query performance.
• Queries that don't filter on the partition key become *cross-partition* queries — still possible, but more expensive in Request Units (RUs) than a query scoped to one logical partition.

Request Units and consistency levels

• Every read and write consumes RUs, a currency abstracting CPU, memory, and IO cost. A point read (by id and partition key) is the cheapest possible operation; a large cross-partition query with `ORDER BY` costs substantially more.
• Consistency level is a deliberate trade-off along a spectrum: *Strong* (always read the latest committed write, highest latency/cost) → *Bounded staleness* → *Session* (the default — a client always sees its own writes) → *Consistent prefix* → *Eventual* (lowest latency/cost, no ordering guarantee). Most AI applications default to Session consistency and only tighten it where correctness genuinely requires it.
• Indexing policy controls which paths are indexed; by default every property is indexed automatically, which is convenient but not free — excluding paths you never query on reduces both storage and write RU cost.

Common confusion

• RU consumption is billed the same whether a query returns zero results or many — a query's *cost* is driven by how much data it has to scan and process, not by how much it returns, which is why a well-chosen partition key and indexing policy matter even for queries with small result sets.

Cosmos DB's native vector search lets you store an item's embedding alongside its source data and metadata in the same document, avoiding the operational overhead of syncing a separate vector store.

Vector storage and search

• A container's vector policy declares which path holds the embedding, its dimensionality, and the distance function (cosine, dot product, or Euclidean). A vector *indexing* policy then determines how that path is indexed for search — separate from, but coordinated with, the vector policy.
• Three vector index types trade off accuracy against cost: `flat` is a brute-force, 100%-accurate scan capped at 505 dimensions; `quantizedFlat` compresses vectors for lower RU cost and latency at a small accuracy loss, still brute-force; `diskANN` builds an approximate-nearest-neighbor graph (Microsoft Research's DiskANN algorithm) that scales to millions of vectors with the lowest latency and RU cost, at the cost of being approximate rather than exact.
• A search is expressed with the `VectorDistance` system function in a normal NoSQL query, for example: `SELECT TOP 10 c.title, VectorDistance(c.contentVector, @queryVector) AS score FROM c ORDER BY VectorDistance(c.contentVector, @queryVector)`. Always including a `TOP N` clause matters — without it, the engine tries to rank and return far more candidates than any application needs, at unnecessary RU cost.
• Because `diskANN` and `quantizedFlat` are approximate, the exact same query can return slightly different ordering across runs once your recall-vs-cost tuning favors speed over exhaustiveness — that's expected behavior, not a bug to chase.

Change feed processor

• The change feed is a persistent, ordered log of every insert and update to a container (deletes aren't included by default). It's the mechanism for reacting to new or changed items — for example, generating an embedding the moment a new document lands and writing it back to the same item.
• The change feed *processor* (available in the .NET and Java SDKs; Python and Node.js use the lower-level pull model instead) distributes leases — one per logical partition range — across however many compute instances are running, and checkpoints progress per lease so processing can resume without reprocessing everything after a restart.
• Processing guarantees are *at-least-once*: if your handler throws partway through a batch, the same batch is retried from the last checkpoint, so handler logic needs to be idempotent (safe to run twice on the same data).

Common confusion

• The change feed only reflects changes going forward from when a processor's lease container was first initialized (or a specific configured start time) — it isn't a way to replay history from before that unless you explicitly enable an all-versions-and-deletes read mode or start from the beginning of the container's lifetime.

Azure Database for PostgreSQL (Flexible Server) gives AI applications a relational option — useful when data already has a natural tabular shape, or when the app needs both structured querying and vector search over the same rows via the `pgvector` extension.

Schema and data types

• Choosing precise column types (`numeric` vs. `float`, `text` vs. `varchar(n)`, native `jsonb` for semi-structured fields) affects both storage footprint and query performance — `jsonb` in particular lets you keep flexible metadata columns alongside strict relational columns without a full NoSQL migration.
• Standard indexing (B-tree for equality/range lookups, GIN for `jsonb` or full-text) applies exactly as in any PostgreSQL deployment; vector similarity search needs its own, separate index type (covered next).

Connection optimization

• Each PostgreSQL connection is a real backend process, materially more expensive to open than a typical NoSQL connection — an AI application issuing many short-lived requests (common in serverless or Functions-triggered workloads) can exhaust available connections quickly without pooling.
• PgBouncer (available as a built-in connection-pooling option on Flexible Server) sits in front of the database and multiplexes many client connections onto a smaller pool of actual backend connections, which is usually the single highest-leverage fix for throughput and latency problems caused by connection churn.
• SDKs should also be configured with sensible client-side pool sizes and timeouts rather than opening a fresh connection per request.

Common confusion

• Adding more compute (vCores/memory) to a PostgreSQL server doesn't fix a connection-exhaustion problem by itself — the number of usable connections is bounded by the `max_connections` setting and by memory per connection, so pooling is often a bigger lever than scaling up.

The `pgvector` extension adds a native `vector` column type and similarity operators to PostgreSQL, letting embeddings live next to the row they describe instead of in a separate system.

Distance operators and access methods

• `<=>` computes cosine distance, `<->` computes L2 (Euclidean) distance, and `<#>` computes negative inner product — the query must use the operator matching the index's configured operator class (for example `vector_cosine_ops` for `<=>`) or the planner won't use the index at all.
• Without any index, `pgvector` performs an exact, brute-force nearest-neighbor scan — perfectly accurate, but linear in the number of rows.

Choosing an index type

• IVFFlat partitions vectors into *lists* built during a one-time training pass over existing data; a query only searches the closest `probes` lists. Fast to build and memory-light, but needs data present before indexing for good list boundaries, and has a lower speed/recall ceiling than graph-based indexes.
• HNSW builds a multi-layer navigable-graph structure. It has no training step (so it can be built on an empty table, unlike IVFFlat) and generally gives a better speed-vs-recall trade-off, at the cost of longer build time and higher memory use. Its two build-time knobs are `m` (connections per layer) and `ef_construction` (candidate list size during build); `ef_search` tunes the accuracy/speed trade-off per query or connection.
• DiskANN is designed to stay fast when the index doesn't fit in memory, since it's optimized for data resident on SSD — the right choice as embedding volume grows past what HNSW can comfortably hold in RAM.
• `pgvector` indexes are capped at 2,000 dimensions; higher-dimensional embeddings must be reduced in dimensionality or left unindexed (falling back to a brute-force scan, or partitioning/sharding for scale).

RAG with metadata filtering

• A retrieval-augmented generation pattern typically combines a vector similarity ORDER BY with a normal `WHERE` clause on metadata columns (tenant id, document category, date range) — letting you scope semantic search to only the rows a given user or use case should ever see, in the same query as the similarity ranking.
• Sizing compute, memory, and storage for a vector workload isn't just about row count: index build time and query latency both scale with vector dimensionality and the chosen index's memory footprint, so a workload with large embeddings needs headroom beyond what row count alone would suggest.

Common confusion

• IVFFlat's `lists` parameter is chosen relative to the number of rows *at index build time* — adding a large amount of new data afterward without rebuilding the index degrades recall, since the list boundaries no longer reflect the data's actual distribution. HNSW doesn't have this problem since it has no training phase.

Azure Managed Redis gives AI applications a very low-latency layer for two related but distinct jobs: general-purpose caching, and — via the RediSearch module — vector similarity search close to the application.

Caching, expiration, and invalidation

• Standard Redis data operations (`SET`/`GET`, hashes, sorted sets) work as an application-level cache in front of a slower backing store — for example, caching a Cosmos DB or PostgreSQL read, or caching an LLM response for a repeated prompt (semantic caching).
• A time-to-live (`EXPIRE`, or `SET ... EX`) on a key evicts it automatically, which is the simplest invalidation strategy for data that's naturally time-bounded (a session token, a rate-limit counter).
• For data that changes on a schedule you control rather than a fixed TTL, explicit invalidation (deleting or overwriting the key when the source data changes) keeps the cache from serving stale results indefinitely.

Vector search with RediSearch

• Vector search requires the RediSearch module, which must be enabled *when the Azure Managed Redis instance is created* — modules can't be added to an existing instance afterward, so this is a provisioning-time decision, not something you can bolt on later.
• Not every tier supports it: Memory Optimized, Balanced, and Compute Optimized tiers support RediSearch; Flash Optimized does not.
• Vectors are stored in hash or JSON structures alongside metadata (document id, source, tenant), and indexed with either a `FLAT` (exact, brute-force) or `HNSW` (approximate, graph-based) index, searched via K-nearest-neighbor or range queries, and combinable with metadata filters for hybrid search — the same core pattern as Cosmos DB and pgvector, just backed by Redis's in-memory performance profile.
• Because vector search shares the instance with caching, session storage, and rate limiting, Azure Managed Redis is often chosen less for raw vector scale and more for keeping a very-low-latency retrieval step (like a semantic cache or a real-time recommendation lookup) physically close to other hot application data.

Common confusion

• Redis vector search trades some capability for speed — features like Cosmos DB's DiskANN-scale approximate search over huge datasets aren't the point of Redis; it's optimized for low-latency lookups over data that comfortably fits the tier's memory profile, not for being the system of record for a large embedding corpus.